Re: Clarification in TumblingProcessing TimeWindow Documentation

Sure thanks a lot! Bowen and Fabian. I will create the JIRA and also submit a PR with a better comment. -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me > On May 28, 2018, at

Clarification in TumblingProcessing TimeWindow Documentation

/api/windowing/assigners/TumblingProcessingTimeWindows.java#L92>) and not for starting the tumbling window every 10 seconds. Thanks ------ Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me

Re: Replaying logs with microsecond delay

Yes, thanks! -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me > On May 15, 2018, at 21:31, Xingcan Cui wrote: > > Yes, that makes sense and maybe you could also generat

Re: Replaying logs with microsecond delay

as is, in the form of event timestamps, only proves the algorithms from a theoretical/simulation perspective. Hope this answers your question to some extent at least. Let me know. Thanks! -- Dhruv Kumar PhD Candidate Department of Computer Science

Re: Replaying logs with microsecond delay

Thanks a lot, Ted. Appreciate your help! The approaches specified in the below links, are giving a very good level of accuracy. Solves my problem for now. Thanks -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering

Replaying logs with microsecond delay

same arrival delay between the records. Let me know if I am not clear above. Thanks -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me

Re: Signal for End of Stream

Fabian, Thanks a lot for your continuous help! Really appreciate it. Sent from Phone. > On May 8, 2018, at 03:06, Fabian Hueske wrote: > > Hi Dhruv, > > The changes look good to me. > > Best, Fabian > > 2018-05-08 5:37 GMT+02:00 Dhruv Kumar : >> Thank

Re: Signal for End of Stream

} } } // collect trailing data if (buffer.length() > 0) { ctx.collect(buffer.toString()); } } ``` -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering Uni

Re: Signal for End of Stream

I notice that there is some DeserializationSchema in org.apache.flink.api.common.serialization which has a function isEndOfStream but I am not sure if I can use it in my use case. -- Dhruv Kumar PhD Candidate Department of Computer Science and

Signal for End of Stream

finish its execution? Is there any information which I can add to the records while they are sent from the replayer program and which can be parsed when the records arrive inside the Flink program? Let me know if anything is not clear. Thanks -- Dhruv

Re: Apache Flink Examples

Thanks. Tests and the example folder will help. -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me > On Apr 27, 2018, at 06:47, Hung wrote: > > in my case I usually

Re: Measure End-to-End latency/delay for each record

Ok thanks Michael for all your help! -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me > On Apr 26, 2018, at 19:24, TechnoMage wrote: > > Yes, Kafka for source and s

Re: Measure End-to-End latency/delay for each record

Ok that answers my questions. What are you keeping the source and sink as? Is it Kafka for both? -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me > On Apr 26, 2018, at 16

Apache Flink Examples

may be absolutely wrong. Please correct me if I am. Thanks. -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me

Re: Measure End-to-End latency/delay for each record

pressure on source and how will it impact the latency calculations? Sorry if these are trivial questions. I am a bit new to the real world streaming systems. -- Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of

Measure End-to-End latency/delay for each record

when all the records have been written into the sink. Does this sound ok? Also, if I use Processing time characteristic for this end-to-end-latency, will it be fine? Thanks -- Dhruv Kumar PhD Candidate Department of Computer Science and

Re: Custom Processing per window

me time. > > Best, Fabian > > 2018-03-19 14:47 GMT+01:00 Dhruv Kumar <mailto:gargdhru...@gmail.com>>: > In other words, while using the Flink streaming APIs, is it possible to take > a decision on emitting a particular key based on the state of some other k

Re: Custom Processing per window

In other words, while using the Flink streaming APIs, is it possible to take a decision on emitting a particular key based on the state of some other key present in the same window? Thanks! -- Dhruv Kumar PhD Candidate Department of Computer

Re: Custom Processing per window

CandidateDepartment of Computer Science and EngineeringUniversity of Minnesotawww.dhruvkumar.me On Mar 19, 2018, at 02:04, Jörn Franke <jornfra...@gmail.com> wrote:How would you start implementing it? Where are you stuck?Did you already try to implement this?On 18. Mar 2018, at 04:10, Dhruv

Custom Processing per window

using Flink. Any help would be greatly appreciated. Dhruv Kumar PhD Candidate Department of Computer Science and Engineering University of Minnesota www.dhruvkumar.me

Emulate Tumbling window in Event Time Space

Hi I was trying to emulate tumbling window in event time space. Here is the link to my code. I am using the process function to do the custom processing which I want to do within every window. I am having an issue of how to emit results at the end of every window since my watermark only gets em