Re: Window + Reduce produces more than 1 output per window

2017-06-19 Thread Piotr Nowojski
No problem. Make sure that your application didn’t run in the background three times, thus producing 3x the expected output. Piotrek > On Jun 19, 2017, at 5:25 PM, FRANCISCO BORJA ROBLES MARTIN > wrote: > > Hello Piotrek! > > Thanks for answering! Yes I have already changed the "TimeCharacte

Re: Window + Reduce produces more than 1 output per window

2017-06-19 Thread FRANCISCO BORJA ROBLES MARTIN
Hello Piotrek! Thanks for answering! Yes I have already changed the "TimeCharacteristic" to "ProcessingTime". I need it for the ".setWriteTimestampToKafka(true)" option as I use the timestamp in the Kafka consumer who reads this app's output. I have already changed the code a bit for using Ke

Re: Window + Reduce produces more than 1 output per window

2017-06-19 Thread Piotr Nowojski
One more thing, please try to minimize your solution by removing this Union and Odd/Even filters at the beginning and check whether you get the same results. Piotrek > On Jun 19, 2017, at 2:43 PM, Piotr Nowojski wrote: > > Hi, > > It is difficult for me to respond fully to your question. Firs

Re: Window + Reduce produces more than 1 output per window

2017-06-19 Thread Piotr Nowojski
Hi, It is difficult for me to respond fully to your question. First of all it would be really useful if you could strip down your example to a minimal version that shows a problem. Unfortunately I was unable to reproduce your issue. I was getting only one output line per window (as expected). C

Window + Reduce produces more than 1 output per window

2017-06-18 Thread FRANCISCO BORJA ROBLES MARTIN
Hello everybody! First of all, thanks for reading :D I am currently working on my bachelor's final project which is a comparison between Spark Streaming and Flink. Now let's focus on the problem: - THE PROBLEM: the problem is that my program is writing to Kafka more than once every window (