Re: Parallelism and keyed streams

2018-07-23 Thread Fabian Hueske
scala.Tuple4(a(i+1)._1, a(i+1)._2, v, a(i+1)._4)) > } > Job.LOG.debug("logRatioWindowFunction [" + a.head._1 + ", " + > a.head._2 + " ... " + a.last._2 +"] collected") > } > } > > } > > > On 17 Jul 2018, at 00:1

Re: Parallelism and keyed streams

2018-07-17 Thread Nicholas Walton
>> wrote: > > Is value(index-1) stored in Keyed State, or just a local variable inside the > operator? > > -Original Message- > From: Nicholas Walton [mailto:nwal...@me.com <mailto:nwal...@me.com>] > Sent: Monday, July 16, 2018 1:33 PM > To: user@

RE: Parallelism and keyed streams

2018-07-16 Thread Martin, Nick
Is value(index-1) stored in Keyed State, or just a local variable inside the operator? -Original Message- From: Nicholas Walton [mailto:nwal...@me.com] Sent: Monday, July 16, 2018 1:33 PM To: user@flink.apache.org Subject: Parallelism and keyed streams I have a stream of tuples

Parallelism and keyed streams

2018-07-16 Thread Nicholas Walton
I have a stream of tuples , which I form into a keyedStream using keyBy on channel. I then need to process each channel in parallel. Each parallel stream must be processed in strict sequential order by index to calculate the ratios value(index)/value(index-1). If I set parallelism to 1 all is