Re: A "per operator instance" window all ?

Hi Xingcan, Ken and Till, OK, thank you. It is clear. I have various option then: * the one suggested by Ken where I can find a way to build a key that will be well distributed (1 key per task) o it relies on the way Flink partitions the key, but it will do the job * or I can

Re: A "per operator instance" window all ?

Hi Julien, you could use the OperatorState to cache the data in a window and the last time your window fired. Then you check the ctx.timerService().currentProcessingTime() in

Re: A "per operator instance" window all ?

Hi Julien, at the moment Flink only supports parallel windows which are keyed. What you would need is something like a per-partition window which is currently not supported. The problem with that is that it is not clear how to rescale a per-partition window because it effectively means that you

Re: A "per operator instance" window all ?

Hi Julien, I'd run into a similar situation, where I need to have a keyed stream, but I want (effectively) one key per task. It’s possible to generate keys that will get distributed as you need, though it does require making assumptions about how Flink generates hashes/key groups. And once

Re: A "per operator instance" window all ?

Hello, I've already tried to key my stream with "resourceId.hashCode%parallelism" (with parallelism of 4 in my example). So all my keys will be either 0,1, 2 or 3. I can then benefit from a time window on this keyed stream and do only 4 queries to my external system. But it is not well

Re: A "per operator instance" window all ?

Hi Julien, If I am not misunderstand, I think you can key your stream on a `Random.nextInt() % parallesm`, this way you can "group" together alerts from different and benefit from multi parallems. 发自网易邮箱大师 On 02/19/2018 09:08，Xingcan Cui wrote： Hi Julien, sorry

Re: A "per operator instance" window all ?

Hi Julien, sorry for my misunderstanding before. For now, the window can only be defined on a KeyedStream or an ordinary DataStream but with parallelism = 1. I’d like to provide three options for your scenario. 1. If your external data is static and can be fit into the memory, you can use