Global window keyBy

Im using trigger and a guid in order to key stream . I have some problem to understand how to clear the window . - FIRE_AND_PURGE in trigger will remove the keyd data only ? if fire and purge is removing all the data then i need to implement it more like this example

Re: Getting Key from keyBy() in ProcessFunction

Hi Ken, thanks for your answer. You're right and I'm doing it already that way. I just hoped that I could avoid the ValueState (I'm using a MapState as well already, which does not store the key) and get the key from the provided Context of the ProcessFunction. This would avoid having the

Re: Getting Key from keyBy() in ProcessFunction

Hi Jürgen, That makes sense to me. Anyone from the Flink team want to comment on (a) if there is a way to get the current key in the timer callback without using an explicit ValueState that’s maintained in the processElement() method, and (b) if not, whether that could be added to the

Checkpoint is not triggering as per configuration

Hi I am new to the flink world, and trying to understand. Currently, I am using Flink 1.3.2 on a small cluster of 4 nodes, I have configured checkpoint directory at HDFS, and run streaming word count example with my own custom input file of 63M entries, I enabled checkpoint every one second

Why does jobmanager running needs slot ?

I find some codes in flink does not make sense to me. Like in some classes below JobMasterGateway.java has a offerSlots method which means Offers the given slots to the job manager. I was wondering why a jobmanager running should need slots ? TaskExecutor.java has a offerSlotsToJobManager

Re: How does BucketingSink generate a SUCCESS file when a directory is finished

Hi , I have tested it. There are some small problems. When checkpoint is finished, the name of the file will change, and the success file will be written before checkpoint. Best, Ben > On 1 Feb 2018, at 8:06 PM, Kien Truong wrote: > > Hi, > > I did not