Re: Window Function that releases when downstream work is completed

2019-08-21 Thread David Anderson
I'm not sure I fully understand the scenario you envision. Are you saying you want to have some sort of window that batches (and deduplicates) up until a downstream map has finished processing the previous deduplicated batch, and then the window should emit the new batch? If that's what you want,

Window Function that releases when downstream work is completed

2019-08-16 Thread Steven Nelson
Hello! I think I know the answer to this, but I thought I would go ahead and ask. We have a process the emits messages to our stream. These messages can include duplicates based on a certain key ( we'll call it TheKey). Our Flink job reads the messages, keys by TheKey and enters a window