Re: How to debug why Flink makes and executes only partial plan

2016-10-19 Thread Aljoscha Krettek
Hi, are we talking about the Plan View in the JobManager dashboard? If yes, then I expect there to be only one "box" for the combination of filter and sink because they are chained together to avoid sending data. For debugging, could you maybe change check() to always return true and see if you th

How to debug why Flink makes and executes only partial plan

2016-10-14 Thread Satish Chandra Gupta
Hi, In my Flink program, after a couple of map, union and connect, I have a final filter and a sink. Something like this (after abstracting out details): val filteredEvents: DataStream[NotificationEvent] = allThisStuffWorking .name("filtered_users") filteredEvents *.filter(x => check(x