Hi, new to Apache Flink. Trying to find some solid input on how best to handle exceptions in streams -- specifically those that should not interrupt the stream.
For example, if an error occurs during deserialization from bytes/Strings to your data-type, in my use-case I would rather queue the data for visual inspection than discard it and filter it out. One way of doing this is to diverge the stream so that good items take one path, while bad items take another. The closest thing I can find in Flink that can achieve this effect is the split operator. The caveat is that split does not also allow for inlined transformations. In other words, the best use of split appears first perform your logic that catches the exception. Then pass the exception into the next stage which uses split to check for an exception and providing names to each piece of the decision, for example "OK" vs "error". Frameworks like RX (Reactive Extensions, eg RxJava) have built in functionality that allows the user to decide if they want to handle exceptions globally or specifically and resume if needed. I was hoping to find similar operations in Flink but so far no luck. At any rate, it would be great to get some feedback to see if I am heading down the good path here, and whether there are any caveats / gotchas to be aware of? Thanks! Mich
