Hi Arvid, In my case it's the latter, thus I have also thought about using the filter (map is not useful in my case).
What I am not sure which is better to be used? In what case would you split a stream with side output and in what case with filter? Would there be any performance gain/pain based on which is used? Regards , Nikola <%28%2B45%29%2060%2054%2032%2016> On Mon, May 10, 2021 at 6:00 PM Arvid Heise <ar...@apache.org> wrote: > Hi Nikola, > > if you just want to apply a different user function to the records > depending on the property "exist" the simplest way is to use > > source -> map(if exist do this else that) -> sink > > If it turns out that you want to apply a different subgraph, you can do > > source -> filter(if exist) -> do this -> union -> sink > source -> filter(if not exist) -> do that -^ > > On Mon, May 10, 2021 at 3:07 PM Nikola Hrusov <n.hru...@gmail.com> wrote: > >> Hi, >> >> I am trying to find some information on what is the best way to split a >> stream of the same data. >> >> For the given scenario: I have an object which has a property "exist" >> >> I want to split the stream based on this property, do something, and >> afterwards join it again into a single stream. >> >> Initial (A) -> Split stream based on exist (B) or not (C) -> union both >> streams (D) >> >> I could find some similar topics on StackOverflow: >> - >> https://stackoverflow.com/questions/53588554/apache-flink-using-filter-or-split-to-split-a-stream >> - >> https://stackoverflow.com/questions/61752728/how-to-get-output-of-the-values-that-are-not-matched-in-filter-function-in-apach >> >> but none of them really gives a definitive answer. >> >> What I am thinking about is using 1) filter or 2) side output. >> >> I know that one of the use cases of side output is that it can have >> different data types. That is not my case as it will be the same object >> going through the whole pipeline. >> >> So both options look more or less the same to me, however I do not know >> the flink internals as good as I would like to as of this point. >> >> Can some of you guys shed some light and perhaps tell me if I am mistaken >> in my thoughts? >> >> Thanks. >> >> Regards >> , >> Nikola >> >
