In my case I have more elaborate logic to select data from the streams. They are not all the same logical type, though I may be able to represent them as the same Java type. My main question is whether it is technically feasible to have a single operator that takes multiple streams as input. For example Operator(stream1, stream2, stream3) and produces an output stream. Can the checkpointing and other logic accomodate this if I write sufficient custom code in the operator?
Michael > On Apr 7, 2018, at 10:42 AM, Ken Krugler <kkrugler_li...@transpac.com> wrote: > > When you say “join” are you talking about a real join (so one or more fields > can be used as a joining key), or some other operation? > > For more than two streams, you can do cascading window joins via multiple > join()s that reduce your source streams down to a single stream. > > If the fields are the same across these streams, then a union() followed by > say a ProcessFunction that implements your joining logic could work. > > Or you can convert all the streams to a common tuple format that consists of > a unions the fields, so you can do a union() and then follow that with > whatever logic is needed to actually do the join. > > Though I’m sure there are more elegant approaches :) > > — Ken > > > >> On Apr 6, 2018, at 5:04 PM, Michael Latta <mla...@technomage.com> wrote: >> >> I would like to “join” several streams (>3) in a custom operator. Is this >> feasible in Flink? >> >> >> Michael > > -------------------------------------------- > http://about.me/kkrugler > +1 530-210-6378 >
