Hi Averell, The watermark of a stream is always the low watermark of all its input streams. If one of the input streams does not have watermarks, Flink does not compute a watermark for the merged stream. If you do not need time-based operations on streams 3 and 4, setting the watermark to MAX_WATERMARK should be a good solution.
Best, Fabian Am Mi., 1. Mai 2019 um 08:50 Uhr schrieb Averell <[email protected]>: > Hi Fabian, Guowei, > > I have some updates: > 1. I added timestamp&watermark extractor on all of my remaining sources (3 > & > 4), and the watermark does propagate to my final operator. > 2. As I could not find a way to set my file sources as IDLE, I tried to > tweak the class ContinuousFileReaderOperator to be always IDLE: > / nextElement = format.nextRecord(nextElement); > if (nextElement != null) { > readerContext.collect(nextElement); > if > (this.format.getFilePaths()[0].getPath().contains("<myPath>")) > readerContext.markAsTemporarilyIdle(); > } else { > / and the result I got was there's no watermark at all for that stream, and > that IDLE status seemed not to be taken into account (my CEP operator > didn't > generate any output). So I do not understand what that IDLE StreamStatus is > for. > My temporary solution, for now, is to use MAX_WATERMARK for those idle > sources. Not sure whether doing that is recommended? > > Thanks for your help. > Regards, > Averell > > > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
