Hi lenduha, > Processing time: We can only use processing time windows & timer. Event time: We are able to use both processing time & event time windows & timers. Processing time is always wall/clock time depending on the host running task manager so independent from the event time?
Yes. > Also Kafka seems to be supporting timestamp/watermark, however is there any source not supporting timestamp/watermark & possible solutions to that like a custom watermark generator? Yes, there are two places in Flink where WatermarkStrategy can be used [1]: 1) directly on sources, like Kafka connector you mentioned. 2) after non-source operation. The first option is preferable, because it allows sources to exploit knowledge about shards/partition/splits in the watermarking logic, which is more accurate. But if sources does not have such ability, you could use second option. Please check document [1] for more details about option1 and option2. Please note, the document[1] is based on version Flink 1.13 and 1.13+. Please check document[2] if you use old version. [1] https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/datastream/event-time/generating_watermarks/ [2] https://ci.apache.org/projects/flink/flink-docs-release-1.12/dev/event_timestamps_watermarks.html Best regards, JING ZHANG Deniz Koçak <lend...@gmail.com> 于2021年6月28日周一 下午5:06写道： > Thanks for the info Jing, just to clarify one point, so when we set > the environment time characteristics to > > Processing time: We can only use processing time windows & timer. > Event time: We are able to use both processing time & event time > windows & timers. Processing time is always wall/clock time depending > on the host running task manager so independent from the event time? > > Also Kafka seems to be supporting timestamp/watermark, however is > there any source not supporting timestamp/watermark & possible > solutions to that like a custom watermark generator? > > Thanks, > > On Mon, Jun 28, 2021 at 8:25 AM JING ZHANG <beyond1...@gmail.com> wrote: > > > > Hi lenduha, > > > > > When set the time characteristics to ProcessingTime via > setStreamTimeCharacteristic(...) call, I cannot see watermarks in the Flink > UI. > > > > The watermark would be swallowed in the case. > > > > > can I use Event Time Timers even if I set the > > time characteristics to ProcessingTime (via > > ctx.timerService().registerEventTimeTimer(...))? > > > > I'm afraid not. The event time Timers could not work after setting > ProcessingTime by setStreamTimeCharacteristic, > > because the watermark should be swallowed in the case, so the event time > timers would not be triggered. > > > > > when I set the time characteristics to Event Time can I use both > > processing time timers & event time timers without any problem? > > > > Event time timers could work, of course. Besides, explicitly using > processing-time windows and timers works in event-time mode. > > Please note that In Flink1.12, the default stream time characteristic > has been changed to EventTime [1]. > > > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.13/release-notes/flink-1.12/#flip-134-batch-execution-for-the-datastream-api > > > > Best regards, > > JING ZHANG > > > > Deniz Koçak <lend...@gmail.com> 于2021年6月28日周一 下午2:43写道： > >> > >> Hi, > >> > >> In environment configuration when set the time characteristics to > >> ProcessingTime via setStreamTimeCharacteristic(...) call, I cannot see > >> watermarks in the Flink UI. I think this is expected, because > >> watermarks disabled in the source (using Kafka as source)? > >> > >> Another point here is, can I use Event Time Timers even if I set the > >> time characteristics to ProcessingTime (via > >> ctx.timerService().registerEventTimeTimer(...))? As far as I > >> understood from the documentation, Event Time Timers needs watermarks > >> which sets the operator time, so I wonder if I can use event time > >> timers in case of ProcessingTime selected in the environment? Also > >> when I set the time characteristics to Event Time can I use both > >> processing time timers & event time timers without any problem? > >> > >> Thanks, >
