Hey, I would also strongly prefer the second option, users need to have the option to force cancel a program in case of something unwanted behaviour.
Cheers, Gyula Matthias J. Sax <mj...@informatik.hu-berlin.de> ezt írta (időpont: 2015. máj. 27., Sze, 1:20): > Hi, > > currently, the only way to stop a streaming job is to "cancel" the job, > This has multiple disadvantage: > 1) a "clean" stopping is not possible (see > https://issues.apache.org/jira/browse/FLINK-1929 -- I think a clean stop > is a pre-requirement for FLINK-1929) and > 2) as a minor issue, all canceled jobs are listed as canceled in the > history (what is somewhat confusing for the user -- at least it was for > me when I started to work with Flink Streaming). > > This issue was raised a few times already, however, no final conclusion > was there (if I remember correctly). I could not find a JIRA for it either. > > From my understanding of the system, there would be two ways to > implement a nice way for stopping streaming jobs: > > 1) "Task"s can be distinguished between "batch" and "streaming" > -> canceling a batch jobs works as always > -> canceling a streaming job only send a "canceling" signal to the > sources, and waits until the job finishes (ie, sources stop emitting > data and finish regularly, triggering the finishing of all operators). > For this case, streaming jobs are stopped in a "clean way" (as is the > input would have be finite) and the job will be listed as "finished" in > the history regularly. > > This approach has the advantage, that it should be simpler to > implement. However, the disadvantages are (1) a "hard canceling" of jobs > is not possible any more, and (2) Flink must be able to distinguishes > batch and streaming jobs (I don't think Flink runtime can distinguish > both right now?) > > 2) A new message "terminate" (or similar) is introduced, that can only > be used for streaming jobs (would be ignored for batch jobs) that stops > the sources and waits until the job finishes regularly. > > This approach has the advantage, that current system behavior is > preserved (it only adds a few feature). The disadvantage is, that all > clients need to be touched and it must be clear to the user, that > "terminate" does not work for streaming jobs. If an error/warning should > be raised if a user tries to "terminate" a batch job, Flink must be able > to distinguish between batch and streaming jobs, too. As an > alternative, "terminate" on batch jobs could be interpreted as "cancel", > too. > > > I personally think, that the second approach is better. Please give > feedback. If we can get to a conclusion how to implement it, I would > like to work on it. > > > -Matthias > >
