Bhupesh, If your intention is to let processing complete, the sources should terminate the stream (that's already implemented, except that you want to initiate it from another operator). It does not really matter if they all stop at the same streaming window.
If you want to just stop the application from an operator (similar to kill), it needs to be signaled to the master and the master needs to terminate. One way of doing it could be the stats listener. Thomas On Tue, Jan 17, 2017 at 10:30 PM, Bhupesh Chawda <bhup...@datatorrent.com> wrote: > This JIRA is to stop the DAG in a crude manner, based on an error > condition. I think this might also need similar functionality as an error > condition can occur anywhere in the DAG. > > Perhaps we can modify the same JIRA to include a graceful + ungraceful > (kill) shutdown from any operator in the DAG. > > ~ Bhupesh > > On Wed, Jan 18, 2017 at 11:55 AM, Tushar Gosavi <tus...@datatorrent.com> > wrote: > > > I think this would be a great addition for batch use cases or use > > cases were DAG needs to be shutdown after detecting some > > completion/error condition through the operator. We have one Jira > > Opened for such functionality > > https://issues.apache.org/jira/browse/APEXCORE-503. > > > > - Tushar. > > > > > > On Wed, Jan 18, 2017 at 11:45 AM, Bhupesh Chawda > > <bhup...@datatorrent.com> wrote: > > > Hi All, > > > > > > Currently we can shutdown an Apex app in the following ways: > > > 1. Throw ShutdownException() from *all* the input operators > > > 2. Use Apex CLI to shutdown an app using the YARN App Id > > > > > > I think we should have some way of shutting down an application from > > within > > > an operator. It is not always true that the trigger for shutdown is > sent > > by > > > the input operator only. Sometimes, an end condition may be detected by > > > some operator in the DAG which wants the processing to end. Such a > > > shutdown, although triggered from some intermediate operator in the > DAG, > > > should guarantee graceful shut down of the application. > > > > > > Thoughts? > > > > > > ~ Bhupesh > > >
