Re: APEXCORE-294 Clean application shutdown
I have updated the documentation for kill v/s shutdown. let me know if anything more is required. - Tushar. On Mon, Jan 23, 2017 at 6:09 PM, Tushar Gosaviwrote: > Hi Thomas, > > I have updated the pull request as well as design document, will take > care of apex documentation shortly. > > - Tushar. > > > On Fri, Jan 20, 2017 at 8:57 PM, Thomas Weise wrote: >> As discussed please also look at appropriate user documentation and clarity >> WRT the CLI commands. >> >> Thanks, >> Thomas >> >> >> On Thu, Jan 19, 2017 at 10:46 PM, Tushar Gosavi >> wrote: >> >>> Hi Thomas, >>> >>> As discussed I will remote the old behaviour of shutdown. >>> >>> - Tushar. >>> >>> On Fri, Jan 20, 2017 at 10:03 AM, Tushar Gosavi >>> wrote: >>> > Hi Thomas, >>> > >>> > I agree with you on the current behavior. The effect of shutdown can >>> > be achieve with kill. I was more worried about backward compatibility >>> > issue >>> > while relaunching the shutdown app, hence kept the default as same as >>> > before (cli and rest api default). If we don't want to retain the >>> > relaunch >>> > behaviour of shutdown app then the code will be more simpler :). Let >>> > me know, I will update the PR accordinlgy. >>> > >>> > - Tushar. >>> > >>> > >>> > On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise wrote: >>> >> Tushar, >>> >> >>> >> I would like to see the use case for "hard shutdown" vs kill. The >>> current >>> >> shutdown behavior isn't useful, confusing and sometimes does not even >>> end >>> >> the app (based on feedback received in the past from users). I would >>> prefer >>> >> we don't retain it without clear understanding of use case and how it >>> will >>> >> really work. >>> >> >>> >> Thanks, >>> >> Thomas >>> >> >>> >> >>> >> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi >>> >> wrote: >>> >> >>> >>> Dear Community, >>> >>> >>> >>> I have changed the implementaion plan based on PR comments. Following >>> >>> is the new proposal for this feature. >>> >>> >>> >>> https://docs.google.com/a/datatorrent.com/document/d/ >>> 1hSLH4xi_15OWwW4KY7-- >>> >>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing >>> >>> >>> >>> Please provide the feedback. I will update the PR accrodinly. >>> >>> >>> >>> - Tushar. >>> >>> >>> >>> >>> >>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi >> > >>> >>> wrote: >>> >>> > Dear Community, >>> >>> > >>> >>> > I have open an pull request for shutting down application by sending >>> >>> > END_STREAM control tuples from all input operator. This is similar to >>> >>> > all input operator have stopped after raising ShutdownException. >>> >>> > >>> >>> > On receiving shutdown request, master will prepare OPERATOR_STOP >>> >>> > command for all input partitions, and send it to container as part of >>> >>> > heartbeat response to container. Container will shutdown the operator >>> >>> > thread after receiving this command. >>> >>> > >>> >>> > https://github.com/apache/apex-core/pull/424 >>> >>> > Please review. >>> >>> > >>> >>> > Thanks, >>> >>> > - Tushar. >>> >>> >>>
Re: APEXCORE-294 Clean application shutdown
As discussed please also look at appropriate user documentation and clarity WRT the CLI commands. Thanks, Thomas On Thu, Jan 19, 2017 at 10:46 PM, Tushar Gosaviwrote: > Hi Thomas, > > As discussed I will remote the old behaviour of shutdown. > > - Tushar. > > On Fri, Jan 20, 2017 at 10:03 AM, Tushar Gosavi > wrote: > > Hi Thomas, > > > > I agree with you on the current behavior. The effect of shutdown can > > be achieve with kill. I was more worried about backward compatibility > > issue > > while relaunching the shutdown app, hence kept the default as same as > > before (cli and rest api default). If we don't want to retain the > > relaunch > > behaviour of shutdown app then the code will be more simpler :). Let > > me know, I will update the PR accordinlgy. > > > > - Tushar. > > > > > > On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise wrote: > >> Tushar, > >> > >> I would like to see the use case for "hard shutdown" vs kill. The > current > >> shutdown behavior isn't useful, confusing and sometimes does not even > end > >> the app (based on feedback received in the past from users). I would > prefer > >> we don't retain it without clear understanding of use case and how it > will > >> really work. > >> > >> Thanks, > >> Thomas > >> > >> > >> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi > >> wrote: > >> > >>> Dear Community, > >>> > >>> I have changed the implementaion plan based on PR comments. Following > >>> is the new proposal for this feature. > >>> > >>> https://docs.google.com/a/datatorrent.com/document/d/ > 1hSLH4xi_15OWwW4KY7-- > >>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing > >>> > >>> Please provide the feedback. I will update the PR accrodinly. > >>> > >>> - Tushar. > >>> > >>> > >>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi > > >>> wrote: > >>> > Dear Community, > >>> > > >>> > I have open an pull request for shutting down application by sending > >>> > END_STREAM control tuples from all input operator. This is similar to > >>> > all input operator have stopped after raising ShutdownException. > >>> > > >>> > On receiving shutdown request, master will prepare OPERATOR_STOP > >>> > command for all input partitions, and send it to container as part of > >>> > heartbeat response to container. Container will shutdown the operator > >>> > thread after receiving this command. > >>> > > >>> > https://github.com/apache/apex-core/pull/424 > >>> > Please review. > >>> > > >>> > Thanks, > >>> > - Tushar. > >>> >
Re: APEXCORE-294 Clean application shutdown
Hi Thomas, As discussed I will remote the old behaviour of shutdown. - Tushar. On Fri, Jan 20, 2017 at 10:03 AM, Tushar Gosaviwrote: > Hi Thomas, > > I agree with you on the current behavior. The effect of shutdown can > be achieve with kill. I was more worried about backward compatibility > issue > while relaunching the shutdown app, hence kept the default as same as > before (cli and rest api default). If we don't want to retain the > relaunch > behaviour of shutdown app then the code will be more simpler :). Let > me know, I will update the PR accordinlgy. > > - Tushar. > > > On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weise wrote: >> Tushar, >> >> I would like to see the use case for "hard shutdown" vs kill. The current >> shutdown behavior isn't useful, confusing and sometimes does not even end >> the app (based on feedback received in the past from users). I would prefer >> we don't retain it without clear understanding of use case and how it will >> really work. >> >> Thanks, >> Thomas >> >> >> On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi >> wrote: >> >>> Dear Community, >>> >>> I have changed the implementaion plan based on PR comments. Following >>> is the new proposal for this feature. >>> >>> https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7-- >>> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing >>> >>> Please provide the feedback. I will update the PR accrodinly. >>> >>> - Tushar. >>> >>> >>> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi >>> wrote: >>> > Dear Community, >>> > >>> > I have open an pull request for shutting down application by sending >>> > END_STREAM control tuples from all input operator. This is similar to >>> > all input operator have stopped after raising ShutdownException. >>> > >>> > On receiving shutdown request, master will prepare OPERATOR_STOP >>> > command for all input partitions, and send it to container as part of >>> > heartbeat response to container. Container will shutdown the operator >>> > thread after receiving this command. >>> > >>> > https://github.com/apache/apex-core/pull/424 >>> > Please review. >>> > >>> > Thanks, >>> > - Tushar. >>>
Re: APEXCORE-294 Clean application shutdown
Hi Thomas, I agree with you on the current behavior. The effect of shutdown can be achieve with kill. I was more worried about backward compatibility issue while relaunching the shutdown app, hence kept the default as same as before (cli and rest api default). If we don't want to retain the relaunch behaviour of shutdown app then the code will be more simpler :). Let me know, I will update the PR accordinlgy. - Tushar. On Thu, Jan 19, 2017 at 9:22 PM, Thomas Weisewrote: > Tushar, > > I would like to see the use case for "hard shutdown" vs kill. The current > shutdown behavior isn't useful, confusing and sometimes does not even end > the app (based on feedback received in the past from users). I would prefer > we don't retain it without clear understanding of use case and how it will > really work. > > Thanks, > Thomas > > > On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosavi > wrote: > >> Dear Community, >> >> I have changed the implementaion plan based on PR comments. Following >> is the new proposal for this feature. >> >> https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7-- >> LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing >> >> Please provide the feedback. I will update the PR accrodinly. >> >> - Tushar. >> >> >> On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi >> wrote: >> > Dear Community, >> > >> > I have open an pull request for shutting down application by sending >> > END_STREAM control tuples from all input operator. This is similar to >> > all input operator have stopped after raising ShutdownException. >> > >> > On receiving shutdown request, master will prepare OPERATOR_STOP >> > command for all input partitions, and send it to container as part of >> > heartbeat response to container. Container will shutdown the operator >> > thread after receiving this command. >> > >> > https://github.com/apache/apex-core/pull/424 >> > Please review. >> > >> > Thanks, >> > - Tushar. >>
Re: APEXCORE-294 Clean application shutdown
Tushar, I would like to see the use case for "hard shutdown" vs kill. The current shutdown behavior isn't useful, confusing and sometimes does not even end the app (based on feedback received in the past from users). I would prefer we don't retain it without clear understanding of use case and how it will really work. Thanks, Thomas On Wed, Jan 18, 2017 at 1:41 AM, Tushar Gosaviwrote: > Dear Community, > > I have changed the implementaion plan based on PR comments. Following > is the new proposal for this feature. > > https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7-- > LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing > > Please provide the feedback. I will update the PR accrodinly. > > - Tushar. > > > On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosavi > wrote: > > Dear Community, > > > > I have open an pull request for shutting down application by sending > > END_STREAM control tuples from all input operator. This is similar to > > all input operator have stopped after raising ShutdownException. > > > > On receiving shutdown request, master will prepare OPERATOR_STOP > > command for all input partitions, and send it to container as part of > > heartbeat response to container. Container will shutdown the operator > > thread after receiving this command. > > > > https://github.com/apache/apex-core/pull/424 > > Please review. > > > > Thanks, > > - Tushar. >
Re: APEXCORE-294 Clean application shutdown
Dear Community, I have changed the implementaion plan based on PR comments. Following is the new proposal for this feature. https://docs.google.com/a/datatorrent.com/document/d/1hSLH4xi_15OWwW4KY7--LU3e2iHHhfthOpATicnC5eE/edit?usp=sharing Please provide the feedback. I will update the PR accrodinly. - Tushar. On Fri, Nov 25, 2016 at 4:40 PM, Tushar Gosaviwrote: > Dear Community, > > I have open an pull request for shutting down application by sending > END_STREAM control tuples from all input operator. This is similar to > all input operator have stopped after raising ShutdownException. > > On receiving shutdown request, master will prepare OPERATOR_STOP > command for all input partitions, and send it to container as part of > heartbeat response to container. Container will shutdown the operator > thread after receiving this command. > > https://github.com/apache/apex-core/pull/424 > Please review. > > Thanks, > - Tushar.
APEXCORE-294 Clean application shutdown
Dear Community, I have open an pull request for shutting down application by sending END_STREAM control tuples from all input operator. This is similar to all input operator have stopped after raising ShutdownException. On receiving shutdown request, master will prepare OPERATOR_STOP command for all input partitions, and send it to container as part of heartbeat response to container. Container will shutdown the operator thread after receiving this command. https://github.com/apache/apex-core/pull/424 Please review. Thanks, - Tushar.