Re: Roadmap for 0.8.0
Hi Jeff, I looked at the PR for ZEPPELIN-1595 <https://issues.apache.org/jira/browse/ZEPPELIN-1595> It does not look it covers %sh interpreter. %sh and %sql interpters are somewhat unique as they don't have access to Zeppelin API (please correct me if I'm wrong) So what https://issues.apache.org/jira/browse/ZEPPELIN-1967 is suggesting, is to introduce syntax that is used in Jupyter notebooks, i.e. {*var1*} will be implied as z.get('var1'), for example: %sh /path/to/script --param8={*var1*} --param9={*var2*} where var1 and var2 would be implied to be fetched as z.get('var1') and z.get('var2') respectively. Or similarly for %sql : %sql create table dwh.table_{*year*} stores as parquet as select * from spark_df1 where year = {*year*} We miss a lot global variables for %sql and %sh so that a Zeppelin note can be used as a single parametrized orchestration for a whole workflow. Thank you, Ruslan Dautkhanov On Wed, Apr 5, 2017 at 12:01 AM, Jeff Zhang <zjf...@gmail.com> wrote: > > Hi Ruslan, > > Regarding 'make zeppelinContext available in shell interpreter', you may > want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595 > > > Ruslan Dautkhanov <dautkha...@gmail.com>于2017年4月3日周一 下午12:05写道: > >> That's exciting to see plans for 0.8.0 on the horizon. >> >> Here's my top list for 0.8 : >> >> - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle >> timeout" >> This is a most-wanted feature by our Zeppelin admins. It was mentioned >> at least once on this email chain. >> >> - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z >> variables to Shell Interpreter" >> We had several of our users asking about this functionality. %sh and >> some other interpreters can't be >> parametrized by global variables. ZEPPELIN-1967 is one way of how this >> can be solved. >> >> - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory >> references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected" >> Less of a critical compared to the above two, but it could complement >> the multi-tenancy feature very well. >> >> >> Best regards, >> Ruslan Dautkhanov >> >> On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <felixcheun...@hotmail.com >> > wrote: >> >> +1 with latest/stable. >> >> >> >> >> -- >> *From:* moon soo Lee <m...@apache.org> >> *Sent:* Tuesday, March 21, 2017 8:41:58 AM >> *To:* us...@zeppelin.apache.org >> *Cc:* dev@zeppelin.apache.org >> >> *Subject:* Re: Roadmap for 0.8.0 >> >> >> And if i suggest simplest way for us to set quality expectation to user, >> which will be labeling release in download page. >> >> Currently releases are divided into 2 categories in download page. >> 'Latest release' and 'Old releases'. I think we can treat 'Latest' as >> unstable and add one more category 'Stable release'. >> >> For example, once 0.8.0 is released, >> >> Latest release : 0.8.0 >> Stable release : 0.7.1 >> Old release : 0.6.2, 0.6.1 >> >> Once we feel confident about the stability of latest release, we can just >> change label from latest to stable in the download page. (and previous >> stable goes to old releases) >> We can even include formal vote for moving release from 'latest' to >> 'stable' in our release process, if it is necessary. >> >> Thanks, >> moon >> >> On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org> wrote: >> >> Yes, having longer RC period will help. >> >> But if i recall 0.7.0 release, although 21 people participated verifying >> through 4 RC for 15days, it wasn't enough to catch all critical problems >> during the release process. After the release, we've got much more number >> of bug reports, in next few days. >> >> Basically, verifying RC is limited to people who subscribe mailing list + >> willing to contribute time to verify RC, which is much smaller number of >> people who download release from download page. So having longer RC period >> will definitely help and i think we should do, but I think it's still not >> enough to make sure the quality, considering past history. >> >> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. >> ASF release process defines how to release source code, but it does not >> really restrict what kind of 'version' the project should have releases. >> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. >> >> Thanks
Re: Roadmap for 0.8.0
Hi Ruslan, Regarding 'make zeppelinContext available in shell interpreter', you may want to check https://issues.apache.org/jira/browse/ZEPPELIN-1595 Ruslan Dautkhanov <dautkha...@gmail.com>于2017年4月3日周一 下午12:05写道: > That's exciting to see plans for 0.8.0 on the horizon. > > Here's my top list for 0.8 : > > - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle > timeout" > This is a most-wanted feature by our Zeppelin admins. It was mentioned at > least once on this email chain. > > - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z > variables to Shell Interpreter" > We had several of our users asking about this functionality. %sh and some > other interpreters can't be > parametrized by global variables. ZEPPELIN-1967 is one way of how this > can be solved. > > - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory > references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected" > Less of a critical compared to the above two, but it could complement > the multi-tenancy feature very well. > > > Best regards, > Ruslan Dautkhanov > > On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <felixcheun...@hotmail.com> > wrote: > > +1 with latest/stable. > > > > > -- > *From:* moon soo Lee <m...@apache.org> > *Sent:* Tuesday, March 21, 2017 8:41:58 AM > *To:* us...@zeppelin.apache.org > *Cc:* dev@zeppelin.apache.org > > *Subject:* Re: Roadmap for 0.8.0 > > > And if i suggest simplest way for us to set quality expectation to user, > which will be labeling release in download page. > > Currently releases are divided into 2 categories in download page. 'Latest > release' and 'Old releases'. I think we can treat 'Latest' as unstable and > add one more category 'Stable release'. > > For example, once 0.8.0 is released, > > Latest release : 0.8.0 > Stable release : 0.7.1 > Old release : 0.6.2, 0.6.1 > > Once we feel confident about the stability of latest release, we can just > change label from latest to stable in the download page. (and previous > stable goes to old releases) > We can even include formal vote for moving release from 'latest' to > 'stable' in our release process, if it is necessary. > > Thanks, > moon > > On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org> wrote: > > Yes, having longer RC period will help. > > But if i recall 0.7.0 release, although 21 people participated verifying > through 4 RC for 15days, it wasn't enough to catch all critical problems > during the release process. After the release, we've got much more number > of bug reports, in next few days. > > Basically, verifying RC is limited to people who subscribe mailing list + > willing to contribute time to verify RC, which is much smaller number of > people who download release from download page. So having longer RC period > will definitely help and i think we should do, but I think it's still not > enough to make sure the quality, considering past history. > > AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. > ASF release process defines how to release source code, but it does not > really restrict what kind of 'version' the project should have releases. > For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. > > Thanks, > moon > > [1] http://spark.apache.org/news/spark-2.0.0-preview.html > > On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote: > > I agree that it will help prolong RC period and use it actually. And also > we need code freeze for the new features and spend time to stabilize RC. > > On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com> > wrote: > > +1 on quality and stabilization. > > I'm not sure if releasing as preview or calling it unstable fits with the > ASF release process though. > > Other projects have code freeze, RC (and longer RC iteration time) etc. - > do we think those will help improve quality when the release is finally cut? > > > _ > From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> > Sent: Monday, March 20, 2017 6:13 PM > Subject: Re: Roadmap for 0.8.0 > > To: <us...@zeppelin.apache.org>, dev <dev@zeppelin.apache.org> > > > > > > Strongly +1 for adding system test for different interpreter modes and > focus on bug fixing than new features. I do heard from some users complain > about the bugs of zeppelin major release. A stabilized release is very > necessary for community. > > > > > Best Regard, > Jeff Zhang > &g
Re: Roadmap for 0.8.0
That's exciting to see plans for 0.8.0 on the horizon. Here's my top list for 0.8 : - https://issues.apache.org/jira/browse/ZEPPELIN-2197 "Interpreter Idle timeout" This is a most-wanted feature by our Zeppelin admins. It was mentioned at least once on this email chain. - https://issues.apache.org/jira/browse/ZEPPELIN-1967 "Passing Z variables to Shell Interpreter" We had several of our users asking about this functionality. %sh and some other interpreters can't be parametrized by global variables. ZEPPELIN-1967 is one way of how this can be solved. - https://issues.apache.org/jira/browse/ZEPPELIN-1660 "Home directory references (i.e. ~/zeppelin/) in zeppelin-env.sh don't work as expected" Less of a critical compared to the above two, but it could complement the multi-tenancy feature very well. Best regards, Ruslan Dautkhanov On Wed, Mar 22, 2017 at 11:29 AM, Felix Cheung <felixcheun...@hotmail.com> wrote: > +1 with latest/stable. > > > > > -- > *From:* moon soo Lee <m...@apache.org> > *Sent:* Tuesday, March 21, 2017 8:41:58 AM > *To:* us...@zeppelin.apache.org > *Cc:* dev@zeppelin.apache.org > > *Subject:* Re: Roadmap for 0.8.0 > > And if i suggest simplest way for us to set quality expectation to user, > which will be labeling release in download page. > > Currently releases are divided into 2 categories in download page. 'Latest > release' and 'Old releases'. I think we can treat 'Latest' as unstable and > add one more category 'Stable release'. > > For example, once 0.8.0 is released, > > Latest release : 0.8.0 > Stable release : 0.7.1 > Old release : 0.6.2, 0.6.1 > > Once we feel confident about the stability of latest release, we can just > change label from latest to stable in the download page. (and previous > stable goes to old releases) > We can even include formal vote for moving release from 'latest' to > 'stable' in our release process, if it is necessary. > > Thanks, > moon > > On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org> wrote: > >> Yes, having longer RC period will help. >> >> But if i recall 0.7.0 release, although 21 people participated verifying >> through 4 RC for 15days, it wasn't enough to catch all critical problems >> during the release process. After the release, we've got much more number >> of bug reports, in next few days. >> >> Basically, verifying RC is limited to people who subscribe mailing list + >> willing to contribute time to verify RC, which is much smaller number of >> people who download release from download page. So having longer RC period >> will definitely help and i think we should do, but I think it's still not >> enough to make sure the quality, considering past history. >> >> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. >> ASF release process defines how to release source code, but it does not >> really restrict what kind of 'version' the project should have releases. >> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. >> >> Thanks, >> moon >> >> [1] http://spark.apache.org/news/spark-2.0.0-preview.html >> >> >> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote: >> >> I agree that it will help prolong RC period and use it actually. And also >> we need code freeze for the new features and spend time to stabilize RC. >> >> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com> >> wrote: >> >> +1 on quality and stabilization. >> >> I'm not sure if releasing as preview or calling it unstable fits with the >> ASF release process though. >> >> Other projects have code freeze, RC (and longer RC iteration time) etc. - >> do we think those will help improve quality when the release is finally cut? >> >> >> _ >> From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> >> Sent: Monday, March 20, 2017 6:13 PM >> Subject: Re: Roadmap for 0.8.0 >> To: <us...@zeppelin.apache.org>, dev <dev@zeppelin.apache.org> >> >> >> >> Strongly +1 for adding system test for different interpreter modes and >> focus on bug fixing than new features. I do heard from some users complain >> about the bugs of zeppelin major release. A stabilized release is very >> necessary for community. >> >> >> >> >> Best Regard, >> Jeff Zhang >> >> >> From: moon soo Lee <m...@apache.org<mailto:m...@apache.org >> <m...@apache.org>>> >> Reply-To: &qu
Re: Roadmap for 0.8.0
+1 with latest/stable. From: moon soo Lee <m...@apache.org> Sent: Tuesday, March 21, 2017 8:41:58 AM To: us...@zeppelin.apache.org Cc: dev@zeppelin.apache.org Subject: Re: Roadmap for 0.8.0 And if i suggest simplest way for us to set quality expectation to user, which will be labeling release in download page. Currently releases are divided into 2 categories in download page. 'Latest release' and 'Old releases'. I think we can treat 'Latest' as unstable and add one more category 'Stable release'. For example, once 0.8.0 is released, Latest release : 0.8.0 Stable release : 0.7.1 Old release : 0.6.2, 0.6.1 Once we feel confident about the stability of latest release, we can just change label from latest to stable in the download page. (and previous stable goes to old releases) We can even include formal vote for moving release from 'latest' to 'stable' in our release process, if it is necessary. Thanks, moon On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org<mailto:m...@apache.org>> wrote: Yes, having longer RC period will help. But if i recall 0.7.0 release, although 21 people participated verifying through 4 RC for 15days, it wasn't enough to catch all critical problems during the release process. After the release, we've got much more number of bug reports, in next few days. Basically, verifying RC is limited to people who subscribe mailing list + willing to contribute time to verify RC, which is much smaller number of people who download release from download page. So having longer RC period will definitely help and i think we should do, but I think it's still not enough to make sure the quality, considering past history. AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. ASF release process defines how to release source code, but it does not really restrict what kind of 'version' the project should have releases. For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. Thanks, moon [1] http://spark.apache.org/news/spark-2.0.0-preview.html On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>> wrote: I agree that it will help prolong RC period and use it actually. And also we need code freeze for the new features and spend time to stabilize RC. On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote: +1 on quality and stabilization. I'm not sure if releasing as preview or calling it unstable fits with the ASF release process though. Other projects have code freeze, RC (and longer RC iteration time) etc. - do we think those will help improve quality when the release is finally cut? _ From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com<mailto:jzh...@hortonworks.com>> Sent: Monday, March 20, 2017 6:13 PM Subject: Re: Roadmap for 0.8.0 To: <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org>>, dev <dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org>> Strongly +1 for adding system test for different interpreter modes and focus on bug fixing than new features. I do heard from some users complain about the bugs of zeppelin major release. A stabilized release is very necessary for community. Best Regard, Jeff Zhang From: moon soo Lee <m...@apache.org<mailto:m...@apache.org><mailto:m...@apache.org>> Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>" <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>> Date: Tuesday, March 21, 2017 at 4:10 AM To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>" <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>>, dev <dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org><mailto:dev@zeppelin.apache.org>> Subject: Re: Roadmap for 0.8.0 Great to see discussion for 0.8.0. List of features for 0.8.0 looks really good. Interpreter factory refactoring Interpreter layer supports various behavior depends on combination of PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for each combination as a first step. Otherwise, any pullrequest will silently break one of behavior at any time no matter we refactor or not. And fixing and testing this behavior is so hard. Once we have complete test cases, not only guarantee the behavior but also make refactoring much easier. 0.8.0 release I'd like to suggest improvements on how we release a new version. In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
Re: Roadmap for 0.8.0
And if i suggest simplest way for us to set quality expectation to user, which will be labeling release in download page. Currently releases are divided into 2 categories in download page. 'Latest release' and 'Old releases'. I think we can treat 'Latest' as unstable and add one more category 'Stable release'. For example, once 0.8.0 is released, Latest release : 0.8.0 Stable release : 0.7.1 Old release : 0.6.2, 0.6.1 Once we feel confident about the stability of latest release, we can just change label from latest to stable in the download page. (and previous stable goes to old releases) We can even include formal vote for moving release from 'latest' to 'stable' in our release process, if it is necessary. Thanks, moon On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org> wrote: > Yes, having longer RC period will help. > > But if i recall 0.7.0 release, although 21 people participated verifying > through 4 RC for 15days, it wasn't enough to catch all critical problems > during the release process. After the release, we've got much more number > of bug reports, in next few days. > > Basically, verifying RC is limited to people who subscribe mailing list + > willing to contribute time to verify RC, which is much smaller number of > people who download release from download page. So having longer RC period > will definitely help and i think we should do, but I think it's still not > enough to make sure the quality, considering past history. > > AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. > ASF release process defines how to release source code, but it does not > really restrict what kind of 'version' the project should have releases. > For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. > > Thanks, > moon > > [1] http://spark.apache.org/news/spark-2.0.0-preview.html > > > On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote: > > I agree that it will help prolong RC period and use it actually. And also > we need code freeze for the new features and spend time to stabilize RC. > > On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com> > wrote: > > +1 on quality and stabilization. > > I'm not sure if releasing as preview or calling it unstable fits with the > ASF release process though. > > Other projects have code freeze, RC (and longer RC iteration time) etc. - > do we think those will help improve quality when the release is finally cut? > > > _____ > From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> > Sent: Monday, March 20, 2017 6:13 PM > Subject: Re: Roadmap for 0.8.0 > To: <us...@zeppelin.apache.org>, dev <dev@zeppelin.apache.org> > > > > Strongly +1 for adding system test for different interpreter modes and > focus on bug fixing than new features. I do heard from some users complain > about the bugs of zeppelin major release. A stabilized release is very > necessary for community. > > > > > Best Regard, > Jeff Zhang > > > From: moon soo Lee <m...@apache.org<mailto:m...@apache.org > <m...@apache.org>>> > Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org > <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org< > mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>> > Date: Tuesday, March 21, 2017 at 4:10 AM > To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org > <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org< > mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev < > dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org > <dev@zeppelin.apache.org>>> > > Subject: Re: Roadmap for 0.8.0 > > Great to see discussion for 0.8.0. > List of features for 0.8.0 looks really good. > > Interpreter factory refactoring > Interpreter layer supports various behavior depends on combination of > PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for > each combination as a first step. > Otherwise, any pullrequest will silently break one of behavior at any time > no matter we refactor or not. And fixing and testing this behavior is so > hard. > Once we have complete test cases, not only guarantee the behavior but also > make refactoring much easier. > > > 0.8.0 release > I'd like to suggest improvements on how we release a new version. > > In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 > months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) > > I think the same thing will happen again with 0.8.0, while we're going to
Re: Roadmap for 0.8.0
Yes, having longer RC period will help. But if i recall 0.7.0 release, although 21 people participated verifying through 4 RC for 15days, it wasn't enough to catch all critical problems during the release process. After the release, we've got much more number of bug reports, in next few days. Basically, verifying RC is limited to people who subscribe mailing list + willing to contribute time to verify RC, which is much smaller number of people who download release from download page. So having longer RC period will definitely help and i think we should do, but I think it's still not enough to make sure the quality, considering past history. AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project. ASF release process defines how to release source code, but it does not really restrict what kind of 'version' the project should have releases. For example, spark released spark-2.0.0-preview[1] before spark-2.0.0. Thanks, moon [1] http://spark.apache.org/news/spark-2.0.0-preview.html On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote: > I agree that it will help prolong RC period and use it actually. And also > we need code freeze for the new features and spend time to stabilize RC. > > On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com> > wrote: > > +1 on quality and stabilization. > > I'm not sure if releasing as preview or calling it unstable fits with the > ASF release process though. > > Other projects have code freeze, RC (and longer RC iteration time) etc. - > do we think those will help improve quality when the release is finally cut? > > > _ > From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> > Sent: Monday, March 20, 2017 6:13 PM > Subject: Re: Roadmap for 0.8.0 > To: <us...@zeppelin.apache.org>, dev <dev@zeppelin.apache.org> > > > > Strongly +1 for adding system test for different interpreter modes and > focus on bug fixing than new features. I do heard from some users complain > about the bugs of zeppelin major release. A stabilized release is very > necessary for community. > > > > > Best Regard, > Jeff Zhang > > > From: moon soo Lee <m...@apache.org<mailto:m...@apache.org > <m...@apache.org>>> > Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org > <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org< > mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>> > Date: Tuesday, March 21, 2017 at 4:10 AM > To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org > <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org< > mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev < > dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org > <dev@zeppelin.apache.org>>> > > Subject: Re: Roadmap for 0.8.0 > > Great to see discussion for 0.8.0. > List of features for 0.8.0 looks really good. > > Interpreter factory refactoring > Interpreter layer supports various behavior depends on combination of > PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for > each combination as a first step. > Otherwise, any pullrequest will silently break one of behavior at any time > no matter we refactor or not. And fixing and testing this behavior is so > hard. > Once we have complete test cases, not only guarantee the behavior but also > make refactoring much easier. > > > 0.8.0 release > I'd like to suggest improvements on how we release a new version. > > In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 > months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) > > I think the same thing will happen again with 0.8.0, while we're going to > make lots of changes and add many new features. > After we released 0.8.0, while 'Stabilizing' the new release, user who > tried the new release may get wrong impression of the quality. Which is > very bad and we already repeated the mistake in 0.6.0 and 0.7.0. > > So from 0.8.0 release, I'd suggest we improve way we release new version > to give user proper expectation. I think there're several ways of doing it. > > 1. Release 0.8.0-preview officially and then release 0.8.0. > 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a > 'stable' release in the download page. Once 0.8.x release becomes stable > enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases. > > > After 0.8.0, > Since Zeppelin projects starts, project went through some major milestone, > like > > - project gets first users and first contributor > - project went
Re: Roadmap for 0.8.0
I agree that it will help prolong RC period and use it actually. And also we need code freeze for the new features and spend time to stabilize RC. On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com> wrote: > +1 on quality and stabilization. > > I'm not sure if releasing as preview or calling it unstable fits with the > ASF release process though. > > Other projects have code freeze, RC (and longer RC iteration time) etc. - > do we think those will help improve quality when the release is finally cut? > > > _ > From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com> > Sent: Monday, March 20, 2017 6:13 PM > Subject: Re: Roadmap for 0.8.0 > To: <us...@zeppelin.apache.org>, dev <dev@zeppelin.apache.org> > > > > Strongly +1 for adding system test for different interpreter modes and > focus on bug fixing than new features. I do heard from some users complain > about the bugs of zeppelin major release. A stabilized release is very > necessary for community. > > > > > Best Regard, > Jeff Zhang > > > From: moon soo Lee <m...@apache.org<mailto:m...@apache.org > <m...@apache.org>>> > Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org > <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org lto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>> > Date: Tuesday, March 21, 2017 at 4:10 AM > To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org > <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org lto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev < > dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org > <dev@zeppelin.apache.org>>> > > Subject: Re: Roadmap for 0.8.0 > > Great to see discussion for 0.8.0. > List of features for 0.8.0 looks really good. > > Interpreter factory refactoring > Interpreter layer supports various behavior depends on combination of > PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for > each combination as a first step. > Otherwise, any pullrequest will silently break one of behavior at any time > no matter we refactor or not. And fixing and testing this behavior is so > hard. > Once we have complete test cases, not only guarantee the behavior but also > make refactoring much easier. > > > 0.8.0 release > I'd like to suggest improvements on how we release a new version. > > In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 > months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) > > I think the same thing will happen again with 0.8.0, while we're going to > make lots of changes and add many new features. > After we released 0.8.0, while 'Stabilizing' the new release, user who > tried the new release may get wrong impression of the quality. Which is > very bad and we already repeated the mistake in 0.6.0 and 0.7.0. > > So from 0.8.0 release, I'd suggest we improve way we release new version > to give user proper expectation. I think there're several ways of doing it. > > 1. Release 0.8.0-preview officially and then release 0.8.0. > 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a > 'stable' release in the download page. Once 0.8.x release becomes stable > enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases. > > > After 0.8.0, > Since Zeppelin projects starts, project went through some major milestone, > like > > - project gets first users and first contributor > - project went into Apache Incubator > - project became TLP. > > And I think it's time to think about hitting another major milestone. > > Considering features we already have, features we're planning on 0.8, wide > adoption of Zeppelin in the industry, I think it's time to focus on make > project more mature and make a 1.0 release. Which i think big milestone for > the project. > > After 0.8.0 release, I suggest we more focus on bug fixes, stability > improvement, optimizing user experience than adding new features. And with > subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about > the quality, release it as a 1.0.0 instead of 0.8.x. > > Once we have 1.0.0 released, then I think we can make larger, experimental > changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x > branch. > > > Thanks, > moon > > On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheun...@hotmail.com< > mailto:felixcheun...@hotmail.com <felixcheun...@hotmail.com>>> wrote: > There are several pending visualization improvements/PRs that would be >
Re: Roadmap for 0.8.0
+1 on quality and stabilization. I'm not sure if releasing as preview or calling it unstable fits with the ASF release process though. Other projects have code freeze, RC (and longer RC iteration time) etc. - do we think those will help improve quality when the release is finally cut? _ From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com<mailto:jzh...@hortonworks.com>> Sent: Monday, March 20, 2017 6:13 PM Subject: Re: Roadmap for 0.8.0 To: <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org>>, dev <dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org>> Strongly +1 for adding system test for different interpreter modes and focus on bug fixing than new features. I do heard from some users complain about the bugs of zeppelin major release. A stabilized release is very necessary for community. Best Regard, Jeff Zhang From: moon soo Lee <m...@apache.org<mailto:m...@apache.org><mailto:m...@apache.org>> Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>" <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>> Date: Tuesday, March 21, 2017 at 4:10 AM To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>" <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org>>, dev <dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org><mailto:dev@zeppelin.apache.org>> Subject: Re: Roadmap for 0.8.0 Great to see discussion for 0.8.0. List of features for 0.8.0 looks really good. Interpreter factory refactoring Interpreter layer supports various behavior depends on combination of PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for each combination as a first step. Otherwise, any pullrequest will silently break one of behavior at any time no matter we refactor or not. And fixing and testing this behavior is so hard. Once we have complete test cases, not only guarantee the behavior but also make refactoring much easier. 0.8.0 release I'd like to suggest improvements on how we release a new version. In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) I think the same thing will happen again with 0.8.0, while we're going to make lots of changes and add many new features. After we released 0.8.0, while 'Stabilizing' the new release, user who tried the new release may get wrong impression of the quality. Which is very bad and we already repeated the mistake in 0.6.0 and 0.7.0. So from 0.8.0 release, I'd suggest we improve way we release new version to give user proper expectation. I think there're several ways of doing it. 1. Release 0.8.0-preview officially and then release 0.8.0. 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a 'stable' release in the download page. Once 0.8.x release becomes stable enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases. After 0.8.0, Since Zeppelin projects starts, project went through some major milestone, like - project gets first users and first contributor - project went into Apache Incubator - project became TLP. And I think it's time to think about hitting another major milestone. Considering features we already have, features we're planning on 0.8, wide adoption of Zeppelin in the industry, I think it's time to focus on make project more mature and make a 1.0 release. Which i think big milestone for the project. After 0.8.0 release, I suggest we more focus on bug fixes, stability improvement, optimizing user experience than adding new features. And with subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about the quality, release it as a 1.0.0 instead of 0.8.x. Once we have 1.0.0 released, then I think we can make larger, experimental changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x branch. Thanks, moon On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com><mailto:felixcheun...@hotmail.com>> wrote: There are several pending visualization improvements/PRs that would be very good to get them in as well. From: Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com><mailto:jongy...@gmail.com>> Sent: Sunday, March 19, 2017 9:03:24 PM To: dev; us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org><mailto:us...@zeppelin.apache.org> Subject: Roadmap for 0.8.0 Hi dev & users, Recently, community submits very new features for Apache Zeppelin. I think it's very positive signals to improve Apache Zeppelin and its community. But
Re: Roadmap for 0.8.0
Strongly +1 for adding system test for different interpreter modes and focus on bug fixing than new features. I do heard from some users complain about the bugs of zeppelin major release. A stabilized release is very necessary for community. Best Regard, Jeff Zhang From: moon soo Lee <m...@apache.org<mailto:m...@apache.org>> Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org>" <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org>> Date: Tuesday, March 21, 2017 at 4:10 AM To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org>" <us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org>>, dev <dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org>> Subject: Re: Roadmap for 0.8.0 Great to see discussion for 0.8.0. List of features for 0.8.0 looks really good. Interpreter factory refactoring Interpreter layer supports various behavior depends on combination of PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for each combination as a first step. Otherwise, any pullrequest will silently break one of behavior at any time no matter we refactor or not. And fixing and testing this behavior is so hard. Once we have complete test cases, not only guarantee the behavior but also make refactoring much easier. 0.8.0 release I'd like to suggest improvements on how we release a new version. In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) I think the same thing will happen again with 0.8.0, while we're going to make lots of changes and add many new features. After we released 0.8.0, while 'Stabilizing' the new release, user who tried the new release may get wrong impression of the quality. Which is very bad and we already repeated the mistake in 0.6.0 and 0.7.0. So from 0.8.0 release, I'd suggest we improve way we release new version to give user proper expectation. I think there're several ways of doing it. 1. Release 0.8.0-preview officially and then release 0.8.0. 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a 'stable' release in the download page. Once 0.8.x release becomes stable enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases. After 0.8.0, Since Zeppelin projects starts, project went through some major milestone, like - project gets first users and first contributor - project went into Apache Incubator - project became TLP. And I think it's time to think about hitting another major milestone. Considering features we already have, features we're planning on 0.8, wide adoption of Zeppelin in the industry, I think it's time to focus on make project more mature and make a 1.0 release. Which i think big milestone for the project. After 0.8.0 release, I suggest we more focus on bug fixes, stability improvement, optimizing user experience than adding new features. And with subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about the quality, release it as a 1.0.0 instead of 0.8.x. Once we have 1.0.0 released, then I think we can make larger, experimental changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x branch. Thanks, moon On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote: There are several pending visualization improvements/PRs that would be very good to get them in as well. From: Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>> Sent: Sunday, March 19, 2017 9:03:24 PM To: dev; us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org> Subject: Roadmap for 0.8.0 Hi dev & users, Recently, community submits very new features for Apache Zeppelin. I think it's very positive signals to improve Apache Zeppelin and its community. But in another aspect, we should focus on what the next release includes. I think we need to summarize and prioritize them. Here is what I know: * Cluster management * Admin feature * Replace some context to separate users * Helium online Feel free to talk if you want to add more things. I think we need to choose which features will be included in 0.8.0, too. Regards, Jongyoul Lee -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Roadmap for 0.8.0
Great to see discussion for 0.8.0. List of features for 0.8.0 looks really good. *Interpreter factory refactoring* Interpreter layer supports various behavior depends on combination of PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for each combination as a first step. Otherwise, any pullrequest will silently break one of behavior at any time no matter we refactor or not. And fixing and testing this behavior is so hard. Once we have complete test cases, not only guarantee the behavior but also make refactoring much easier. *0.8.0 release* I'd like to suggest improvements on how we release a new version. In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months) I think the same thing will happen again with 0.8.0, while we're going to make lots of changes and add many new features. After we released 0.8.0, while 'Stabilizing' the new release, user who tried the new release may get wrong impression of the quality. Which is very bad and we already repeated the mistake in 0.6.0 and 0.7.0. So from 0.8.0 release, I'd suggest we improve way we release new version to give user proper expectation. I think there're several ways of doing it. 1. Release 0.8.0-preview officially and then release 0.8.0. 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a 'stable' release in the download page. Once 0.8.x release becomes stable enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases. *After 0.8.0, * Since Zeppelin projects starts, project went through some major milestone, like - project gets first users and first contributor - project went into Apache Incubator - project became TLP. And I think it's time to think about hitting another major milestone. Considering features we already have, features we're planning on 0.8, wide adoption of Zeppelin in the industry, I think it's time to focus on make project more mature and make a 1.0 release. Which i think big milestone for the project. After 0.8.0 release, I suggest we more focus on bug fixes, stability improvement, optimizing user experience than adding new features. And with subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about the quality, release it as a 1.0.0 instead of 0.8.x. Once we have 1.0.0 released, then I think we can make larger, experimental changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x branch. Thanks, moon On Mon, Mar 20, 2017 at 8:55 AM Felix Cheungwrote: > There are several pending visualization improvements/PRs that would be > very good to get them in as well. > > > -- > *From:* Jongyoul Lee > *Sent:* Sunday, March 19, 2017 9:03:24 PM > *To:* dev; us...@zeppelin.apache.org > *Subject:* Roadmap for 0.8.0 > > Hi dev & users, > > Recently, community submits very new features for Apache Zeppelin. I think > it's very positive signals to improve Apache Zeppelin and its community. > But in another aspect, we should focus on what the next release includes. I > think we need to summarize and prioritize them. Here is what I know: > > * Cluster management > * Admin feature > * Replace some context to separate users > * Helium online > > Feel free to talk if you want to add more things. I think we need to > choose which features will be included in 0.8.0, too. > > Regards, > Jongyoul Lee > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net >
Re: Roadmap for 0.8.0
There are several pending visualization improvements/PRs that would be very good to get them in as well. From: Jongyoul LeeSent: Sunday, March 19, 2017 9:03:24 PM To: dev; us...@zeppelin.apache.org Subject: Roadmap for 0.8.0 Hi dev & users, Recently, community submits very new features for Apache Zeppelin. I think it's very positive signals to improve Apache Zeppelin and its community. But in another aspect, we should focus on what the next release includes. I think we need to summarize and prioritize them. Here is what I know: * Cluster management * Admin feature * Replace some context to separate users * Helium online Feel free to talk if you want to add more things. I think we need to choose which features will be included in 0.8.0, too. Regards, Jongyoul Lee -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Roadmap for 0.8.0
Yeah, make sense. Jongyoul Lee于2017年3月20日周一 下午7:21写道: > Thanks for letting me know. I agree almost things we should develop. > Personally, concerning refactoring it, I'm doing a bit with several PRs but > we need to restructure InterpreterFactory. At first, list up all issues and > make some groups and handle it. How do you think? > > On Mon, Mar 20, 2017 at 2:12 PM, Jeff Zhang wrote: > > > > Here's some candidates for 0.8 IMO > >- Restructing InterpreterFacotry >https://issues.apache.org/jira/browse/ZEPPELIN-2056Although it is >refactoring ticket, I feel it is pretty important thing to do. As I see >many bugs are due to interpreter factory component, and I do feel it needs >refactoring. >- Admin Feature https://issues.apache.org/jira/browse/ZEPPELIN-2236 >- User Level Interpreter Setting >https://issues.apache.org/jira/browse/ZEPPELIN-1338 >- Interpreter Lifecycle Control >https://issues.apache.org/jira/browse/ZEPPELIN-2197 > > > > Jongyoul Lee 于2017年3月20日周一 下午12:03写道: > > Hi dev & users, > > Recently, community submits very new features for Apache Zeppelin. I think > it's very positive signals to improve Apache Zeppelin and its community. > But in another aspect, we should focus on what the next release includes. I > think we need to summarize and prioritize them. Here is what I know: > > * Cluster management > * Admin feature > * Replace some context to separate users > * Helium online > > Feel free to talk if you want to add more things. I think we need to choose > which features will be included in 0.8.0, too. > > Regards, > Jongyoul Lee > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net > > > > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net >
Re: Roadmap for 0.8.0
Thanks for letting me know. I agree almost things we should develop. Personally, concerning refactoring it, I'm doing a bit with several PRs but we need to restructure InterpreterFactory. At first, list up all issues and make some groups and handle it. How do you think? On Mon, Mar 20, 2017 at 2:12 PM, Jeff Zhangwrote: > > > Here's some candidates for 0.8 IMO > >- Restructing InterpreterFacotry https://issues.apache.org/ >jira/browse/ZEPPELIN-2056Although it is refactoring ticket, I feel >it is pretty important thing to do. As I see many bugs are due to >interpreter factory component, and I do feel it needs refactoring. >- Admin Feature https://issues.apache.org/jira/browse/ZEPPELIN-2236 >- User Level Interpreter Setting https://issues.apache.org/ >jira/browse/ZEPPELIN-1338 >- Interpreter Lifecycle Control https://issues.apache. >org/jira/browse/ZEPPELIN-2197 > > > > Jongyoul Lee 于2017年3月20日周一 下午12:03写道: > >> Hi dev & users, >> >> Recently, community submits very new features for Apache Zeppelin. I think >> it's very positive signals to improve Apache Zeppelin and its community. >> But in another aspect, we should focus on what the next release includes. >> I >> think we need to summarize and prioritize them. Here is what I know: >> >> * Cluster management >> * Admin feature >> * Replace some context to separate users >> * Helium online >> >> Feel free to talk if you want to add more things. I think we need to >> choose >> which features will be included in 0.8.0, too. >> >> Regards, >> Jongyoul Lee >> >> -- >> 이종열, Jongyoul Lee, 李宗烈 >> http://madeng.net >> > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net
Re: Roadmap for 0.8.0
Here's some candidates for 0.8 IMO - Restructing InterpreterFacotry https://issues.apache.org/jira/browse/ZEPPELIN-2056Although it is refactoring ticket, I feel it is pretty important thing to do. As I see many bugs are due to interpreter factory component, and I do feel it needs refactoring. - Admin Feature https://issues.apache.org/jira/browse/ZEPPELIN-2236 - User Level Interpreter Setting https://issues.apache.org/jira/browse/ZEPPELIN-1338 - Interpreter Lifecycle Control https://issues.apache.org/jira/browse/ZEPPELIN-2197 Jongyoul Lee于2017年3月20日周一 下午12:03写道: > Hi dev & users, > > Recently, community submits very new features for Apache Zeppelin. I think > it's very positive signals to improve Apache Zeppelin and its community. > But in another aspect, we should focus on what the next release includes. I > think we need to summarize and prioritize them. Here is what I know: > > * Cluster management > * Admin feature > * Replace some context to separate users > * Helium online > > Feel free to talk if you want to add more things. I think we need to choose > which features will be included in 0.8.0, too. > > Regards, > Jongyoul Lee > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net >