It sounds as a most flexible way, let's try it for 0.6 release and see if it addresses all user needs well.
Untill we have GUI for interpreter loading, I feel we also should try our best to make sure netinst usage is documented well, so users on different environments (no internet, corporate proxy, etc) all know how to use it. On Tue, Jun 21, 2016, 15:25 mina lee <mina.hyeji....@gmail.com> wrote: > Moon, having netinst package for the sake of simplicity and flexibility > totally makes sense to me. > If there is no strong objection, I would like to follow your approach for > 0.6.0 release. > > On Mon, Jun 20, 2016 at 6:23 PM, moon soo Lee <m...@apache.org> wrote: > >> "zeppelin-bin-min" package, I worried about lack of written policy which >> goes in which does not. >> Without policy, yes, we can always vote for list of interpreters. But >> then, we'll need vote everytime we add/remove interpreter, and this >> sounds not good. >> >> Even if it is true that majority of user uses 'spark', >> other users may ask "zeppelin-bin-cassandra-min", >> "zeppelin-bin-flink-min" and so on. >> Once we have 'zeppelin-bin-min' package with 'spark', then there will be >> no good excuse of not having other *min packages. >> And we can end up with a lot of binary packages in each release. which is >> not really optimal. >> >> For this reasons, I believe we'll need a written policy based on >> community consensus to make 'zeppelin-bin-min'. >> But I think making netinst script will be a lot easier and give more >> flexibility to users than making written policy for 'zeppelin-bin-min'. >> >> Anyway, it's really great to hear volunteer some time to help. >> Thanks Mohit Jaggi. >> Whether we have multiple packages or not, we'll need a lot of help on >> improving release script [1] and verification of release candidates. >> >> Regarding maintainers/contributors of each interpreter(s), >> Spark community recently removed 'maintainer' role from review process >> [2] for some reasons. >> DuyHai Doan, could you give little more details about how your idea >> different from maintainers of Spark? >> >> Thanks, >> moon >> >> [1] https://github.com/apache/zeppelin/blob/master/dev/create_release.sh >> [2] >> https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=34835307&originalVersion=61&revisedVersion=66 >> >> >> On Mon, Jun 20, 2016 at 3:10 AM DuyHai Doan <doanduy...@gmail.com> wrote: >> >>> +1 for zeppelin-bin-min release package >>> >>> What I would suggest is that for a specific package of Zeppelin with XXX >>> interpreter(s) built-in is that the maintainers/contributors of each >>> interpreter(s) can help releasing those "custom" builds for the community. >>> Any thought on this idea ? >>> >>> On Mon, Jun 20, 2016 at 10:30 AM, Partridge, Lucas (GE Aviation) < >>> lucas.partri...@ge.com> wrote: >>> >>>> I like the 'zeppelin-bin-netinst’ idea too. Hopefully it would be easy >>>> to configure it to work with a proxy for users behind a corporate firewall. >>>> >>>> Thanks, Lucas. >>>> >>>> >>>> >>>> *From:* Mohit Jaggi [mailto:mohitja...@gmail.com] >>>> *Sent:* 17 June 2016 18:06 >>>> *To:* users@zeppelin.apache.org >>>> *Subject:* EXT: Re: Ask opinion regarding 0.6.0 release package >>>> >>>> >>>> >>>> sure…that is possible. one can also make a build from source and >>>> customize as needed. but not having to do that makes things easier. i do >>>> believe that for the vast majority of cases a minimal build with spark (and >>>> possibly other small items like shell, jdbc, python) will be quite >>>> valuable, imho. >>>> >>>> is there a lot of overhead involved in having multiple binaries >>>> available? i am happy to volunteer some time to help with this if needed. >>>> >>>> >>>> >>>> On Jun 17, 2016, at 9:45 PM, moon soo Lee <m...@apache.org> wrote: >>>> >>>> >>>> >>>> In case of no internet access, how about >>>> >>>> >>>> >>>> a. download 'zeppelin-bin-netinst' and run >>>> 'bin/install-interpreter.sh', and then copy the package to production env. >>>> >>>> b. download 'zeppelin-bin-all' and copy the package to production env. >>>> >>>> >>>> >>>> ? >>>> >>>> >>>> >>>> Thanks, >>>> >>>> moon >>>> >>>> >>>> >>>> >>>> >>>> On Fri, Jun 17, 2016 at 9:10 AM Mohit Jaggi <mohitja...@gmail.com> >>>> wrote: >>>> >>>> Many production environments have no internet access. A script like >>>> this can be useful to some but it should not replace the proposed min >>>> binary. >>>> >>>> Sent from my iPhone >>>> >>>> >>>> On Jun 17, 2016, at 9:20 PM, moon soo Lee <m...@apache.org> wrote: >>>> >>>> Hi, >>>> >>>> >>>> >>>> Thanks for bringing this discussion. >>>> >>>> it's great idea minimize binary package size. >>>> >>>> >>>> >>>> Can we set a policy to decide which interpreter goes to >>>> 'zeppelin-bin-min', which is not? >>>> >>>> >>>> >>>> One alternative is, instead of making 'zeppelin-bin-min', we can make >>>> 'zeppelin-bin-netinst'. >>>> >>>> We can provide a shell script such as, 'bin/install-interpreter.sh' and >>>> the script will download interpreters and their dependencies from maven >>>> repository and store under /interpreter dir. By >>>> leveraging DependencyResolver[1], i think we can make this feature in >>>> couple of hours. >>>> >>>> >>>> >>>> Only spark interpreter can not be installed in simple way, while it >>>> requires some python and R packages under /interpreter dir and they're not >>>> available on maven repository, so it'll need special treatment, but all >>>> other interpreters can be installed in the simple way. >>>> >>>> >>>> >>>> Then, 'zeppelin-bin-netinst' version can have minimal package size, and >>>> still gives easy way to install all the interpreters. >>>> >>>> Also 'bin/install-interpreter.sh' will still useful even if we have >>>> dynamic interpreter loading feature [2], to build offline package. >>>> >>>> >>>> >>>> what do you think? >>>> >>>> >>>> >>>> [1] >>>> https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/dep/DependencyResolver.java >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_zeppelin_blob_master_zeppelin-2Dinterpreter_src_main_java_org_apache_zeppelin_dep_DependencyResolver.java&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=b48EeMu0glDkXuGn72ZTy8ZteEiVBzmpbTqELmhgsRc&e=> >>>> >>>> [2] https://issues.apache.org/jira/browse/ZEPPELIN-598 >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_ZEPPELIN-2D598&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=MK9lpcZjSIlgFO0CVk6kMWB1bCPqpWK_0qhSjOQ5FzA&e=> >>>> >>>> >>>> >>>> >>>> >>>> On Fri, Jun 17, 2016 at 1:02 AM mina lee <mina...@apache.org> wrote: >>>> >>>> Hi all! >>>> >>>> >>>> >>>> Zeppelin just started release process. Prior to creating release >>>> candidate I want to ask users' opinion about how you want it to be >>>> packaged. >>>> >>>> >>>> >>>> For the last release(0.5.6), we have released one binary package which >>>> includes all interpreters. >>>> >>>> The concern with providing one type of binary package is that package >>>> size will be quite big(~600MB). >>>> >>>> So I am planning to provide two binary packages: >>>> >>>> - zeppelin-0.6.0-bin-all.tgz (includes all interpreters) >>>> >>>> - zeppelin-0.6.0-bin-min.tgz (includes only most used interpreters) >>>> >>>> >>>> >>>> I am thinking about putting *spark(pyspark, sparkr, sql), python, >>>> jdbc, shell, markdown, angular* in minimized package. >>>> >>>> Could you give your opinion on whether these sets are enough, or some >>>> of them are ok to be excluded? >>>> >>>> >>>> >>>> Community's opinion will be helpful to make decision not only for 0.6.0 >>>> but also for 0.7.0 release since we are planning to provide only minimized >>>> package from 0.7.0 release. From the 0.7.0 version, interpreters those are >>>> not included in binary package will be able to use dynamic interpreter >>>> feature [1] which is in progress under [2]. >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Mina >>>> >>>> >>>> >>>> [1] >>>> http://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/manual/dynamicinterpreterload.html >>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__zeppelin.apache.org_docs_0.6.0-2DSNAPSHOT_manual_dynamicinterpreterload.html&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=4zHncvKGMfOlq-dTmD3m23Rv0jjkaqwWEnowkaJSHks&e=> >>>> >>>> [2] https://github.com/apache/zeppelin/pull/908 >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_zeppelin_pull_908&d=CwMFaQ&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=c1CCNND4PG-Q_V2AJWDWrugZAXQ8Y3EE_f_mAHcpXcs&m=5yX9TVM8vp2oYgFtB4gACTyCQL3FWTK2OoSXVzsJpdg&s=TB3EaiWKtliKYXmXWHJLyZK4Kti6Ev97GVBJFfhCcVw&e=> >>>> >>>> >>>> >>> >>> >
