Apologies for silence. Thanks for the idea about popularity. Do you have any good way to track # of downloads per each interpreter? Because of they're distributed all together it's not going to be easy.
I have little bit different opinion. I believe core interpreters are the interpreters that zeppelin community can maintain, regardless of popularity. For example, let's say one day someone contributed awesome interpreter, but no one in the community does improve, fix bugs, answer the question, then i think it is not maintainable and better not belongs to core. Best, moon On 2015년 7월 26일 (일) at 오후 3:50 tog <[email protected]> wrote: > Hi moon > > I believe popularity can be measured by (unique) downloads and/or starring > by users. Probably tracking the # of downloads is easier. > > Cheers > Guillaume > > On Sunday, July 26, 2015, moon soo Lee <[email protected]> wrote: > > > Thanks for the opinion. How do you define popularity? > > > > Best, > > moon > > On 2015년 7월 24일 (금) at 오후 2:52 tog <[email protected] > > <javascript:;>> wrote: > > > > > >About the maven profile DuyHai suggested, how do we decide which > > > >interpreter should be in core and which interpreter should not be in? > > > > > > Several strategies here ;-) > > > > > > My initial take would be: > > > - > > > (1) Core: > > > 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-web' > > > - (2) Externals but supported by Zeppelin project: spark, PySpark. > > > - (3) All others externals at the beginning - then you could move > > things > > > between (2) and (3) ie deprecate plugins that are getting less popular > > and > > > promote plugins that are getting more tractions > > > > > > On Friday, July 24, 2015, moon soo Lee <[email protected] > <javascript:;>> > > wrote: > > > > > > > Documenting adding external interpreter and making an sample minimal > > > > project sounds great!! That would help both users and developers. > > > > > > > > About the maven profile DuyHai suggested, how do we decide which > > > > interpreter should be in core and which interpreter should not be in? > > > > > > > > Another way of doing similar action is using -pl flag. > > > > > > > > # build Zeppelin with spark and flink interpreter > > > > mvn package -pl > > > > > > > > > > > > > > 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-web,spark,fllnk' > > > > > > > > # build Zeppelin except for spark and flink interpreter > > > > mvn package -pl '!spark,!flink' > > > > > > > > > > > > Thanks, > > > > moon > > > > > > > > On 2015년 7월 23일 (목) at 오후 6:43 tog <[email protected] > > <javascript:;> > > > > <javascript:;>> wrote: > > > > > > > > > That would indeed be a nice addition. > > > > > Apparently moon said it should be possible to drop a jar under > > > > > interpreter/[name] directory ... that jar would nevertheless need > to > > > > extend > > > > > interpreter class so we may want to provide a sample minimal > project > > > > > showing that or a tutorial (the shell interpreter is good for that > > > > because > > > > > simple) > > > > > Can work with Eran on that. > > > > > > > > > > On 23 July 2015 at 10:31, Corneau Damien <[email protected] > > <javascript:;> > > > > <javascript:;>> wrote: > > > > > > > > > > > Any way to add an interpreter without rebuilding zeppelin? > > > > > > > > > > > > On Thu, Jul 23, 2015 at 5:12 PM, DuyHai Doan < > [email protected] > > <javascript:;> > > > > <javascript:;>> > > > > > wrote: > > > > > > > > > > > > > For decoupling the build of core Zeppelin vs third-party > > > > interpreters, > > > > > we > > > > > > > can use Maven profiles: > > > > > > > > > > > > > > in the root pom.xml, you can define in the default profile all > > the > > > > > Maven > > > > > > > modules that should be part of the core Zeppelin build. > > Additional > > > > > > > interpreters can be defined in separated profiles so that > people > > > keen > > > > > on > > > > > > > using them can build them easily. Ex: > > > > > > > > > > > > > > <profiles> > > > > > > > > > > > > > > <!-- Core profile --> > > > > > > > <profile> > > > > > > > <id>core</id> > > > > > > > <activation> > > > > > > > <activeByDefault>true</activeByDefault> > > > > > > > </activation> > > > > > > > <modules> > > > > > > > <module>zeppelin-interpreter</module> > > > > > > > <module>zeppelin-zengine</module> > > > > > > > <module>spark</module> > > > > > > > <module>markdown</module> > > > > > > > <module>angular</module> > > > > > > > <module>shell</module> > > > > > > > <module>zeppelin-web</module> > > > > > > > <module>zeppelin-server</module> > > > > > > > <module>zeppelin-distribution</module> > > > > > > > </modules> > > > > > > > </profile> > > > > > > > > > > > > > > <!-- Cassandra interpreter --> > > > > > > > <profile> > > > > > > > <id>cassandra</id> > > > > > > > <modules> > > > > > > > <module>cassandra</module> > > > > > > > </modules> > > > > > > > </profile> > > > > > > > > > > > > > > <!-- Lens interpreter --> > > > > > > > <profile> > > > > > > > <id>lens</id> > > > > > > > <modules> > > > > > > > <module>lens</module> > > > > > > > </modules> > > > > > > > </profile> > > > > > > > </profiles> > > > > > > > > > > > > > > To build Zeppelin with support for Cassandra & Lens: > > > > > > > > > > > > > > mvn clean package -Pcore,cassandra,lens -D.... > > > > > > > > > > > > > > > > > > > > > What do you think ? > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 7:38 AM, IT CTO <[email protected] > > <javascript:;> > > > > <javascript:;>> wrote: > > > > > > > > > > > > > > > I volunteer to write a doc for the zeppelin documentation > site > > on > > > > > step > > > > > > by > > > > > > > > step for adding an "external interpreter" targeted for non > > > > developers > > > > > > who > > > > > > > > wants to add someone's else external interpreter. > > > > > > > > The challenge I see is from where to get the compiled > > interpreter > > > > > from? > > > > > > > > - my answer is to have a list of external interpreter on our > > site > > > > > which > > > > > > > > points to the github of the external interpreter and the user > > > will > > > > > > > download > > > > > > > > himself. - I think this should help with licensing issues as > > well > > > > as > > > > > an > > > > > > > > external site can hold a non apache license interpreter > (MySQL, > > > > > Oracle) > > > > > > > > > > > > > > > > Eran > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 2:38 AM moon soo Lee < > [email protected] > > <javascript:;> > > > > <javascript:;>> > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks for the opinions and feedback. > > > > > > > > > > > > > > > > > > My question was more like simply asking about merging code > > that > > > > > does > > > > > > > not > > > > > > > > > have test, but apparently went back to recurrent subject. > :-) > > > > > > > > > > > > > > > > > > Actually, plug'n'play interpreter is already possible by > > > dropping > > > > > all > > > > > > > the > > > > > > > > > necessary jars under interpreter/[name] directory and add > > > config > > > > to > > > > > > > > > zeppelin-site.xml. I think nothing stops making externally > > > > managed > > > > > > > > > interpreter. > > > > > > > > > > > > > > > > > > I think having list of all internal + external interpreters > > in > > > > > > homepage > > > > > > > > or > > > > > > > > > wiki and let user update it would help, as a first step. > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > moon > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 9:42 PM Paul Curtis < > > > > [email protected] <javascript:;> <javascript:;>> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > I agree with tog ... the core interpreters should be > those > > > that > > > > > can > > > > > > > > > > complete all the tests and provide the functionality as > > well. > > > > > Spark > > > > > > > > local > > > > > > > > > > being the best example. The Apache Drill interpreter I > > wrote > > > > > would > > > > > > > > need a > > > > > > > > > > single node Drill installation in order to provide the > same > > > > > testing > > > > > > > > > > capability. I am working to provide this. > > > > > > > > > > > > > > > > > > > > However, I would suggest that review may be needed. As > each > > > > test > > > > > > and > > > > > > > > > > interpreter adds Zeppelin code, it also adds to the > > > > distributable > > > > > > as > > > > > > > > > well. > > > > > > > > > > Currently, the distributable is around ~500MB in size. Is > > the > > > > > goal > > > > > > to > > > > > > > > > > provide a completely standalone Zeppelin with all the > > > > > interpreters > > > > > > > and > > > > > > > > > > environments included? Or is the goal to provide a front > > end > > > to > > > > > > those > > > > > > > > > > environments? > > > > > > > > > > > > > > > > > > > > Maybe that's the decision: which environments and > > > interpreters > > > > > are > > > > > > > > > included > > > > > > > > > > in a standalone distribution. > > > > > > > > > > > > > > > > > > > > paul > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 7:52 AM, IT CTO < > [email protected] > > <javascript:;> > > > > <javascript:;>> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > I think that we can have the interpreters add in the > > build > > > > > system > > > > > > > > using > > > > > > > > > > > compilation parameters (e.g. same as the list in the > > > > > > > zeppelin-env.xml > > > > > > > > > > that > > > > > > > > > > > way the basic build builds only the core interpreters > and > > > the > > > > > > user > > > > > > > > can > > > > > > > > > > > easily interperters to the build process. > > > > > > > > > > > With regard to a release, this can be just as adding > the > > > jar > > > > > and > > > > > > > the > > > > > > > > > > config > > > > > > > > > > > or just the jar with auto-discovery > > > > > > > > > > > Eran > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 1:40 PM DuyHai Doan < > > > > > > [email protected] <javascript:;> <javascript:;>> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > If we want to make the intepreters system modular to > > > > decouple > > > > > > > souce > > > > > > > > > > code, > > > > > > > > > > > > the process of activating an interpreter should be > > > flexible > > > > > and > > > > > > > > easy > > > > > > > > > to > > > > > > > > > > > > use. > > > > > > > > > > > > > > > > > > > > > > > > Asking end-users to make a custom build is not a > viable > > > > > > strategy. > > > > > > > > > > > Recently > > > > > > > > > > > > I've seen some people trying to make a custom build > of > > > > > Zeppelin > > > > > > > and > > > > > > > > > > > facing > > > > > > > > > > > > a lot of issues (incorrect Maven version, no Bower > > > > installed, > > > > > > > > > incorrect > > > > > > > > > > > > repository policies settings for Maven etc ...) > > > > > > > > > > > > > > > > > > > > > > > > Ideally, activating an interpreter should be as > simple > > as > > > > > > > dropping > > > > > > > > a > > > > > > > > > > jar > > > > > > > > > > > > into the lib directory > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 12:28 PM, < > > > > [email protected] <javascript:;> <javascript:;> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > +1 what tog says > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: tog > > > > > > > > > > > > > > > > > > > > > > > > > > Sent: Wednesday, July 22, 12:22 AM > > > > > > > > > > > > > > > > > > > > > > > > > > Subject: Interpreter with no test. > > > > > > > > > > > > > > > > > > > > > > > > > > To: [email protected] > <javascript:;> > > <javascript:;> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > This seems to be a recurrent topic ;-) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It has to be decided what are the core interpreters > > the > > > > > > > Zeppelin > > > > > > > > > want > > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > support - then I would believe you can have a > > > > wiki/webpage > > > > > > > > > dedicated > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > > > listing all interpreters that are known to be > working > > > > with > > > > > > > > > Zeppelin. > > > > > > > > > > > You > > > > > > > > > > > > > > > > > > > > > > > > > > may even consider having recommended plugins and/or > > > > ranking > > > > > > in > > > > > > > > > case 2 > > > > > > > > > > > > > > > > > > > > > > > > > > plugins are doing the same job. Grails (but it is > > > > probably > > > > > > not > > > > > > > > the > > > > > > > > > > only > > > > > > > > > > > > > > > > > > > > > > > > > > one) is implementing this (see > > > > https://grails.org/plugins/ > > > > > ) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In order to keep Zeppelin low in term of footprint > - > > > the > > > > > core > > > > > > > > > > commiters > > > > > > > > > > > > can > > > > > > > > > > > > > > > > > > > > > > > > > > even decide to support some of those plugins (shell > > for > > > > > > > example) > > > > > > > > > > > > > > > > > > > > > > > > > > At one end of the spectrum you could imagine > Zeppelin > > > > > without > > > > > > > any > > > > > > > > > > > > > > > > > > > > > > > > > > interpreters and at the other end Zeppelin with all > > > known > > > > > > > > plugins - > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > truth is probably in the middle. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wednesday, July 22, 2015, Anthony Corbacho < > > > > > > > > > > > > [email protected] <javascript:;> > > <javascript:;> > > > > > > > > > > > > > > > > > > > > > > > > > > <javascript:_e(%7B%7D,'cvml',' > > > [email protected] <javascript:;> > > > > <javascript:;> > > > > > > ');>> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IT CTO is right, right now, zeppelin merge every > > > > > > interpreter > > > > > > > in > > > > > > > > > the > > > > > > > > > > > > code > > > > > > > > > > > > > > > > > > > > > > > > > > > base and its mean that we (committer) have to > take > > > care > > > > > of > > > > > > > > build > > > > > > > > > > > > falling > > > > > > > > > > > > > or > > > > > > > > > > > > > > > > > > > > > > > > > > > questions for those interpreters. and personally > i > > > have > > > > > no > > > > > > > > > > experience > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > > > > > > > > > > presto, cassandra etcetc. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think, It can be interesting to detach > > interpreters > > > > > from > > > > > > > > > zeppelin > > > > > > > > > > > > code > > > > > > > > > > > > > > > > > > > > > > > > > > > base and create a sortof plug'n'play module where > > the > > > > > users > > > > > > > can > > > > > > > > > > plug > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > interpreter he wants to use. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With this approach, we can keep zeppelin code > base > > > > > smaller > > > > > > > (we > > > > > > > > > > > provide > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > core + maybe some interpreters (spark, md, shell) > > as > > > > > > > default). > > > > > > > > > and > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > community can build and manage other interpreters > > (i > > > > > assume > > > > > > > if > > > > > > > > > they > > > > > > > > > > > > build > > > > > > > > > > > > > > > > > > > > > > > > > > > it they have experiences and probably can answer > > > > > > questions). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 2:34 PM, IT CTO < > > > > > [email protected] <javascript:;> <javascript:;> > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the question is what\who is going to > fix > > > > issues > > > > > > in > > > > > > > > > these > > > > > > > > > > > > > > > > > > > > > > > > > > > > Interpreters if something fails? > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am guessing that if one uses these > interpreters > > > and > > > > > > > > approach > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > community with questions then we might not have > > the > > > > > right > > > > > > > > > support > > > > > > > > > > > for > > > > > > > > > > > > > > > > > > > > > > > > > > > him. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 8:04 AM moon soo Lee < > > > > > > > > [email protected] <javascript:;> <javascript:;>> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There're some open pullrequests with complete > > > > > > interpreter > > > > > > > > > > > > > > > > > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > > > > > > > > > > but no test (eg. > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/110 > > > > > > > > > > > > > > > > > > > > > > > > > > > , > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/68 > > > > > ). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Personally, I'm feeling not safe having code > > > > without > > > > > > > test, > > > > > > > > > but > > > > > > > > > > at > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > > > same > > > > > > > > > > > > > > > > > > > > > > > > > > > > > time, keeping these great contributions > > unmerged > > > > > sounds > > > > > > > not > > > > > > > > > > cool. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to hear opinions about merging them > > with > > > > > some > > > > > > > mark > > > > > > > > > > such > > > > > > > > > > > as > > > > > > > > > > > > > > > > > > > > > > > > > > > > 'beta' > > > > > > > > > > > > > > > > > > > > > > > > > > > > > or 'untested', and create issue for adding > > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > moon > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > *Paul Curtis - *Senior Product Specialist - Field > > Enablement > > > > Team > > > > > > > > > > *O: *+1 203-660-0015 - *M:* +1 203-539-9705 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > > > > > > -- > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > -- > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net >
