Hi moon I believe popularity can be measured by (unique) downloads and/or starring by users. Probably tracking the # of downloads is easier.
Cheers Guillaume On Sunday, July 26, 2015, moon soo Lee <[email protected]> wrote: > Thanks for the opinion. How do you define popularity? > > Best, > moon > On 2015년 7월 24일 (금) at 오후 2:52 tog <[email protected] > <javascript:;>> wrote: > > > >About the maven profile DuyHai suggested, how do we decide which > > >interpreter should be in core and which interpreter should not be in? > > > > Several strategies here ;-) > > > > My initial take would be: > > - > > (1) Core: > > 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-web' > > - (2) Externals but supported by Zeppelin project: spark, PySpark. > > - (3) All others externals at the beginning - then you could move > things > > between (2) and (3) ie deprecate plugins that are getting less popular > and > > promote plugins that are getting more tractions > > > > On Friday, July 24, 2015, moon soo Lee <[email protected] <javascript:;>> > wrote: > > > > > Documenting adding external interpreter and making an sample minimal > > > project sounds great!! That would help both users and developers. > > > > > > About the maven profile DuyHai suggested, how do we decide which > > > interpreter should be in core and which interpreter should not be in? > > > > > > Another way of doing similar action is using -pl flag. > > > > > > # build Zeppelin with spark and flink interpreter > > > mvn package -pl > > > > > > > > > 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-web,spark,fllnk' > > > > > > # build Zeppelin except for spark and flink interpreter > > > mvn package -pl '!spark,!flink' > > > > > > > > > Thanks, > > > moon > > > > > > On 2015년 7월 23일 (목) at 오후 6:43 tog <[email protected] > <javascript:;> > > > <javascript:;>> wrote: > > > > > > > That would indeed be a nice addition. > > > > Apparently moon said it should be possible to drop a jar under > > > > interpreter/[name] directory ... that jar would nevertheless need to > > > extend > > > > interpreter class so we may want to provide a sample minimal project > > > > showing that or a tutorial (the shell interpreter is good for that > > > because > > > > simple) > > > > Can work with Eran on that. > > > > > > > > On 23 July 2015 at 10:31, Corneau Damien <[email protected] > <javascript:;> > > > <javascript:;>> wrote: > > > > > > > > > Any way to add an interpreter without rebuilding zeppelin? > > > > > > > > > > On Thu, Jul 23, 2015 at 5:12 PM, DuyHai Doan <[email protected] > <javascript:;> > > > <javascript:;>> > > > > wrote: > > > > > > > > > > > For decoupling the build of core Zeppelin vs third-party > > > interpreters, > > > > we > > > > > > can use Maven profiles: > > > > > > > > > > > > in the root pom.xml, you can define in the default profile all > the > > > > Maven > > > > > > modules that should be part of the core Zeppelin build. > Additional > > > > > > interpreters can be defined in separated profiles so that people > > keen > > > > on > > > > > > using them can build them easily. Ex: > > > > > > > > > > > > <profiles> > > > > > > > > > > > > <!-- Core profile --> > > > > > > <profile> > > > > > > <id>core</id> > > > > > > <activation> > > > > > > <activeByDefault>true</activeByDefault> > > > > > > </activation> > > > > > > <modules> > > > > > > <module>zeppelin-interpreter</module> > > > > > > <module>zeppelin-zengine</module> > > > > > > <module>spark</module> > > > > > > <module>markdown</module> > > > > > > <module>angular</module> > > > > > > <module>shell</module> > > > > > > <module>zeppelin-web</module> > > > > > > <module>zeppelin-server</module> > > > > > > <module>zeppelin-distribution</module> > > > > > > </modules> > > > > > > </profile> > > > > > > > > > > > > <!-- Cassandra interpreter --> > > > > > > <profile> > > > > > > <id>cassandra</id> > > > > > > <modules> > > > > > > <module>cassandra</module> > > > > > > </modules> > > > > > > </profile> > > > > > > > > > > > > <!-- Lens interpreter --> > > > > > > <profile> > > > > > > <id>lens</id> > > > > > > <modules> > > > > > > <module>lens</module> > > > > > > </modules> > > > > > > </profile> > > > > > > </profiles> > > > > > > > > > > > > To build Zeppelin with support for Cassandra & Lens: > > > > > > > > > > > > mvn clean package -Pcore,cassandra,lens -D.... > > > > > > > > > > > > > > > > > > What do you think ? > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 7:38 AM, IT CTO <[email protected] > <javascript:;> > > > <javascript:;>> wrote: > > > > > > > > > > > > > I volunteer to write a doc for the zeppelin documentation site > on > > > > step > > > > > by > > > > > > > step for adding an "external interpreter" targeted for non > > > developers > > > > > who > > > > > > > wants to add someone's else external interpreter. > > > > > > > The challenge I see is from where to get the compiled > interpreter > > > > from? > > > > > > > - my answer is to have a list of external interpreter on our > site > > > > which > > > > > > > points to the github of the external interpreter and the user > > will > > > > > > download > > > > > > > himself. - I think this should help with licensing issues as > well > > > as > > > > an > > > > > > > external site can hold a non apache license interpreter (MySQL, > > > > Oracle) > > > > > > > > > > > > > > Eran > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 2:38 AM moon soo Lee <[email protected] > <javascript:;> > > > <javascript:;>> > > > > wrote: > > > > > > > > > > > > > > > Thanks for the opinions and feedback. > > > > > > > > > > > > > > > > My question was more like simply asking about merging code > that > > > > does > > > > > > not > > > > > > > > have test, but apparently went back to recurrent subject. :-) > > > > > > > > > > > > > > > > Actually, plug'n'play interpreter is already possible by > > dropping > > > > all > > > > > > the > > > > > > > > necessary jars under interpreter/[name] directory and add > > config > > > to > > > > > > > > zeppelin-site.xml. I think nothing stops making externally > > > managed > > > > > > > > interpreter. > > > > > > > > > > > > > > > > I think having list of all internal + external interpreters > in > > > > > homepage > > > > > > > or > > > > > > > > wiki and let user update it would help, as a first step. > > > > > > > > > > > > > > > > Best, > > > > > > > > moon > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 9:42 PM Paul Curtis < > > > [email protected] <javascript:;> <javascript:;>> > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > I agree with tog ... the core interpreters should be those > > that > > > > can > > > > > > > > > complete all the tests and provide the functionality as > well. > > > > Spark > > > > > > > local > > > > > > > > > being the best example. The Apache Drill interpreter I > wrote > > > > would > > > > > > > need a > > > > > > > > > single node Drill installation in order to provide the same > > > > testing > > > > > > > > > capability. I am working to provide this. > > > > > > > > > > > > > > > > > > However, I would suggest that review may be needed. As each > > > test > > > > > and > > > > > > > > > interpreter adds Zeppelin code, it also adds to the > > > distributable > > > > > as > > > > > > > > well. > > > > > > > > > Currently, the distributable is around ~500MB in size. Is > the > > > > goal > > > > > to > > > > > > > > > provide a completely standalone Zeppelin with all the > > > > interpreters > > > > > > and > > > > > > > > > environments included? Or is the goal to provide a front > end > > to > > > > > those > > > > > > > > > environments? > > > > > > > > > > > > > > > > > > Maybe that's the decision: which environments and > > interpreters > > > > are > > > > > > > > included > > > > > > > > > in a standalone distribution. > > > > > > > > > > > > > > > > > > paul > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 7:52 AM, IT CTO <[email protected] > <javascript:;> > > > <javascript:;>> > > > > wrote: > > > > > > > > > > > > > > > > > > > I think that we can have the interpreters add in the > build > > > > system > > > > > > > using > > > > > > > > > > compilation parameters (e.g. same as the list in the > > > > > > zeppelin-env.xml > > > > > > > > > that > > > > > > > > > > way the basic build builds only the core interpreters and > > the > > > > > user > > > > > > > can > > > > > > > > > > easily interperters to the build process. > > > > > > > > > > With regard to a release, this can be just as adding the > > jar > > > > and > > > > > > the > > > > > > > > > config > > > > > > > > > > or just the jar with auto-discovery > > > > > > > > > > Eran > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 1:40 PM DuyHai Doan < > > > > > [email protected] <javascript:;> <javascript:;>> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > If we want to make the intepreters system modular to > > > decouple > > > > > > souce > > > > > > > > > code, > > > > > > > > > > > the process of activating an interpreter should be > > flexible > > > > and > > > > > > > easy > > > > > > > > to > > > > > > > > > > > use. > > > > > > > > > > > > > > > > > > > > > > Asking end-users to make a custom build is not a viable > > > > > strategy. > > > > > > > > > > Recently > > > > > > > > > > > I've seen some people trying to make a custom build of > > > > Zeppelin > > > > > > and > > > > > > > > > > facing > > > > > > > > > > > a lot of issues (incorrect Maven version, no Bower > > > installed, > > > > > > > > incorrect > > > > > > > > > > > repository policies settings for Maven etc ...) > > > > > > > > > > > > > > > > > > > > > > Ideally, activating an interpreter should be as simple > as > > > > > > dropping > > > > > > > a > > > > > > > > > jar > > > > > > > > > > > into the lib directory > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 12:28 PM, < > > > [email protected] <javascript:;> <javascript:;> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > +1 what tog says > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: tog > > > > > > > > > > > > > > > > > > > > > > > > Sent: Wednesday, July 22, 12:22 AM > > > > > > > > > > > > > > > > > > > > > > > > Subject: Interpreter with no test. > > > > > > > > > > > > > > > > > > > > > > > > To: [email protected] <javascript:;> > <javascript:;> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > This seems to be a recurrent topic ;-) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It has to be decided what are the core interpreters > the > > > > > > Zeppelin > > > > > > > > want > > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > support - then I would believe you can have a > > > wiki/webpage > > > > > > > > dedicated > > > > > > > > > to > > > > > > > > > > > > > > > > > > > > > > > > listing all interpreters that are known to be working > > > with > > > > > > > > Zeppelin. > > > > > > > > > > You > > > > > > > > > > > > > > > > > > > > > > > > may even consider having recommended plugins and/or > > > ranking > > > > > in > > > > > > > > case 2 > > > > > > > > > > > > > > > > > > > > > > > > plugins are doing the same job. Grails (but it is > > > probably > > > > > not > > > > > > > the > > > > > > > > > only > > > > > > > > > > > > > > > > > > > > > > > > one) is implementing this (see > > > https://grails.org/plugins/ > > > > ) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In order to keep Zeppelin low in term of footprint - > > the > > > > core > > > > > > > > > commiters > > > > > > > > > > > can > > > > > > > > > > > > > > > > > > > > > > > > even decide to support some of those plugins (shell > for > > > > > > example) > > > > > > > > > > > > > > > > > > > > > > > > At one end of the spectrum you could imagine Zeppelin > > > > without > > > > > > any > > > > > > > > > > > > > > > > > > > > > > > > interpreters and at the other end Zeppelin with all > > known > > > > > > > plugins - > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > truth is probably in the middle. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wednesday, July 22, 2015, Anthony Corbacho < > > > > > > > > > > > [email protected] <javascript:;> > <javascript:;> > > > > > > > > > > > > > > > > > > > > > > > > <javascript:_e(%7B%7D,'cvml',' > > [email protected] <javascript:;> > > > <javascript:;> > > > > > ');>> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IT CTO is right, right now, zeppelin merge every > > > > > interpreter > > > > > > in > > > > > > > > the > > > > > > > > > > > code > > > > > > > > > > > > > > > > > > > > > > > > > base and its mean that we (committer) have to take > > care > > > > of > > > > > > > build > > > > > > > > > > > falling > > > > > > > > > > > > or > > > > > > > > > > > > > > > > > > > > > > > > > questions for those interpreters. and personally i > > have > > > > no > > > > > > > > > experience > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > > > > > > > > presto, cassandra etcetc. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think, It can be interesting to detach > interpreters > > > > from > > > > > > > > zeppelin > > > > > > > > > > > code > > > > > > > > > > > > > > > > > > > > > > > > > base and create a sortof plug'n'play module where > the > > > > users > > > > > > can > > > > > > > > > plug > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > interpreter he wants to use. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With this approach, we can keep zeppelin code base > > > > smaller > > > > > > (we > > > > > > > > > > provide > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > core + maybe some interpreters (spark, md, shell) > as > > > > > > default). > > > > > > > > and > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > community can build and manage other interpreters > (i > > > > assume > > > > > > if > > > > > > > > they > > > > > > > > > > > build > > > > > > > > > > > > > > > > > > > > > > > > > it they have experiences and probably can answer > > > > > questions). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 2:34 PM, IT CTO < > > > > [email protected] <javascript:;> <javascript:;> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the question is what\who is going to fix > > > issues > > > > > in > > > > > > > > these > > > > > > > > > > > > > > > > > > > > > > > > > > Interpreters if something fails? > > > > > > > > > > > > > > > > > > > > > > > > > > I am guessing that if one uses these interpreters > > and > > > > > > > approach > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > community with questions then we might not have > the > > > > right > > > > > > > > support > > > > > > > > > > for > > > > > > > > > > > > > > > > > > > > > > > > > him. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 8:04 AM moon soo Lee < > > > > > > > [email protected] <javascript:;> <javascript:;>> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There're some open pullrequests with complete > > > > > interpreter > > > > > > > > > > > > > > > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > > > > > > > > but no test (eg. > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/110 > > > > > > > > > > > > > > > > > > > > > > > > > , > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/68 > > > > ). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Personally, I'm feeling not safe having code > > > without > > > > > > test, > > > > > > > > but > > > > > > > > > at > > > > > > > > > > > the > > > > > > > > > > > > > > > > > > > > > > > > > > same > > > > > > > > > > > > > > > > > > > > > > > > > > > time, keeping these great contributions > unmerged > > > > sounds > > > > > > not > > > > > > > > > cool. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to hear opinions about merging them > with > > > > some > > > > > > mark > > > > > > > > > such > > > > > > > > > > as > > > > > > > > > > > > > > > > > > > > > > > > > > 'beta' > > > > > > > > > > > > > > > > > > > > > > > > > > > or 'untested', and create issue for adding > test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > > > > moon > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > *Paul Curtis - *Senior Product Specialist - Field > Enablement > > > Team > > > > > > > > > *O: *+1 203-660-0015 - *M:* +1 203-539-9705 > > > > > > > > > > > > > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > -- > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > -- PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net
