Documenting adding external interpreter and making an sample minimal project sounds great!! That would help both users and developers.
About the maven profile DuyHai suggested, how do we decide which interpreter should be in core and which interpreter should not be in? Another way of doing similar action is using -pl flag. # build Zeppelin with spark and flink interpreter mvn package -pl 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-web,spark,fllnk' # build Zeppelin except for spark and flink interpreter mvn package -pl '!spark,!flink' Thanks, moon On 2015년 7월 23일 (목) at 오후 6:43 tog <[email protected]> wrote: > That would indeed be a nice addition. > Apparently moon said it should be possible to drop a jar under > interpreter/[name] directory ... that jar would nevertheless need to extend > interpreter class so we may want to provide a sample minimal project > showing that or a tutorial (the shell interpreter is good for that because > simple) > Can work with Eran on that. > > On 23 July 2015 at 10:31, Corneau Damien <[email protected]> wrote: > > > Any way to add an interpreter without rebuilding zeppelin? > > > > On Thu, Jul 23, 2015 at 5:12 PM, DuyHai Doan <[email protected]> > wrote: > > > > > For decoupling the build of core Zeppelin vs third-party interpreters, > we > > > can use Maven profiles: > > > > > > in the root pom.xml, you can define in the default profile all the > Maven > > > modules that should be part of the core Zeppelin build. Additional > > > interpreters can be defined in separated profiles so that people keen > on > > > using them can build them easily. Ex: > > > > > > <profiles> > > > > > > <!-- Core profile --> > > > <profile> > > > <id>core</id> > > > <activation> > > > <activeByDefault>true</activeByDefault> > > > </activation> > > > <modules> > > > <module>zeppelin-interpreter</module> > > > <module>zeppelin-zengine</module> > > > <module>spark</module> > > > <module>markdown</module> > > > <module>angular</module> > > > <module>shell</module> > > > <module>zeppelin-web</module> > > > <module>zeppelin-server</module> > > > <module>zeppelin-distribution</module> > > > </modules> > > > </profile> > > > > > > <!-- Cassandra interpreter --> > > > <profile> > > > <id>cassandra</id> > > > <modules> > > > <module>cassandra</module> > > > </modules> > > > </profile> > > > > > > <!-- Lens interpreter --> > > > <profile> > > > <id>lens</id> > > > <modules> > > > <module>lens</module> > > > </modules> > > > </profile> > > > </profiles> > > > > > > To build Zeppelin with support for Cassandra & Lens: > > > > > > mvn clean package -Pcore,cassandra,lens -D.... > > > > > > > > > What do you think ? > > > > > > > > > On Thu, Jul 23, 2015 at 7:38 AM, IT CTO <[email protected]> wrote: > > > > > > > I volunteer to write a doc for the zeppelin documentation site on > step > > by > > > > step for adding an "external interpreter" targeted for non developers > > who > > > > wants to add someone's else external interpreter. > > > > The challenge I see is from where to get the compiled interpreter > from? > > > > - my answer is to have a list of external interpreter on our site > which > > > > points to the github of the external interpreter and the user will > > > download > > > > himself. - I think this should help with licensing issues as well as > an > > > > external site can hold a non apache license interpreter (MySQL, > Oracle) > > > > > > > > Eran > > > > > > > > > > > > On Thu, Jul 23, 2015 at 2:38 AM moon soo Lee <[email protected]> > wrote: > > > > > > > > > Thanks for the opinions and feedback. > > > > > > > > > > My question was more like simply asking about merging code that > does > > > not > > > > > have test, but apparently went back to recurrent subject. :-) > > > > > > > > > > Actually, plug'n'play interpreter is already possible by dropping > all > > > the > > > > > necessary jars under interpreter/[name] directory and add config to > > > > > zeppelin-site.xml. I think nothing stops making externally managed > > > > > interpreter. > > > > > > > > > > I think having list of all internal + external interpreters in > > homepage > > > > or > > > > > wiki and let user update it would help, as a first step. > > > > > > > > > > Best, > > > > > moon > > > > > > > > > > On Wed, Jul 22, 2015 at 9:42 PM Paul Curtis <[email protected]> > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > I agree with tog ... the core interpreters should be those that > can > > > > > > complete all the tests and provide the functionality as well. > Spark > > > > local > > > > > > being the best example. The Apache Drill interpreter I wrote > would > > > > need a > > > > > > single node Drill installation in order to provide the same > testing > > > > > > capability. I am working to provide this. > > > > > > > > > > > > However, I would suggest that review may be needed. As each test > > and > > > > > > interpreter adds Zeppelin code, it also adds to the distributable > > as > > > > > well. > > > > > > Currently, the distributable is around ~500MB in size. Is the > goal > > to > > > > > > provide a completely standalone Zeppelin with all the > interpreters > > > and > > > > > > environments included? Or is the goal to provide a front end to > > those > > > > > > environments? > > > > > > > > > > > > Maybe that's the decision: which environments and interpreters > are > > > > > included > > > > > > in a standalone distribution. > > > > > > > > > > > > paul > > > > > > > > > > > > On Wed, Jul 22, 2015 at 7:52 AM, IT CTO <[email protected]> > wrote: > > > > > > > > > > > > > I think that we can have the interpreters add in the build > system > > > > using > > > > > > > compilation parameters (e.g. same as the list in the > > > zeppelin-env.xml > > > > > > that > > > > > > > way the basic build builds only the core interpreters and the > > user > > > > can > > > > > > > easily interperters to the build process. > > > > > > > With regard to a release, this can be just as adding the jar > and > > > the > > > > > > config > > > > > > > or just the jar with auto-discovery > > > > > > > Eran > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 1:40 PM DuyHai Doan < > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > If we want to make the intepreters system modular to decouple > > > souce > > > > > > code, > > > > > > > > the process of activating an interpreter should be flexible > and > > > > easy > > > > > to > > > > > > > > use. > > > > > > > > > > > > > > > > Asking end-users to make a custom build is not a viable > > strategy. > > > > > > > Recently > > > > > > > > I've seen some people trying to make a custom build of > Zeppelin > > > and > > > > > > > facing > > > > > > > > a lot of issues (incorrect Maven version, no Bower installed, > > > > > incorrect > > > > > > > > repository policies settings for Maven etc ...) > > > > > > > > > > > > > > > > Ideally, activating an interpreter should be as simple as > > > dropping > > > > a > > > > > > jar > > > > > > > > into the lib directory > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 12:28 PM, <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 what tog says > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: tog > > > > > > > > > > > > > > > > > > Sent: Wednesday, July 22, 12:22 AM > > > > > > > > > > > > > > > > > > Subject: Interpreter with no test. > > > > > > > > > > > > > > > > > > To: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > This seems to be a recurrent topic ;-) > > > > > > > > > > > > > > > > > > > > > > > > > > > It has to be decided what are the core interpreters the > > > Zeppelin > > > > > want > > > > > > > to > > > > > > > > > > > > > > > > > > support - then I would believe you can have a wiki/webpage > > > > > dedicated > > > > > > to > > > > > > > > > > > > > > > > > > listing all interpreters that are known to be working with > > > > > Zeppelin. > > > > > > > You > > > > > > > > > > > > > > > > > > may even consider having recommended plugins and/or ranking > > in > > > > > case 2 > > > > > > > > > > > > > > > > > > plugins are doing the same job. Grails (but it is probably > > not > > > > the > > > > > > only > > > > > > > > > > > > > > > > > > one) is implementing this (see https://grails.org/plugins/ > ) > > > > > > > > > > > > > > > > > > > > > > > > > > > In order to keep Zeppelin low in term of footprint - the > core > > > > > > commiters > > > > > > > > can > > > > > > > > > > > > > > > > > > even decide to support some of those plugins (shell for > > > example) > > > > > > > > > > > > > > > > > > At one end of the spectrum you could imagine Zeppelin > without > > > any > > > > > > > > > > > > > > > > > > interpreters and at the other end Zeppelin with all known > > > > plugins - > > > > > > the > > > > > > > > > > > > > > > > > > truth is probably in the middle. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wednesday, July 22, 2015, Anthony Corbacho < > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > <javascript:_e(%7B%7D,'cvml','[email protected] > > ');>> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IT CTO is right, right now, zeppelin merge every > > interpreter > > > in > > > > > the > > > > > > > > code > > > > > > > > > > > > > > > > > > > base and its mean that we (committer) have to take care > of > > > > build > > > > > > > > falling > > > > > > > > > or > > > > > > > > > > > > > > > > > > > questions for those interpreters. and personally i have > no > > > > > > experience > > > > > > > > in > > > > > > > > > > > > > > > > > > > presto, cassandra etcetc. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think, It can be interesting to detach interpreters > from > > > > > zeppelin > > > > > > > > code > > > > > > > > > > > > > > > > > > > base and create a sortof plug'n'play module where the > users > > > can > > > > > > plug > > > > > > > > the > > > > > > > > > > > > > > > > > > > interpreter he wants to use. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With this approach, we can keep zeppelin code base > smaller > > > (we > > > > > > > provide > > > > > > > > > the > > > > > > > > > > > > > > > > > > > core + maybe some interpreters (spark, md, shell) as > > > default). > > > > > and > > > > > > > the > > > > > > > > > > > > > > > > > > > community can build and manage other interpreters (i > assume > > > if > > > > > they > > > > > > > > build > > > > > > > > > > > > > > > > > > > it they have experiences and probably can answer > > questions). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 2:34 PM, IT CTO < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the question is what\who is going to fix issues > > in > > > > > these > > > > > > > > > > > > > > > > > > > > Interpreters if something fails? > > > > > > > > > > > > > > > > > > > > I am guessing that if one uses these interpreters and > > > > approach > > > > > > the > > > > > > > > > > > > > > > > > > > > community with questions then we might not have the > right > > > > > support > > > > > > > for > > > > > > > > > > > > > > > > > > > him. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 8:04 AM moon soo Lee < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There're some open pullrequests with complete > > interpreter > > > > > > > > > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > > but no test (eg. > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/110 > > > > > > > > > > > > > > > > > > > , > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/68 > ). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Personally, I'm feeling not safe having code without > > > test, > > > > > but > > > > > > at > > > > > > > > the > > > > > > > > > > > > > > > > > > > > same > > > > > > > > > > > > > > > > > > > > > time, keeping these great contributions unmerged > sounds > > > not > > > > > > cool. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to hear opinions about merging them with > some > > > mark > > > > > > such > > > > > > > as > > > > > > > > > > > > > > > > > > > > 'beta' > > > > > > > > > > > > > > > > > > > > > or 'untested', and create issue for adding test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > moon > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > *Paul Curtis - *Senior Product Specialist - Field Enablement Team > > > > > > *O: *+1 203-660-0015 - *M:* +1 203-539-9705 > > > > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > > > > < > > > > > > > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net >
