That would indeed be a nice addition. Apparently moon said it should be possible to drop a jar under interpreter/[name] directory ... that jar would nevertheless need to extend interpreter class so we may want to provide a sample minimal project showing that or a tutorial (the shell interpreter is good for that because simple) Can work with Eran on that.
On 23 July 2015 at 10:31, Corneau Damien <[email protected]> wrote: > Any way to add an interpreter without rebuilding zeppelin? > > On Thu, Jul 23, 2015 at 5:12 PM, DuyHai Doan <[email protected]> wrote: > > > For decoupling the build of core Zeppelin vs third-party interpreters, we > > can use Maven profiles: > > > > in the root pom.xml, you can define in the default profile all the Maven > > modules that should be part of the core Zeppelin build. Additional > > interpreters can be defined in separated profiles so that people keen on > > using them can build them easily. Ex: > > > > <profiles> > > > > <!-- Core profile --> > > <profile> > > <id>core</id> > > <activation> > > <activeByDefault>true</activeByDefault> > > </activation> > > <modules> > > <module>zeppelin-interpreter</module> > > <module>zeppelin-zengine</module> > > <module>spark</module> > > <module>markdown</module> > > <module>angular</module> > > <module>shell</module> > > <module>zeppelin-web</module> > > <module>zeppelin-server</module> > > <module>zeppelin-distribution</module> > > </modules> > > </profile> > > > > <!-- Cassandra interpreter --> > > <profile> > > <id>cassandra</id> > > <modules> > > <module>cassandra</module> > > </modules> > > </profile> > > > > <!-- Lens interpreter --> > > <profile> > > <id>lens</id> > > <modules> > > <module>lens</module> > > </modules> > > </profile> > > </profiles> > > > > To build Zeppelin with support for Cassandra & Lens: > > > > mvn clean package -Pcore,cassandra,lens -D.... > > > > > > What do you think ? > > > > > > On Thu, Jul 23, 2015 at 7:38 AM, IT CTO <[email protected]> wrote: > > > > > I volunteer to write a doc for the zeppelin documentation site on step > by > > > step for adding an "external interpreter" targeted for non developers > who > > > wants to add someone's else external interpreter. > > > The challenge I see is from where to get the compiled interpreter from? > > > - my answer is to have a list of external interpreter on our site which > > > points to the github of the external interpreter and the user will > > download > > > himself. - I think this should help with licensing issues as well as an > > > external site can hold a non apache license interpreter (MySQL, Oracle) > > > > > > Eran > > > > > > > > > On Thu, Jul 23, 2015 at 2:38 AM moon soo Lee <[email protected]> wrote: > > > > > > > Thanks for the opinions and feedback. > > > > > > > > My question was more like simply asking about merging code that does > > not > > > > have test, but apparently went back to recurrent subject. :-) > > > > > > > > Actually, plug'n'play interpreter is already possible by dropping all > > the > > > > necessary jars under interpreter/[name] directory and add config to > > > > zeppelin-site.xml. I think nothing stops making externally managed > > > > interpreter. > > > > > > > > I think having list of all internal + external interpreters in > homepage > > > or > > > > wiki and let user update it would help, as a first step. > > > > > > > > Best, > > > > moon > > > > > > > > On Wed, Jul 22, 2015 at 9:42 PM Paul Curtis <[email protected]> > > > wrote: > > > > > > > > > +1 > > > > > > > > > > I agree with tog ... the core interpreters should be those that can > > > > > complete all the tests and provide the functionality as well. Spark > > > local > > > > > being the best example. The Apache Drill interpreter I wrote would > > > need a > > > > > single node Drill installation in order to provide the same testing > > > > > capability. I am working to provide this. > > > > > > > > > > However, I would suggest that review may be needed. As each test > and > > > > > interpreter adds Zeppelin code, it also adds to the distributable > as > > > > well. > > > > > Currently, the distributable is around ~500MB in size. Is the goal > to > > > > > provide a completely standalone Zeppelin with all the interpreters > > and > > > > > environments included? Or is the goal to provide a front end to > those > > > > > environments? > > > > > > > > > > Maybe that's the decision: which environments and interpreters are > > > > included > > > > > in a standalone distribution. > > > > > > > > > > paul > > > > > > > > > > On Wed, Jul 22, 2015 at 7:52 AM, IT CTO <[email protected]> wrote: > > > > > > > > > > > I think that we can have the interpreters add in the build system > > > using > > > > > > compilation parameters (e.g. same as the list in the > > zeppelin-env.xml > > > > > that > > > > > > way the basic build builds only the core interpreters and the > user > > > can > > > > > > easily interperters to the build process. > > > > > > With regard to a release, this can be just as adding the jar and > > the > > > > > config > > > > > > or just the jar with auto-discovery > > > > > > Eran > > > > > > > > > > > > On Wed, Jul 22, 2015 at 1:40 PM DuyHai Doan < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > If we want to make the intepreters system modular to decouple > > souce > > > > > code, > > > > > > > the process of activating an interpreter should be flexible and > > > easy > > > > to > > > > > > > use. > > > > > > > > > > > > > > Asking end-users to make a custom build is not a viable > strategy. > > > > > > Recently > > > > > > > I've seen some people trying to make a custom build of Zeppelin > > and > > > > > > facing > > > > > > > a lot of issues (incorrect Maven version, no Bower installed, > > > > incorrect > > > > > > > repository policies settings for Maven etc ...) > > > > > > > > > > > > > > Ideally, activating an interpreter should be as simple as > > dropping > > > a > > > > > jar > > > > > > > into the lib directory > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 12:28 PM, <[email protected]> > > > wrote: > > > > > > > > > > > > > > > +1 what tog says > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: tog > > > > > > > > > > > > > > > > Sent: Wednesday, July 22, 12:22 AM > > > > > > > > > > > > > > > > Subject: Interpreter with no test. > > > > > > > > > > > > > > > > To: [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > This seems to be a recurrent topic ;-) > > > > > > > > > > > > > > > > > > > > > > > > It has to be decided what are the core interpreters the > > Zeppelin > > > > want > > > > > > to > > > > > > > > > > > > > > > > support - then I would believe you can have a wiki/webpage > > > > dedicated > > > > > to > > > > > > > > > > > > > > > > listing all interpreters that are known to be working with > > > > Zeppelin. > > > > > > You > > > > > > > > > > > > > > > > may even consider having recommended plugins and/or ranking > in > > > > case 2 > > > > > > > > > > > > > > > > plugins are doing the same job. Grails (but it is probably > not > > > the > > > > > only > > > > > > > > > > > > > > > > one) is implementing this (see https://grails.org/plugins/) > > > > > > > > > > > > > > > > > > > > > > > > In order to keep Zeppelin low in term of footprint - the core > > > > > commiters > > > > > > > can > > > > > > > > > > > > > > > > even decide to support some of those plugins (shell for > > example) > > > > > > > > > > > > > > > > At one end of the spectrum you could imagine Zeppelin without > > any > > > > > > > > > > > > > > > > interpreters and at the other end Zeppelin with all known > > > plugins - > > > > > the > > > > > > > > > > > > > > > > truth is probably in the middle. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wednesday, July 22, 2015, Anthony Corbacho < > > > > > > > [email protected] > > > > > > > > > > > > > > > > <javascript:_e(%7B%7D,'cvml','[email protected] > ');>> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > IT CTO is right, right now, zeppelin merge every > interpreter > > in > > > > the > > > > > > > code > > > > > > > > > > > > > > > > > base and its mean that we (committer) have to take care of > > > build > > > > > > > falling > > > > > > > > or > > > > > > > > > > > > > > > > > questions for those interpreters. and personally i have no > > > > > experience > > > > > > > in > > > > > > > > > > > > > > > > > presto, cassandra etcetc. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think, It can be interesting to detach interpreters from > > > > zeppelin > > > > > > > code > > > > > > > > > > > > > > > > > base and create a sortof plug'n'play module where the users > > can > > > > > plug > > > > > > > the > > > > > > > > > > > > > > > > > interpreter he wants to use. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > With this approach, we can keep zeppelin code base smaller > > (we > > > > > > provide > > > > > > > > the > > > > > > > > > > > > > > > > > core + maybe some interpreters (spark, md, shell) as > > default). > > > > and > > > > > > the > > > > > > > > > > > > > > > > > community can build and manage other interpreters (i assume > > if > > > > they > > > > > > > build > > > > > > > > > > > > > > > > > it they have experiences and probably can answer > questions). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 2:34 PM, IT CTO <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I think the question is what\who is going to fix issues > in > > > > these > > > > > > > > > > > > > > > > > > Interpreters if something fails? > > > > > > > > > > > > > > > > > > I am guessing that if one uses these interpreters and > > > approach > > > > > the > > > > > > > > > > > > > > > > > > community with questions then we might not have the right > > > > support > > > > > > for > > > > > > > > > > > > > > > > > him. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jul 22, 2015 at 8:04 AM moon soo Lee < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > There're some open pullrequests with complete > interpreter > > > > > > > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > but no test (eg. > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/110 > > > > > > > > > > > > > > > > > , > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-zeppelin/pull/68). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Personally, I'm feeling not safe having code without > > test, > > > > but > > > > > at > > > > > > > the > > > > > > > > > > > > > > > > > > same > > > > > > > > > > > > > > > > > > > time, keeping these great contributions unmerged sounds > > not > > > > > cool. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to hear opinions about merging them with some > > mark > > > > > such > > > > > > as > > > > > > > > > > > > > > > > > > 'beta' > > > > > > > > > > > > > > > > > > > or 'untested', and create issue for adding test. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > moon > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > *Paul Curtis - *Senior Product Specialist - Field Enablement Team > > > > > *O: *+1 203-660-0015 - *M:* +1 203-539-9705 > > > > > > > > > > > > > > > Now Available - Free Hadoop On-Demand Training > > > > > < > > > > > > > > > > > > > > > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > > > > > > > > > > > > > > > > > > > > -- PGP KeyID: 2048R/EA31CFC9 subkeys.pgp.net
