Consider the Apache Camel project. There are a *ton* of components available within Camel and most everything (with compatible license) is within the Camel project itself, not outside. Keeping the interpreters outside and independent gets into a nightmare situation when new versions come out, because you don't know which ones work with which versions (usually folks end up combining them into some grouping anyway to avoid the fragmentation). You may want to use two different ones in your notebook, but they support completely different versions of the core. Also, as I said, it would be much tougher to grow a community around the interpreters in isolation. It would be much better to bring them into the fold and grow a large community around all of it together. Now, that doesn't mean that you couldn't still keep some outside (zeppelin-extras perhaps?). It also doesn't mean that the core has to follow the same lifecycle as the "canonical" interpreters.
On Wed, May 13, 2015 at 9:34 PM Jongyoul Lee <[email protected]> wrote: > +1 for me. I think we can use a wiki for maintaining third party > interpreters. > > On Thu, May 14, 2015 at 10:23 AM, Alexander Bezzubov <[email protected]> > wrote: > > > Hi, > > > > Damien, thanks for bringing the discussion in, and thank everybody for > > opinions: it is indeed very timely as we see more and more submodules > with > > interpreters implementations and having a consistent strategy on how to > > deal with them is very important now. > > > > What I see right now is: > > - *number of interpreters is potentially unbounded* > > that means that bringing all of them to the root repo is neither > > feasible nor a maintainable solution: we need then not only to make sure > > that patches with such contributions are good but also that ALL > maintainers > > will stick around long enough to support further evolution of API until > at > > least a stable release like 1.0 > > As zeppelin is 0.x and is far from being stable on API level - this > will > > bring us to the situation when the project does not compile because of > some > > interpret is not update really quick. > > > > - *example of other successful extendable systems* > > Other systems like Apache Corduva (while being a Phonegap) or Homebrew > > in early days all had a repos with extension code but then moved to just > a > > 'registry' model, meaning that they just host a registry in 'npm' fashion > > and only provide a tools and a workflow to use\contribute those > extensions. > > I.e Ipython does not host all kernels in main repo rather just list > all > > them in the project wiki > > > > - *importance of keeping Zeppelin codebase small and simple for fast > > iterations* > > Last but not least, this is kind of implication from the first > statement > > but is very important for longevity of the project. The whole field of > > large scale data analytics systems is very dynamic, and it makes perfect > > sense, at least to me to, to keep the focus of the project on delivering > > the core value rather then covering all potential applications. > > > > That being said, if this would be the vote I would support a > > "registry-like" model, with an external codebases for interpreters AND > some > > tools on zeppelin side to simplify management like install\update\delete > > interpreters i.e from Maven or local\remote filesystem URL. > > > > Those changes would require further discussion in case we have a > consensus > > here. > > > > > > On Wed, May 13, 2015 at 9:13 PM, James Carman < > [email protected]> > > wrote: > > > > > Consider this a huge -1 from me > > > On Wed, May 13, 2015 at 7:39 AM James Carman < > [email protected] > > > > > > wrote: > > > > > > > Why not invite the contributors to be committers? You don't want to > > > leave > > > > the contributors out in the cold. Likewise, the interpreters are > less > > > > likely to attract new contributions outside the ASF. You want to > > build a > > > > community around the code. Vote them in! > > > > > > > > On Wed, May 13, 2015 at 4:18 AM Corneau Damien < > [email protected]> > > > > wrote: > > > > > > > >> Hi, > > > >> > > > >> I just want to open a discussion about Zeppelin Interpreters. > > > >> > > > >> Currently we are accepting interpreters and merging them to the > > Zeppelin > > > >> Branch when people have some done. > > > >> > > > >> I see more and more issues/post on the mailing list regarding some > > > >> problems > > > >> with interpreters, while it is a big part of Zeppelin, I think it > will > > > >> become quickly hard to maintain. > > > >> > > > >> It could be a better approach to separate Interpreters from the > build > > > >> (kinda like plugins/package), that people can include easily > depending > > > on > > > >> their needs, and let the creator take care of issues related to it. > > > >> > > > >> Any thoughts? > > > >> > > > > > > > > > > > > > -- > 이종열, Jongyoul Lee, 李宗烈 > http://madeng.net >
