Hi Pierre - Yes, Sumit has already implemented an interpreter to do that.
I am more interested in what you see as the value-add or the reason that someone would want to use the Knox DSL instead of existing interpreters for largely the same access. >From a language perspective the DSL does have closures for async operations which are handy but I think they are available in things like scala as well. If you haven't had a particular itch in mind that the DSL scratches then maybe it is fine to say that we have the ability to use the same DSL within a notebook as we are from your desktop. We would likely want to be able to provide some sort of visualization as is generally wanted in notebooks. Can we combine visualizations that are already available in Zeppelin for other interpreters with out own? I've seen that there is a MD interpreter for instance - can that be used to format render JSON results in a table for instance? What would we want to use as a showcase script in Zeppelin? thanks! --larry On Mon, Jan 30, 2017 at 7:57 PM, Pierre Regazzoni <[email protected]> wrote: > Hi Larry, > > The basic premise would be to be able to open a knox shell within the > notebook as follow: > > %knox > Hdfs.rm(session).file(“/path/to/file”).now() > > knox host, port and credentials would need to be set in the plug-in > configuration. > > This would allow directly client interaction with the cluster and > leveraging the shell api within the notebook. > > —Pierre > > > On Jan 30, 2017, at 9:29 AM, larry mccay <[email protected]> wrote: > > Hi Sumit - > > Thanks for the check point summary and DISCUSS thread. > The summary actually sounds like we are really making some good progress - > which I knew but hadn't seen it put all together like this! > > 1. What are the use cases driving the Zeppelin interpreter? How is > that expected to be used and how can we make it easy to use out of > box? > > <ljm> > Excellent question. Much of what we can do with the DSL in the interpreter > is available in other interpreters. > The DSL has async operations which are handy and a similar programming > mode across all the APIs - due to the fluent code style of the DSL. > One of the advantages of using the DSL over other CLI type approaches is > the ability to source control the scripts - zeppelin would also provide a > similar way to do this with notebooks. > > @Pierre - this was a suggestion that you made to me a while ago. > Can you articulate the value add that you envision for it? > </ljm> > > 2. Do we need a release module for KnoxShell? How do we want to > provide the download to users? > > <ljm> > I believe that we do at some point and probably before we go to a 1.0 > release for Knox. > If we could add this for 0.12.0 as an early attempt that would be great > and shouldn't be that difficult. > </ljm> > > 3. Do we need all of KIP-4 in to call this complete or is what we have > so far in the works good enough for 0.12.0? > > <ljm> > I don't believe that 0.12.0 has to be blocked by KIP-4. > As with KIP-1 (LDAP Improvements), they are used as the driving usecases > for the releases but can and will continue to need work and completion > beyond the initial target release. Focusing this way seems to be providing > a great way to bootstrap progress in specific areas that can continue to be > completed and improve from release to release. > > I do want to get the #2 improvement from KIP-4 (Token service and > credential collector) feature branch merged for 0.12.0. > I think this opens up lots of possibilities and will be great to get some > early adopters. > </ljm> > > I'm sure there are more questions to be had. I am excited by the > uptake of the client DSL library and its usefulness to end users. I > hope we can make it more useful and easier to consume in 0.12.0. > > <ljm> > I am also really excited about these improvements and uptake. > As we move to more and more cloud deployment scenarios, this aspect of > Knox is going to be more and more important. > > One thing that I would really love to have articulated are some usecases > that currently require SSH access by data workers that could be done > through the KnoxShell and eliminate the need for SSH. Without some of these > usecases we will likely fall short by a task or two and it will be > difficult to cut off SSH. > </ljm> > > On Mon, Jan 30, 2017 at 11:38 AM, sumit gupta <[email protected]> wrote: > >> Hey everyone, >> >> The list of JIRAs for 0.12.0 have steadily increased over the last few >> weeks. We have also had a lot of great activity and contributions >> related to KIP-4 and KnoxShell improvements. I wanted to start a >> discuss thread to tie things up a little bit for a reasonable >> deliverable in this area in the 0.12.0 release. >> >> Just to reiterate where we are: >> >> We have had a lot of contributions that can be mapped to KIP-4 goals, >> especially improvement number 4 in the list of improvements on KIP-4. >> >> I believe Larry Mccay has a feature branch going for improvement number 2. >> >> I have taken a stab at a Zeppelin interpreter (improvement number 3) >> in a forked zeppelin repo that can be found here (the branch is >> 'knoxshell-interpreter'): >> >> https://github.com/sumitg/zeppelin/tree/knoxshell-interpreter >> >> and we have added some tests as part of KNOX-845 (improvement number 5). >> >> Some open questions I have: >> >> 1. What are the use cases driving the Zeppelin interpreter? How is >> that expected to be used and how can we make it easy to use out of >> box? >> >> 2. Do we need a release module for KnoxShell? How do we want to >> provide the download to users? >> >> 3. Do we need all of KIP-4 in to call this complete or is what we have >> so far in the works good enough for 0.12.0? >> >> I'm sure there are more questions to be had. I am excited by the >> uptake of the client DSL library and its usefulness to end users. I >> hope we can make it more useful and easier to consume in 0.12.0. >> >> Thanks, >> Sumit >> > > >
