Hi Larry, The basic premise would be to be able to open a knox shell within the notebook as follow:
%knox Hdfs.rm(session).file(“/path/to/file”).now() knox host, port and credentials would need to be set in the plug-in configuration. This would allow directly client interaction with the cluster and leveraging the shell api within the notebook. —Pierre > On Jan 30, 2017, at 9:29 AM, larry mccay <[email protected]> wrote: > > Hi Sumit - > > Thanks for the check point summary and DISCUSS thread. > The summary actually sounds like we are really making some good progress - > which I knew but hadn't seen it put all together like this! > > 1. What are the use cases driving the Zeppelin interpreter? How is > that expected to be used and how can we make it easy to use out of > box? > > <ljm> > Excellent question. Much of what we can do with the DSL in the interpreter is > available in other interpreters. > The DSL has async operations which are handy and a similar programming mode > across all the APIs - due to the fluent code style of the DSL. > One of the advantages of using the DSL over other CLI type approaches is the > ability to source control the scripts - zeppelin would also provide a similar > way to do this with notebooks. > > @Pierre - this was a suggestion that you made to me a while ago. > Can you articulate the value add that you envision for it? > </ljm> > > 2. Do we need a release module for KnoxShell? How do we want to > provide the download to users? > > <ljm> > I believe that we do at some point and probably before we go to a 1.0 release > for Knox. > If we could add this for 0.12.0 as an early attempt that would be great and > shouldn't be that difficult. > </ljm> > > 3. Do we need all of KIP-4 in to call this complete or is what we have > so far in the works good enough for 0.12.0? > > <ljm> > I don't believe that 0.12.0 has to be blocked by KIP-4. > As with KIP-1 (LDAP Improvements), they are used as the driving usecases for > the releases but can and will continue to need work and completion beyond the > initial target release. Focusing this way seems to be providing a great way > to bootstrap progress in specific areas that can continue to be completed and > improve from release to release. > > I do want to get the #2 improvement from KIP-4 (Token service and credential > collector) feature branch merged for 0.12.0. > I think this opens up lots of possibilities and will be great to get some > early adopters. > </ljm> > > I'm sure there are more questions to be had. I am excited by the > uptake of the client DSL library and its usefulness to end users. I > hope we can make it more useful and easier to consume in 0.12.0. > > <ljm> > I am also really excited about these improvements and uptake. > As we move to more and more cloud deployment scenarios, this aspect of Knox > is going to be more and more important. > > One thing that I would really love to have articulated are some usecases that > currently require SSH access by data workers that could be done through the > KnoxShell and eliminate the need for SSH. Without some of these usecases we > will likely fall short by a task or two and it will be difficult to cut off > SSH. > </ljm> > > On Mon, Jan 30, 2017 at 11:38 AM, sumit gupta <[email protected] > <mailto:[email protected]>> wrote: > Hey everyone, > > The list of JIRAs for 0.12.0 have steadily increased over the last few > weeks. We have also had a lot of great activity and contributions > related to KIP-4 and KnoxShell improvements. I wanted to start a > discuss thread to tie things up a little bit for a reasonable > deliverable in this area in the 0.12.0 release. > > Just to reiterate where we are: > > We have had a lot of contributions that can be mapped to KIP-4 goals, > especially improvement number 4 in the list of improvements on KIP-4. > > I believe Larry Mccay has a feature branch going for improvement number 2. > > I have taken a stab at a Zeppelin interpreter (improvement number 3) > in a forked zeppelin repo that can be found here (the branch is > 'knoxshell-interpreter'): > > https://github.com/sumitg/zeppelin/tree/knoxshell-interpreter > <https://github.com/sumitg/zeppelin/tree/knoxshell-interpreter> > > and we have added some tests as part of KNOX-845 (improvement number 5). > > Some open questions I have: > > 1. What are the use cases driving the Zeppelin interpreter? How is > that expected to be used and how can we make it easy to use out of > box? > > 2. Do we need a release module for KnoxShell? How do we want to > provide the download to users? > > 3. Do we need all of KIP-4 in to call this complete or is what we have > so far in the works good enough for 0.12.0? > > I'm sure there are more questions to be had. I am excited by the > uptake of the client DSL library and its usefulness to end users. I > hope we can make it more useful and easier to consume in 0.12.0. > > Thanks, > Sumit >
