Hi Larry, Thanks for the thoughts. Yes, I realize this is somewhat outside of what may be outside of Knox's main use case as a REST gateway, however I feel it is functionality that could logically be part of the application. We plan to use Knox to satisfy some other security requirements and bundling this functionality in Knox makes sense to our team.
"The users" are system administrators who have sudo access and would like to use a command terminal to do the usual job of system administrators (changing configuration files, restarting services, chowning things in hdfs and local disk, checking on the namenode and resource manager, etc.). I have a requirement that all administrators who maintain the Hadoop cluster have every action logged for audit purposes. One way to do this reliably is to force them to tunnel through a bastion host that they do not control. Presumably by making the bastion administrators different people than the cluster administrators, a higher level of security is achieved. Doing this within Knox makes sense as it will give us one 'pane of glass' for our cluster security requirements. For convenience sake, I would like them to be able to do this without running kinit every time they log into the cluster, however this is a secondary goal. This is what I mean by 'interacting normally with hadoop' - using any shell command the way they do before Knox is in the picture (so hdfs dfs -ls /tmp is one of many examples). I have contemplated two implementations: one is a HTTP based endpoint that violates normal REST conventions by keeping the HTTP session open indefinitely. Another is simply discarding HTTP based interaction and providing an SSH-based tunneling facility. The second option is nice because it "looks" normal or almost normal to the system administrator. Another option could be a Knox CLI extension. On Tue, Feb 4, 2014 at 6:27 PM, larry mccay <[email protected]> wrote: > Hi Ed - > > Thanks for the interesting thoughts. > > Unfortunately, I don't think that have the full picture of what you are > thinking in my mind yet. > > Let's leave out the implementation options and talk purely about the > usecase for a second. > > * You have a set of users (what are their roles - sounds like admins of > some sort?) > * You'd like them to be able to gain access to the cluster through Knox > (Knox is obviously a REST API Gateway) > * What is it that these particular users need to do? > * When you say "the user could interact normally with Hadoop" - do you mean > that they have shell access for running the hadoop CLI's - eg: "hadoop fs > -l /tmp"? > * are you assuming a browser based application for this or is this a Knox > CLI extension? > > When we were first talking about the ClientDSL we briefly considered using > Apache Mina sshd but it didn't seem necessary given that we are able to use > the REST APIs remotely. > > Again, if you can articulate the (a) usecase end to end that would be a > great starting place. > > Thanks again for your ideas! > > I look forward to contributions from you on this. > > --larry > > > > > On Tue, Feb 4, 2014 at 4:22 PM, Ed Kohlwey <[email protected]> wrote: > > > I'm interested in implementing a knox service that implements full shell > > access to hosts within a hadoop cluster. The point of doing so is to > > provide administrative auditing capability for all cluster user accounts, > > including privileged users. A smaller number of users would have access > to > > the knox host than the full cluster. > > > > My current thinking on how to do this is to implement a servlet. Using a > > GET or POST request, the request body would be passed to standard in, and > > the response body would contain standard out. Knox itself would use ssh > > public key based login to connect as a knox user to the other cluster > > hosts, and then use a combinaiton of sudo and exec commands to change the > > shell to the appropriate user. Knox would set up a hadoop delegation > token > > so the user could interact normally with Hadoop. > > > > Another option could be to start a java based SSH server on another > > (non-22) port and perform normal tunneling, probably by adding an > > interactive shell that asks what host the user would like to connect to. > > This has the advantage of basically looking like a normal SSH connection. > > > > I noticed this is not how the Knox DSL is implemented, which seems to use > > some session state. Can anyone discuss the motivations for doing this? > Does > > the community have any opinions on how best to go about providing audited > > secure shell access? > > >
