> @Mike Yeah, I agree. The Jira for that doesn't exist yet, pretty much pending this exact conversation winding down.
Yup, understood. On Fri, Nov 30, 2018 at 7:36 AM Justin Leet <justinjl...@gmail.com> wrote: > To start with, I'm thinking just the configuration, in particular anything > that touches the ConfigurationsUtils. I think for the first pass, we could > leave reading from configs out of it and focus on who can write configs > (since that's more disruptive to the system). > > To the best of my knowledge, this is: > > - Configuration through the Management UI > - Configuration through the CLI (e.g. the push script + Stellar + any > other scripts that have to do it). > > In terms of how fine-grained, I would essentially expect this to be at the > users+groups level and the individual sensor / topology level (e.g. parsers > can be authorized individually by users/group, indexing is on the whole). > Essentially, I don't propose trying to rearchitect things to try to have > multiple indexing topologies for complete separation, etc. Just to take > the existing setup and be able to apply authorization on top of it. > > I would expect anything covered by the ZK configs to be done as part of > this effort (possibly in stages). As noted, I would expect this to be a > feature branch rather than piecemeal replacement. > > @Mike Yeah, I agree. The Jira for that doesn't exist yet, pretty much > pending this exact conversation winding down. > > On Tue, Nov 20, 2018 at 12:08 PM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > Justin, It probably makes sense to provide a list of these configuration > > items as subtasks on the FB Jira so that we can crosscheck what entry > > points have been implemented against the test scripts. Do you think this > > will impact streaming enrichments or the profiler at all? That is to say, > > as Ali asked, just how far are you looking to take the fine grained auth > > scope for this? > > > > M > > > > On Mon, Nov 19, 2018 at 11:37 PM Ali Nazemian <alinazem...@gmail.com> > > wrote: > > > > > Hi Justin, > > > > > > By configuration do you mean the sensor related configurations only? > Are > > > you limiting the scope of this activity to the management-UI or also > > > Alert-UI as well? For example, defining different roles (pre-defined > > > or customizable) and the fine-grained integration with Ranger? > > > > > > Cheers, > > > Ali > > > > > > On Wed, Nov 14, 2018 at 1:25 AM Justin Leet <justinjl...@gmail.com> > > wrote: > > > > > > > Hi all, > > > > > > > > Right now, our various configs can be modified by anyone with access > to > > > the > > > > various scripts. I'd like to start a discussion around building out > > some > > > > authorization to be able to add some more fine grained controls > around > > > > this. > > > > > > > > Other projects have some variants on how to accomplish this. > > Typically, > > > > this follows a pattern of calling out to a interface/class that takes > > in > > > > the operation and context (user and other info) and returns > true/false > > if > > > > something is authorized. > > > > > > > > In my mind, what we would need out of this is > > > > > > > > - Ability to apply fine grained permissions > > > > - The various scripts and UI should flow this authorization > > framework. > > > > I believe most (if not all) of our configuration flows through > > > > ConfigurationsUtils. Anything that doesn't should either be > hooked > > in > > > > or > > > > refactored to flow through the same codepaths. > > > > - Pluggability. We shouldn't force only one authorizer. > > > > > > > > In particular, I'm proposing we use Apache Ranger > > > > <https://ranger.apache.org/> as a supported authorization framework, > > > > implementing it alongside the authorization framework to validate > what > > we > > > > build. In my mind, the main catch with Ranger is that, based on my > > > > understanding, we won't be able to restrict users with direct access > to > > > > ZooKeeper via it's CLI (e.g. Ranger can't mirror it's ACLs down into > > ZK's > > > > ACLs). I believe this is a reasonable restriction, especially as the > > > > management UI gets improved to handle more of the configuration > burden > > > and > > > > the number of users with access to ZK CLI begins to decrease. Users > > can > > > > still add ZK ACLs separately to enforce that access there. > > > > > > > > For anyone not familiar with Ranger, essentially you build a plugin > > that > > > > hooks into the existing component's authorization framework (e.g. for > > > > Storm, the plugin runs through the IAuthorizer > > > > < > > > > > > > > > > https://storm.apache.org/releases/1.2.2/javadocs/org/apache/storm/security/auth/IAuthorizer.html > > > > > > > > > interface, for Yarn it runs through YarnAuthorizationProvider > > > > < > > > > > > > > > > http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-common/apidocs/org/apache/hadoop/yarn/security/YarnAuthorizationProvider.html > > > > >). > > > > Additionally, Ranger provides auditing capabilities for this > > > authorization > > > > and has plugins for a good portion of our stack already (so users can > > > > already setup ACLs on HDFS, Storm, etc.). Checkout the Ranger Github > > > > <https://github.com/apache/ranger> for a list of the plugins they > have > > > > built in. > > > > > > > > What this means for Metron is building out an authorization setup > > similar > > > > to Storm or Yarn or whatever we choose. We'll want this anyway, to > > allow > > > > our solution to be pluggable. At that point, we build an > > implementation > > > of > > > > the authorizer compatible with Ranger along with the plugin. > > > > > > > > I think this could probably be a fairly small feature branch, which > I'm > > > > suggesting primarily to do the Ranger implementation alongside the > > > general > > > > authorization work to validate what's being built. I think the main > > > > tasking would be something similar to: > > > > > > > > - Build out pluggable authorization for our configs. > > > > - This includes testing (and possibly doing something similar to > > > Storm, > > > > where they have a some testing IAuthorizers, e.g. NoopAuthorizer, > > > > DenyAuthorizer, etc.) > > > > - Ensure that all the code paths consistently flow through this > > > > Authorization. > > > > - Build a Ranger compatible version of this. > > > > - Define the Ranger plugin for this. > > > > - Make sure auditing is defined. > > > > - Integration testing (particularly with Kerberos. After all, if > > they > > > > want to do authorization and auditing, they're almost certainly > > using > > > > Kerberos). > > > > > > > > Is there anything missing that we'd need or want for this? Are there > > any > > > > other concerns we'd want to make sure are taken care of? > > > > > > > > > > > > > -- > > > A.Nazemian > > > > > >
