Bosco, I did create the review board here - https://reviews.apache.org/r/46995/ Didn't really know the process so I also posted the patch on the JIRA.
Sailaja, Thanks for looking into it and providing the recommendations. I can take a shot at that tomorrow and submit a new patch/review. Thanks, Bryan On Wed, May 4, 2016 at 5:34 PM, Don Bosco Durai <[email protected]> wrote: > Sailaja, can you review the patch by Bryan. If it unblocks him, then we > can commit it and come with a better solution in later release? > > Bryan, generally we use review board for patch review. It is easier for > everyone to give comment. > > Thanks > > Bosco > > > > > > On 5/4/16, 2:18 PM, "Sailaja Polavarapu" <[email protected]> > wrote: > > >After following the code in the ranger admin for adding portal users, > following are some of the observations to be considered: > >1. In the DB level email field can be null > >2. In UserMrg.java, if the emailAddress (from the request) is null or > empty, we are generating a randomUID and assigning using that email address > to create or update portal user. Of course in this case we are not doing > any validation checks. > > > >One suggestion I have here is, instead of removing the validation checks > for email address - > >1. Remove auto population of email address in the ranger usersync for > File source (and other sync sources if needed) > >2. Remove auto population of email address with randomUID in ranger admin > code. > >3. Do validation only when email address is not null & not empty. > > > >Thanks, > >Sailaja. > > > > > > > > > >On 5/4/16, 10:01 AM, "Bryan Bende" <[email protected]> wrote: > > > >>Ok I noticed it wasn't required in the UI so that makes sense. > >> > >>Let me see what I can do, I'll create a JIRA. > >> > >>On Wed, May 4, 2016 at 12:52 PM, Don Bosco Durai <[email protected]> > wrote: > >> > >>> Emails are no long used in Ranger. I feel, we should disable all > >>> validations for email. If it is quick for you, do you want to quickly > >>> remove the validation and upload the patch to be committed? > >>> > >>> Thanks > >>> > >>> Bosco > >>> > >>> > >>> > >>> > >>> > >>> On 5/4/16, 9:02 AM, "Bryan Bende" <[email protected]> wrote: > >>> > >>> >Ramesh, > >>> > > >>> >I am working off the master branch, just pulled latest code from > about an > >>> >hour ago. > >>> > > >>> >Thanks, > >>> > > >>> >Bryan > >>> > > >>> > > >>> >On Wed, May 4, 2016 at 11:50 AM, Ramesh Mani <[email protected]> > >>> wrote: > >>> > > >>> >> Bryan, > >>> >> > >>> >> Which version of the ranger you are trying this? > >>> >> > >>> >> Thanks, > >>> >> Ramesh > >>> >> > >>> >> On 5/4/16, 8:22 AM, "Bryan Bende" <[email protected]> wrote: > >>> >> > >>> >> >Sure I will create a JIRA about the validation of the usernames. > >>> >> > > >>> >> >I was able to get the user sync running with the file source and > >>> >> >encountered the following... I created a file usergroups.csv based > off > >>> >> > > >>> >> > >>> > https://cwiki.apache.org/confluence/display/RANGER/File+Source+User+Group+ > >>> >> >Sync+process > >>> >> >: > >>> >> > > >>> >> >"bob", > >>> >> >"john", > >>> >> >"cn=bbende,dc=example,dc=org", > >>> >> > > >>> >> >When the user sync process runs it gets: > >>> >> > > >>> >> >016-05-04 17:01:28,318 [http-bio-6080-exec-6] INFO > >>> >> > org.apache.ranger.biz.UserMgr (UserMgr.java:1140) - > >>> create:bob@localhost > >>> >> >2016-05-04 17:01:28,318 [http-bio-6080-exec-6] INFO > >>> >> > org.apache.ranger.biz.UserMgr (UserMgr.java:1150) - Invalid email > >>> >> >address:bob@localhost > >>> >> >2016-05-04 17:01:28,319 [http-bio-6080-exec-6] INFO > >>> >> > org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:64) - > >>> Request > >>> >> >failed. SessionId=13, loginId=rangerusersync, logMessage=Please > provide > >>> >> >valid email address. > >>> >> >javax.ws.rs.WebApplicationException > >>> >> >at > >>> >> > >>> > >org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.j > >>> >> >ava:55) > >>> >> >at > >>> >> > >>> > >org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.j > >>> >> >ava:310) > >>> >> >at > >>> >> > >>> > >org.apache.ranger.biz.UserMgr.createDefaultAccountUser(UserMgr.java:1151) > >>> >> > > >>> >> >From looking at the code it looks like the email address gets > created > >>> by > >>> >> >taking the username and adding the hostname detected > >>> >> >from InetAddress.getLocalHost().getCanonicalHostName(): > >>> >> > > >>> >> > >>> > https://github.com/apache/incubator-ranger/blob/8614032c909dd5599fedc35c8f > >>> >> > >>> > >4b80f71b3a950d/ugsync/src/main/java/org/apache/ranger/unixusersync/process > >>> >> >/PolicyMgrUserGroupBuilder.java#L782 > >>> >> > > >>> >> >Is there a recommended approach to make this work? > >>> >> >Seems like validation on the REST endpoint would need to be > relaxed, > >>> or we > >>> >> >need a way to provide the email address in the file. > >>> >> > > >>> >> >Thanks, > >>> >> > > >>> >> >Bryan > >>> >> > > >>> >> > > >>> >> >On Wed, May 4, 2016 at 1:45 AM, Gautam Borad <[email protected]> > wrote: > >>> >> > > >>> >> >> > > >>> >> >> > >For #1, I wasn't able to add a raw DN as a user through the > Ranger > >>> >> >>UI. I > >>> >> >> > think the '=' character violates the validation rules, but > maybe > >>> that > >>> >> >>is > >>> >> >> an > >>> >> >> > easy change to allow it. > >>> >> >> > Yes, this should be an easy change. Can you create a JIRA. I > feel, > >>> in > >>> >> >>the > >>> >> >> > long run we should probably take some of these rules by > property. > >>> It > >>> >> >> might > >>> >> >> > already by. Gautam, are you aware of it? > >>> >> >> > >>> >> >> > >>> >> >> Bosco, we currently dont have this feature of taking the rules by > >>> >> >>property. > >>> >> >> However it sounds like a good idea to do so! Thanks. > >>> >> >> > >>> >> >> > >>> >> >> On Wed, May 4, 2016 at 6:25 AM, Don Bosco Durai < > [email protected]> > >>> >> >>wrote: > >>> >> >> > >>> >> >> > >I think the file upload could be the best option for now, > >>> depending > >>> >> >>if > >>> >> >> > there are any issues with special characters. > >>> >> >> > > >>> >> >> > > >>> >> >> > >>> >> >> > >>> >> > >>> > https://cwiki.apache.org/confluence/display/RANGER/File+Source+User+Group > >>> >> >>+Sync+process > >>> >> >> > > >>> >> >> > > >>> >> >> > >For #1, I wasn't able to add a raw DN as a user through the > Ranger > >>> >> >>UI. I > >>> >> >> > think the '=' character violates the validation rules, but > maybe > >>> that > >>> >> >>is > >>> >> >> an > >>> >> >> > easy change to allow it. > >>> >> >> > Yes, this should be an easy change. Can you create a JIRA. I > feel, > >>> in > >>> >> >>the > >>> >> >> > long run we should probably take some of these rules by > property. > >>> It > >>> >> >> might > >>> >> >> > already by. Gautam, are you aware of it? > >>> >> >> > > >>> >> >> > >For #2, I think the issue would be that two users could have > the > >>> >> >>same CN > >>> >> >> > from different organizations, and so the full the DN is really > the > >>> >> >>unique > >>> >> >> > identifier. > >>> >> >> > Yes, it is possible. Note, if you will be doing any Hadoop > related > >>> >> >> > operations behalf of the user, then you will will get into > other > >>> >> >>issues. > >>> >> >> If > >>> >> >> > you are, then you will have to tokenize it to unix friendly > name > >>> >> >> > > >>> >> >> > >For grouping, I see that RangerAccessRequest allows setting > the > >>> user > >>> >> >> > groups. > >>> >> >> > Grouping will help you minimize the number of policies. If you > >>> feel, > >>> >> >>we > >>> >> >> > can break the DN and create logical groups, and if that works, > >>> then it > >>> >> >> will > >>> >> >> > easy for the admins to configure policies. Essentially, you > could > >>> have > >>> >> >> each > >>> >> >> > level as a group. And give group level permissions... > >>> >> >> > "OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, C=US², > "O=Apache, > >>> >> >> > L=Santa Monica, ST=CA, C=US², "L=Santa Monica, ST=CA, C=US², Š > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > >One scenario is when two NiFi instances communicate directly > over > >>> a > >>> >> >> > secure connection, we would need to create a policy in Ranger > for > >>> the > >>> >> >>DN > >>> >> >> of > >>> >> >> > one instance to give access to the resource being accessed on > the > >>> >> >>other > >>> >> >> > instance. > >>> >> >> > This would be easy. You might need very few policy line items > and > >>> it > >>> >> >>will > >>> >> >> > be easy to manage. > >>> >> >> > > >>> >> >> > >We could also have scenarios where regular users are issued > >>> >> >>certificates > >>> >> >> > and accessing the NiFi UI with those certificates. > >>> >> >> > If these are human users, then there might be more > policies/line > >>> >> >>items, > >>> >> >> > but still manageable. If each device is an user, then you > should > >>> >> >>consider > >>> >> >> > creating the policies using REST API. > >>> >> >> > > >>> >> >> > Thanks > >>> >> >> > > >>> >> >> > Bosco > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > On 5/3/16, 7:15 AM, "Bryan Bende" <[email protected]> wrote: > >>> >> >> > > >>> >> >> > >Hi Bosco, > >>> >> >> > > > >>> >> >> > >Thanks for the response. Could you elaborate on Ranger's file > >>> upload > >>> >> >> > >feature? Is this done through the REST API? > >>> >> >> > >I think the file upload could be the best option for now, > >>> depending > >>> >> >>if > >>> >> >> > >there are any issues with special characters. > >>> >> >> > > > >>> >> >> > >For #1, I wasn't able to add a raw DN as a user through the > Ranger > >>> >> >>UI. I > >>> >> >> > >think the '=' character violates the validation rules, but > maybe > >>> >> >>that is > >>> >> >> > an > >>> >> >> > >easy change to allow it. > >>> >> >> > > > >>> >> >> > >For #2, I think the issue would be that two users could have > the > >>> >> >>same CN > >>> >> >> > >from different organizations, and so the full the DN is > really the > >>> >> >> unique > >>> >> >> > >identifier. > >>> >> >> > > > >>> >> >> > >For grouping, I see that RangerAccessRequest allows setting > the > >>> user > >>> >> >> > >groups. I think the issue is that on the NiFi side when we > >>> >> >>authenticate > >>> >> >> a > >>> >> >> > >user who presents a certificate, > >>> >> >> > >we don't have knowledge of a group for that user, so we > wouldn't > >>> know > >>> >> >> what > >>> >> >> > >to set on the access request. > >>> >> >> > > > >>> >> >> > >For some general background, NiFi has a pluggable > authentication > >>> >> >> mechanism > >>> >> >> > >and currently has three mechanisms: 2-way SSL, LDAP, and > >>> Kerberos.... > >>> >> >> > 2-way > >>> >> >> > >SSL is always enabled when running a secured instance. > >>> >> >> > >One scenario is when two NiFi instances communicate directly > over > >>> a > >>> >> >> secure > >>> >> >> > >connection, we would need to create a policy in Ranger for > the DN > >>> of > >>> >> >>one > >>> >> >> > >instance to give access to the resource being accessed on the > >>> other > >>> >> >> > >instance. > >>> >> >> > >We could also have scenarios where regular users are issued > >>> >> >>certificates > >>> >> >> > >and accessing the NiFi UI with those certificates. > >>> >> >> > > > >>> >> >> > >Thanks, > >>> >> >> > > > >>> >> >> > >Bryan > >>> >> >> > > > >>> >> >> > > > >>> >> >> > >On Tue, May 3, 2016 at 1:06 AM, Don Bosco Durai < > [email protected] > >>> > > >>> >> >> wrote: > >>> >> >> > > > >>> >> >> > >> From the Ranger point of view it is just any other user, > but we > >>> >> >>have > >>> >> >> to > >>> >> >> > >> check whether Ranger supports all the characters valid in > the > >>> DN. > >>> >> >> > >> > >>> >> >> > >> The interesting part is how we classify this user. Will it > be in > >>> >> >> LDAP/AD > >>> >> >> > >> or if it is device, then it might not be. So we have a > couple of > >>> >> >> > options: > >>> >> >> > >> > >>> >> >> > >> 1. Add the DN to Ranger in the raw format and give > permissions > >>> to > >>> >> >>it > >>> >> >> > using > >>> >> >> > >> policy. It will have usability issue in UI. > >>> >> >> > >> 2. Map the DN to simple name. E.g. In Hadoop, it could be > the > >>> CN or > >>> >> >> UID > >>> >> >> > >> attribute. Or sAMAccountName from AD. In your case, both > >>> >> >>provisioning > >>> >> >> to > >>> >> >> > >> Ranger and NiFiRangerAuthorizer has to do the same > conversion. > >>> >> >> > >> > >>> >> >> > >> Do you think, #2 is possible for you? > >>> >> >> > >> > >>> >> >> > >> Regardless, you could use Ranger¹s file upload feature to > load > >>> the > >>> >> >> > users. > >>> >> >> > >> I feel, we might get into special character issues like > space or > >>> >> >> comma. > >>> >> >> > I > >>> >> >> > >> think, we can fix this if required. > >>> >> >> > >> > >>> >> >> > >> Another suggestion is, can we have group concept for these > DN? > >>> >> >> > >> > >>> >> >> > >> Thanks > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > >> Bosco > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > >> On 5/2/16, 9:43 AM, "Bryan Bende" <[email protected]> wrote: > >>> >> >> > >> > >>> >> >> > >> >Hello, > >>> >> >> > >> > > >>> >> >> > >> >If an application is authenticating users with 2-way SSL, > how > >>> >> >>would > >>> >> >> > those > >>> >> >> > >> >users be entered into Ranger in order to define policies > for > >>> >> >>them? or > >>> >> >> > is > >>> >> >> > >> >that not really a supported scenario? > >>> >> >> > >> > > >>> >> >> > >> >For example, if I authenticate to my application with a > >>> >> >>certificate, > >>> >> >> > the > >>> >> >> > >> >identity passed to the plugin will be the DN from the > >>> certificate > >>> >> >> like: > >>> >> >> > >> > > >>> >> >> > >> >CN=localhost, OU=Apache NiFi, O=Apache, L=Santa Monica, > ST=CA, > >>> >> >>C=US > >>> >> >> > >> > > >>> >> >> > >> >So I was trying to see if it was possible to define a > policy > >>> for > >>> >> >>that > >>> >> >> > >> user. > >>> >> >> > >> > > >>> >> >> > >> >Thanks, > >>> >> >> > >> > > >>> >> >> > >> >Bryan > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> > > >>> >> >> > > >>> >> >> > >>> >> >> > >>> >> >> -- > >>> >> >> Regards, > >>> >> >> Gautam. > >>> >> >> > >>> >> > >>> >> > >>> > >>> > >
