Sailaja, can you review the patch by Bryan. If it unblocks him, then we can commit it and come with a better solution in later release?
Bryan, generally we use review board for patch review. It is easier for everyone to give comment. Thanks Bosco On 5/4/16, 2:18 PM, "Sailaja Polavarapu" <[email protected]> wrote: >After following the code in the ranger admin for adding portal users, >following are some of the observations to be considered: >1. In the DB level email field can be null >2. In UserMrg.java, if the emailAddress (from the request) is null or empty, >we are generating a randomUID and assigning using that email address to create >or update portal user. Of course in this case we are not doing any validation >checks. > >One suggestion I have here is, instead of removing the validation checks for >email address - >1. Remove auto population of email address in the ranger usersync for File >source (and other sync sources if needed) >2. Remove auto population of email address with randomUID in ranger admin code. >3. Do validation only when email address is not null & not empty. > >Thanks, >Sailaja. > > > > >On 5/4/16, 10:01 AM, "Bryan Bende" <[email protected]> wrote: > >>Ok I noticed it wasn't required in the UI so that makes sense. >> >>Let me see what I can do, I'll create a JIRA. >> >>On Wed, May 4, 2016 at 12:52 PM, Don Bosco Durai <[email protected]> wrote: >> >>> Emails are no long used in Ranger. I feel, we should disable all >>> validations for email. If it is quick for you, do you want to quickly >>> remove the validation and upload the patch to be committed? >>> >>> Thanks >>> >>> Bosco >>> >>> >>> >>> >>> >>> On 5/4/16, 9:02 AM, "Bryan Bende" <[email protected]> wrote: >>> >>> >Ramesh, >>> > >>> >I am working off the master branch, just pulled latest code from about an >>> >hour ago. >>> > >>> >Thanks, >>> > >>> >Bryan >>> > >>> > >>> >On Wed, May 4, 2016 at 11:50 AM, Ramesh Mani <[email protected]> >>> wrote: >>> > >>> >> Bryan, >>> >> >>> >> Which version of the ranger you are trying this? >>> >> >>> >> Thanks, >>> >> Ramesh >>> >> >>> >> On 5/4/16, 8:22 AM, "Bryan Bende" <[email protected]> wrote: >>> >> >>> >> >Sure I will create a JIRA about the validation of the usernames. >>> >> > >>> >> >I was able to get the user sync running with the file source and >>> >> >encountered the following... I created a file usergroups.csv based off >>> >> > >>> >> >>> https://cwiki.apache.org/confluence/display/RANGER/File+Source+User+Group+ >>> >> >Sync+process >>> >> >: >>> >> > >>> >> >"bob", >>> >> >"john", >>> >> >"cn=bbende,dc=example,dc=org", >>> >> > >>> >> >When the user sync process runs it gets: >>> >> > >>> >> >016-05-04 17:01:28,318 [http-bio-6080-exec-6] INFO >>> >> > org.apache.ranger.biz.UserMgr (UserMgr.java:1140) - >>> create:bob@localhost >>> >> >2016-05-04 17:01:28,318 [http-bio-6080-exec-6] INFO >>> >> > org.apache.ranger.biz.UserMgr (UserMgr.java:1150) - Invalid email >>> >> >address:bob@localhost >>> >> >2016-05-04 17:01:28,319 [http-bio-6080-exec-6] INFO >>> >> > org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:64) - >>> Request >>> >> >failed. SessionId=13, loginId=rangerusersync, logMessage=Please provide >>> >> >valid email address. >>> >> >javax.ws.rs.WebApplicationException >>> >> >at >>> >> >>> >org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.j >>> >> >ava:55) >>> >> >at >>> >> >>> >org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.j >>> >> >ava:310) >>> >> >at >>> >> >>> >org.apache.ranger.biz.UserMgr.createDefaultAccountUser(UserMgr.java:1151) >>> >> > >>> >> >From looking at the code it looks like the email address gets created >>> by >>> >> >taking the username and adding the hostname detected >>> >> >from InetAddress.getLocalHost().getCanonicalHostName(): >>> >> > >>> >> >>> https://github.com/apache/incubator-ranger/blob/8614032c909dd5599fedc35c8f >>> >> >>> >4b80f71b3a950d/ugsync/src/main/java/org/apache/ranger/unixusersync/process >>> >> >/PolicyMgrUserGroupBuilder.java#L782 >>> >> > >>> >> >Is there a recommended approach to make this work? >>> >> >Seems like validation on the REST endpoint would need to be relaxed, >>> or we >>> >> >need a way to provide the email address in the file. >>> >> > >>> >> >Thanks, >>> >> > >>> >> >Bryan >>> >> > >>> >> > >>> >> >On Wed, May 4, 2016 at 1:45 AM, Gautam Borad <[email protected]> wrote: >>> >> > >>> >> >> > >>> >> >> > >For #1, I wasn't able to add a raw DN as a user through the Ranger >>> >> >>UI. I >>> >> >> > think the '=' character violates the validation rules, but maybe >>> that >>> >> >>is >>> >> >> an >>> >> >> > easy change to allow it. >>> >> >> > Yes, this should be an easy change. Can you create a JIRA. I feel, >>> in >>> >> >>the >>> >> >> > long run we should probably take some of these rules by property. >>> It >>> >> >> might >>> >> >> > already by. Gautam, are you aware of it? >>> >> >> >>> >> >> >>> >> >> Bosco, we currently dont have this feature of taking the rules by >>> >> >>property. >>> >> >> However it sounds like a good idea to do so! Thanks. >>> >> >> >>> >> >> >>> >> >> On Wed, May 4, 2016 at 6:25 AM, Don Bosco Durai <[email protected]> >>> >> >>wrote: >>> >> >> >>> >> >> > >I think the file upload could be the best option for now, >>> depending >>> >> >>if >>> >> >> > there are any issues with special characters. >>> >> >> > >>> >> >> > >>> >> >> >>> >> >> >>> >> >>> https://cwiki.apache.org/confluence/display/RANGER/File+Source+User+Group >>> >> >>+Sync+process >>> >> >> > >>> >> >> > >>> >> >> > >For #1, I wasn't able to add a raw DN as a user through the Ranger >>> >> >>UI. I >>> >> >> > think the '=' character violates the validation rules, but maybe >>> that >>> >> >>is >>> >> >> an >>> >> >> > easy change to allow it. >>> >> >> > Yes, this should be an easy change. Can you create a JIRA. I feel, >>> in >>> >> >>the >>> >> >> > long run we should probably take some of these rules by property. >>> It >>> >> >> might >>> >> >> > already by. Gautam, are you aware of it? >>> >> >> > >>> >> >> > >For #2, I think the issue would be that two users could have the >>> >> >>same CN >>> >> >> > from different organizations, and so the full the DN is really the >>> >> >>unique >>> >> >> > identifier. >>> >> >> > Yes, it is possible. Note, if you will be doing any Hadoop related >>> >> >> > operations behalf of the user, then you will will get into other >>> >> >>issues. >>> >> >> If >>> >> >> > you are, then you will have to tokenize it to unix friendly name >>> >> >> > >>> >> >> > >For grouping, I see that RangerAccessRequest allows setting the >>> user >>> >> >> > groups. >>> >> >> > Grouping will help you minimize the number of policies. If you >>> feel, >>> >> >>we >>> >> >> > can break the DN and create logical groups, and if that works, >>> then it >>> >> >> will >>> >> >> > easy for the admins to configure policies. Essentially, you could >>> have >>> >> >> each >>> >> >> > level as a group. And give group level permissions... >>> >> >> > "OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, C=US², "O=Apache, >>> >> >> > L=Santa Monica, ST=CA, C=US², "L=Santa Monica, ST=CA, C=US², Š >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >One scenario is when two NiFi instances communicate directly over >>> a >>> >> >> > secure connection, we would need to create a policy in Ranger for >>> the >>> >> >>DN >>> >> >> of >>> >> >> > one instance to give access to the resource being accessed on the >>> >> >>other >>> >> >> > instance. >>> >> >> > This would be easy. You might need very few policy line items and >>> it >>> >> >>will >>> >> >> > be easy to manage. >>> >> >> > >>> >> >> > >We could also have scenarios where regular users are issued >>> >> >>certificates >>> >> >> > and accessing the NiFi UI with those certificates. >>> >> >> > If these are human users, then there might be more policies/line >>> >> >>items, >>> >> >> > but still manageable. If each device is an user, then you should >>> >> >>consider >>> >> >> > creating the policies using REST API. >>> >> >> > >>> >> >> > Thanks >>> >> >> > >>> >> >> > Bosco >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> > On 5/3/16, 7:15 AM, "Bryan Bende" <[email protected]> wrote: >>> >> >> > >>> >> >> > >Hi Bosco, >>> >> >> > > >>> >> >> > >Thanks for the response. Could you elaborate on Ranger's file >>> upload >>> >> >> > >feature? Is this done through the REST API? >>> >> >> > >I think the file upload could be the best option for now, >>> depending >>> >> >>if >>> >> >> > >there are any issues with special characters. >>> >> >> > > >>> >> >> > >For #1, I wasn't able to add a raw DN as a user through the Ranger >>> >> >>UI. I >>> >> >> > >think the '=' character violates the validation rules, but maybe >>> >> >>that is >>> >> >> > an >>> >> >> > >easy change to allow it. >>> >> >> > > >>> >> >> > >For #2, I think the issue would be that two users could have the >>> >> >>same CN >>> >> >> > >from different organizations, and so the full the DN is really the >>> >> >> unique >>> >> >> > >identifier. >>> >> >> > > >>> >> >> > >For grouping, I see that RangerAccessRequest allows setting the >>> user >>> >> >> > >groups. I think the issue is that on the NiFi side when we >>> >> >>authenticate >>> >> >> a >>> >> >> > >user who presents a certificate, >>> >> >> > >we don't have knowledge of a group for that user, so we wouldn't >>> know >>> >> >> what >>> >> >> > >to set on the access request. >>> >> >> > > >>> >> >> > >For some general background, NiFi has a pluggable authentication >>> >> >> mechanism >>> >> >> > >and currently has three mechanisms: 2-way SSL, LDAP, and >>> Kerberos.... >>> >> >> > 2-way >>> >> >> > >SSL is always enabled when running a secured instance. >>> >> >> > >One scenario is when two NiFi instances communicate directly over >>> a >>> >> >> secure >>> >> >> > >connection, we would need to create a policy in Ranger for the DN >>> of >>> >> >>one >>> >> >> > >instance to give access to the resource being accessed on the >>> other >>> >> >> > >instance. >>> >> >> > >We could also have scenarios where regular users are issued >>> >> >>certificates >>> >> >> > >and accessing the NiFi UI with those certificates. >>> >> >> > > >>> >> >> > >Thanks, >>> >> >> > > >>> >> >> > >Bryan >>> >> >> > > >>> >> >> > > >>> >> >> > >On Tue, May 3, 2016 at 1:06 AM, Don Bosco Durai <[email protected] >>> > >>> >> >> wrote: >>> >> >> > > >>> >> >> > >> From the Ranger point of view it is just any other user, but we >>> >> >>have >>> >> >> to >>> >> >> > >> check whether Ranger supports all the characters valid in the >>> DN. >>> >> >> > >> >>> >> >> > >> The interesting part is how we classify this user. Will it be in >>> >> >> LDAP/AD >>> >> >> > >> or if it is device, then it might not be. So we have a couple of >>> >> >> > options: >>> >> >> > >> >>> >> >> > >> 1. Add the DN to Ranger in the raw format and give permissions >>> to >>> >> >>it >>> >> >> > using >>> >> >> > >> policy. It will have usability issue in UI. >>> >> >> > >> 2. Map the DN to simple name. E.g. In Hadoop, it could be the >>> CN or >>> >> >> UID >>> >> >> > >> attribute. Or sAMAccountName from AD. In your case, both >>> >> >>provisioning >>> >> >> to >>> >> >> > >> Ranger and NiFiRangerAuthorizer has to do the same conversion. >>> >> >> > >> >>> >> >> > >> Do you think, #2 is possible for you? >>> >> >> > >> >>> >> >> > >> Regardless, you could use Ranger¹s file upload feature to load >>> the >>> >> >> > users. >>> >> >> > >> I feel, we might get into special character issues like space or >>> >> >> comma. >>> >> >> > I >>> >> >> > >> think, we can fix this if required. >>> >> >> > >> >>> >> >> > >> Another suggestion is, can we have group concept for these DN? >>> >> >> > >> >>> >> >> > >> Thanks >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> Bosco >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> On 5/2/16, 9:43 AM, "Bryan Bende" <[email protected]> wrote: >>> >> >> > >> >>> >> >> > >> >Hello, >>> >> >> > >> > >>> >> >> > >> >If an application is authenticating users with 2-way SSL, how >>> >> >>would >>> >> >> > those >>> >> >> > >> >users be entered into Ranger in order to define policies for >>> >> >>them? or >>> >> >> > is >>> >> >> > >> >that not really a supported scenario? >>> >> >> > >> > >>> >> >> > >> >For example, if I authenticate to my application with a >>> >> >>certificate, >>> >> >> > the >>> >> >> > >> >identity passed to the plugin will be the DN from the >>> certificate >>> >> >> like: >>> >> >> > >> > >>> >> >> > >> >CN=localhost, OU=Apache NiFi, O=Apache, L=Santa Monica, ST=CA, >>> >> >>C=US >>> >> >> > >> > >>> >> >> > >> >So I was trying to see if it was possible to define a policy >>> for >>> >> >>that >>> >> >> > >> user. >>> >> >> > >> > >>> >> >> > >> >Thanks, >>> >> >> > >> > >>> >> >> > >> >Bryan >>> >> >> > >> >>> >> >> > >> >>> >> >> > >>> >> >> > >>> >> >> >>> >> >> >>> >> >> -- >>> >> >> Regards, >>> >> >> Gautam. >>> >> >> >>> >> >>> >> >>> >>>
