Missed the attachment before sending the email ... rookie mistake. Here is
the actual file (sorry for the spam).
Best regards,
Loïc CHANEL
Technical leader Big Data
Capgemini (Lyon, France)
Le ven. 8 mars 2024 à 17:33, Loïc CHANEL <loic.cha...@telecomnancy.net> a
écrit :
> I'm using the 2.4.0 release. You will find the ranger-ugsync-site.xml config
> file in attachment, feel free to let me know if you need more.
> Best regards,
>
>
> Loïc CHANEL
> Technical leader Big Data
> Capgemini (Lyon, France)
>
>
> Le ven. 8 mars 2024 à 17:24, Sailaja Polavarapu <spolavar...@cloudera.com>
> a écrit :
>
>> Which branch are you using? And can you share the usersync config?
>>
>> On Fri, Mar 8, 2024 at 8:10 AM Loïc CHANEL <loic.cha...@telecomnancy.net>
>> wrote:
>>
>>> Hi Sailaja,
>>>
>>> I just updated the file and restarted the usersync process, I have the
>>> exact same logs and the users are still not created. Is there a case in
>>> which the file is printed but not synced ?
>>> No error before, except the following warning that's quite usual :
>>> 08 Mar 2024 16:48:50 WARN o.a.h.u.NativeCodeLoader [UnixUserSyncThread]
>>> - Unable to load native-hadoop library for your platform... using
>>> builtin-java classes where applicable
>>> Still, it's weird that it shows with "UnixUserSyncThread" while it's
>>> syncing users from a file, isn't it ?
>>> I'm a bit lost on this, because it is clear that the usersync process is
>>> able to read the user file and I see it communicating with Ranger Admin to
>>> load all the users, but the users in the file are not created. Is there
>>> some configuration missing from
>>> https://cwiki.apache.org/confluence/display/RANGER/File+Source+User+Group+Sync+process
>>> ?
>>> Thanks,
>>>
>>>
>>> Loïc CHANEL
>>> Technical leader Big Data
>>> Capgemini (Lyon, France)
>>>
>>>
>>> Le ven. 8 mars 2024 à 16:00, Sailaja Polavarapu <
>>> spolavar...@cloudera.com> a écrit :
>>>
>>>> This is strange as I don't see any logs from updateSink() method
>>>> <https://github.com/apache/ranger/blob/master/ugsync/src/main/java/org/apache/ranger/unixusersync/process/FileSourceUserGroupBuilder.java#L145>
>>>> .
>>>> Can you check the timestamp on the file and try updating the file? and also
>>>> check if there are any errors before. Looks like this file is read once
>>>> successfully and hence you see the logs from print() method which logs the
>>>> data from usersync cache.
>>>>
>>>> On Fri, Mar 8, 2024 at 4:40 AM Loïc CHANEL <
>>>> loic.cha...@telecomnancy.net> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Here are the logs on usersync side :
>>>>> 08 Mar 2024 12:26:54 DEBUG o.a.r.u.p.PolicyMgrUserGroupBuilder
>>>>> [UnixUserSyncThread] - PolicyMgrUserGroupBuilderOld.init()==>
>>>>> PolMgrBaseUrl
>>>>> : http://hdp-rec-ranger1-r.l.infra.com:6080 KeyStore File :
>>>>> /etc/ranger/usersync/conf/cert/unixauthservice.jks TrustStore File :
>>>>> Authentication Type : kerberos
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.AbstractMapper [UnixUserSyncThread]
>>>>> - Initializing for ranger.usersync.mapping.username.regex
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.AbstractMapper [UnixUserSyncThread]
>>>>> - Initializing for ranger.usersync.mapping.groupname.regex
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.UserGroupSync [UnixUserSyncThread]
>>>>> - initializing source:
>>>>> org.apache.ranger.unixusersync.process.FileSourceUserGroupBuilder
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.AbstractMapper [UnixUserSyncThread]
>>>>> - Initializing for ranger.usersync.mapping.username.regex
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.AbstractMapper [UnixUserSyncThread]
>>>>> - Initializing for ranger.usersync.mapping.groupname.regex
>>>>> 08 Mar 2024 12:26:54 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>> [UnixUserSyncThread] - USER:user_one
>>>>> 08 Mar 2024 12:26:54 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>> [UnixUserSyncThread] - USER:user_two
>>>>> 08 Mar 2024 12:26:54 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>> [UnixUserSyncThread] - GROUP: 900021
>>>>> 08 Mar 2024 12:26:54 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>> [UnixUserSyncThread] - USER:other_user
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.UserGroupSync [UnixUserSyncThread]
>>>>> - Begin: initial load of user/group from source==>sink
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.UserGroupSync [UnixUserSyncThread]
>>>>> - End: initial load of user/group from source==>sink
>>>>> 08 Mar 2024 12:26:54 INFO o.a.r.u.UserGroupSync [UnixUserSyncThread]
>>>>> - Done initializing user/group source and sink
>>>>> 08 Mar 2024 12:26:54 DEBUG o.a.r.u.UserGroupSync [UnixUserSyncThread]
>>>>> - Sleeping for [300000] milliSeconds
>>>>>
>>>>> While on the admin side, I only see the following :
>>>>> 10.18.1.43 - - [08/Mar/2024:12:26:52 +0000] "GET
>>>>> /service/xusers/users/?startIndex=4000&pageSize=1000 HTTP/1.1" 200 1065581
>>>>> 27402 "-" "Java/1.8.0_275"
>>>>> 10.18.1.43 - - [08/Mar/2024:12:26:54 +0000] "GET
>>>>> /service/xusers/ugsync/groupusers?startIndex=0&pageSize=1000 HTTP/1.1" 200
>>>>> 4429925 2553 "-" "Java/1.8.0_275"
>>>>>
>>>>> Looks like usersync isn't even trying to create the users. Anything
>>>>> I'm missing ?
>>>>> Best regards,
>>>>>
>>>>>
>>>>> Loïc CHANEL
>>>>> Technical leader Big Data
>>>>> Capgemini (Lyon, France)
>>>>>
>>>>>
>>>>> Le ven. 8 mars 2024 à 04:44, Bhavik Patel <bhavikpatel...@gmail.com>
>>>>> a écrit :
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Can you kindly check Ranger admin logs and did you confirmed provided
>>>>>> rangeruseraync user password is same at Rangeradmin end.
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Bhavik Patel
>>>>>> +91-7208744109
>>>>>>
>>>>>>
>>>>>> On Thu, 7 Mar 2024, 6:39 pm Loïc CHANEL, <
>>>>>> loic.cha...@telecomnancy.net> wrote:
>>>>>>
>>>>>>> Hi guys,
>>>>>>>
>>>>>>> I'm currently trying to create Ranger users from a text file and
>>>>>>> followed this documentation to configure Usersync to work with a text
>>>>>>> file
>>>>>>> as a source :
>>>>>>> https://cwiki.apache.org/confluence/display/RANGER/File+Source+User+Group+Sync+process
>>>>>>> Still, even if the logs show that the CSV file containing the users
>>>>>>> is properly read, nothing happens : I can't see the users in Ranger UI.
>>>>>>> Here are the logs :
>>>>>>>
>>>>>>> 07 Mar 2024 12:22:13 DEBUG o.a.r.u.p.PolicyMgrUserGroupBuilder
>>>>>>> [UnixUserSyncThread] - <==
>>>>>>> PolicyMgrUserGroupBuilder.buildGroupUserLinkList()
>>>>>>> 07 Mar 2024 12:22:13 DEBUG o.a.r.u.p.PolicyMgrUserGroupBuilder
>>>>>>> [UnixUserSyncThread] - PolicyMgrUserGroupBuilderOld.init()==>
>>>>>>> PolMgrBaseUrl
>>>>>>> : http://hdp-rec-ranger1-r.l.internal.com:6080 KeyStore File :
>>>>>>> /etc/ranger/usersync/conf/cert/unixauthservice.jks TrustStore File :
>>>>>>> Authentication Type : kerberos
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.AbstractMapper
>>>>>>> [UnixUserSyncThread] - Initializing for
>>>>>>> ranger.usersync.mapping.username.regex
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.AbstractMapper
>>>>>>> [UnixUserSyncThread] - Initializing for
>>>>>>> ranger.usersync.mapping.groupname.regex
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.UserGroupSync
>>>>>>> [UnixUserSyncThread] - initializing source:
>>>>>>> org.apache.ranger.unixusersync.process.FileSourceUserGroupBuilder
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.AbstractMapper
>>>>>>> [UnixUserSyncThread] - Initializing for
>>>>>>> ranger.usersync.mapping.username.regex
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.AbstractMapper
>>>>>>> [UnixUserSyncThread] - Initializing for
>>>>>>> ranger.usersync.mapping.groupname.regex
>>>>>>> 07 Mar 2024 12:22:13 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>>>> [UnixUserSyncThread] - USER:user_one
>>>>>>> 07 Mar 2024 12:22:13 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>>>> [UnixUserSyncThread] - USER:user_two
>>>>>>> 07 Mar 2024 12:22:13 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>>>> [UnixUserSyncThread] - GROUP: 900021
>>>>>>> 07 Mar 2024 12:22:13 DEBUG o.a.r.u.p.FileSourceUserGroupBuilder
>>>>>>> [UnixUserSyncThread] - USER:other_user
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.UserGroupSync
>>>>>>> [UnixUserSyncThread] - Begin: initial load of user/group from
>>>>>>> source==>sink
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.UserGroupSync
>>>>>>> [UnixUserSyncThread] - End: initial load of user/group from
>>>>>>> source==>sink
>>>>>>> 07 Mar 2024 12:22:13 INFO o.a.r.u.UserGroupSync
>>>>>>> [UnixUserSyncThread] - Done initializing user/group source and sink
>>>>>>> 07 Mar 2024 12:22:13 DEBUG o.a.r.u.UserGroupSync
>>>>>>> [UnixUserSyncThread] - Sleeping for [300000] milliSeconds
>>>>>>>
>>>>>>> Am I missing something ?
>>>>>>> Thanks for your help,
>>>>>>>
>>>>>>>
>>>>>>> Loïc CHANEL
>>>>>>> Technical leader Big Data
>>>>>>> Capgemini (Lyon, France)
>>>>>>>
>>>>>>
<configuration>
<property>
<name>ranger.usersync.credstore.filename</name>
<value>/etc/ranger/usersync/conf/rangerusersync.jceks</value>
</property>
<property>
<name>ranger.usersync.enabled</name>
<value>false</value>
</property>
<property>
<name>ranger.usersync.group.memberattributename</name>
<value />
</property>
<property>
<name>ranger.usersync.group.nameattribute</name>
<value />
</property>
<property>
<name>ranger.usersync.group.objectclass</name>
<value />
</property>
<property>
<name>ranger.usersync.group.searchbase</name>
<value />
</property>
<property>
<name>ranger.usersync.group.searchenabled</name>
<value />
</property>
<property>
<name>ranger.usersync.group.searchfilter</name>
<value />
</property>
<property>
<name>ranger.usersync.group.searchscope</name>
<value />
</property>
<property>
<name>ranger.usersync.ldap.binddn</name>
<value />
</property>
<property>
<name>ranger.usersync.ldap.groupname.caseconversion</name>
<value>none</value>
</property>
<property>
<name>ranger.usersync.ldap.ldapbindpassword</name>
<value>_</value>
</property>
<property>
<name>ranger.usersync.ldap.searchBase</name>
<value />
</property>
<property>
<name>ranger.usersync.ldap.url</name>
<value />
</property>
<property>
<name>ranger.usersync.ldap.deltasync</name>
<value />
</property>
<property>
<name>ranger.usersync.ldap.user.groupnameattribute</name>
<value>memberof,ismemberof</value>
</property>
<property>
<name>ranger.usersync.ldap.user.nameattribute</name>
<value>cn</value>
</property>
<property>
<name>ranger.usersync.ldap.user.objectclass</name>
<value>person</value>
</property>
<property>
<name>ranger.usersync.ldap.user.searchbase</name>
<value />
</property>
<property>
<name>ranger.usersync.ldap.user.searchfilter</name>
<value />
</property>
<property>
<name>ranger.usersync.ldap.user.searchscope</name>
<value>sub</value>
</property>
<property>
<name>ranger.usersync.ldap.username.caseconversion</name>
<value>none</value>
</property>
<property>
<name>ranger.usersync.logdir</name>
<value>/var/log/ranger/usersync</value>
</property>
<property>
<name>ranger.usersync.pagedresultsenabled</name>
<value />
</property>
<property>
<name>ranger.usersync.pagedresultssize</name>
<value />
</property>
<property>
<name>ranger.usersync.passwordvalidator.path</name>
<value>./native/credValidator.uexe</value>
</property>
<property>
<name>ranger.usersync.policymanager.baseURL</name>
<value>http://hdp-rec-ranger1-r.l.infra.com:6080</value>
</property>
<property>
<name>ranger.usersync.policymanager.maxrecordsperapicall</name>
<value>1000</value>
</property>
<property>
<name>ranger.usersync.policymanager.mockrun</name>
<value>false</value>
</property>
<property>
<name>ranger.usersync.port</name>
<value>5152</value>
</property>
<property>
<name>ranger.usersync.sink.impl.class</name>
<value>org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder</value>
</property>
<property>
<name>ranger.usersync.sleeptimeinmillisbetweensynccycle</name>
<value>300000</value>
</property>
<property>
<name>ranger.usersync.source.impl.class</name>
<!-- <value>org.apache.ranger.unixusersync.process.UnixUserGroupBuilder</value> -->
<value>org.apache.ranger.unixusersync.process.FileSourceUserGroupBuilder</value>
</property>
<property>
<name>ranger.usersync.filesource.file</name>
<value>/etc/ranger/usersync/conf/userlist.csv</value>
</property>
<property>
<name>ranger.usersync.filesource.text.delimiter</name>
<value>,</value>
</property>
<property>
<name>ranger.usersync.ssl</name>
<value>true</value>
</property>
<property>
<name>ranger.usersync.unix.minUserId</name>
<value>100</value>
</property>
<property>
<name>ranger.usersync.unix.minGroupId</name>
<value>100</value>
</property>
<property>
<name>ranger.usersync.keystore.file</name>
<value>/etc/ranger/usersync/conf/cert/unixauthservice.jks</value>
</property>
<property>
<name>ranger.usersync.truststore.file</name>
<value />
</property>
<property>
<name>ranger.usersync.policymgr.username</name>
<value>rangerusersync</value>
</property>
<property>
<name>ranger.usersync.policymgr.alias</name>
<value>ranger.usersync.policymgr.password</value>
</property>
<property>
<name>ranger.usersync.policymgr.keystore</name>
<value>/etc/ranger/usersync/conf/rangerusersync.jceks</value>
</property>
<property>
<name>ranger.usersync.sync.source</name>
<value>unix</value>
</property>
<property>
<name>ranger.usersync.ldap.referral</name>
<value>ignore</value>
</property>
<property>
<name>ranger.usersync.kerberos.principal</name>
<value>rangerusers...@hadooprec.ad.com</value>
</property>
<property>
<name>ranger.usersync.kerberos.keytab</name>
<value>/etc/security/keytabs/rangerusersync.hdpr.keytab</value>
</property>
<property>
<name>ranger.usersync.keystore.password</name>
<value>_</value>
</property>
<property>
<name>ranger.usersync.truststore.password</name>
<value>_</value>
</property>
<property>
<name>ranger.usersync.role.assignment.list.delimiter</name>
<value>&</value>
</property>
<property>
<name>ranger.usersync.users.groups.assignment.list.delimiter</name>
<value>:</value>
</property>
<property>
<name>ranger.usersync.username.groupname.assignment.list.delimiter</name>
<value>,</value>
</property>
<property>
<name>ranger.usersync.group.based.role.assignment.rules</name>
<value />
</property>
<property>
<name>ranger.usersync.dest.ranger.session.cookie.name</name>
<value>RANGERADMINSESSIONID</value>
</property>
<property>
<name>ranger.usersync.metrics.enabled</name>
<value />
</property>
<property>
<name>ranger.usersync.metrics.filepath</name>
<value />
</property>
<property>
<name>ranger.usersync.metrics.frequencytimeinmillis</name>
<value />
</property>
<property>
<name>ranger.usersync.metrics.filename</name>
<value />
</property>
</configuration>
