Re: Code conventions

[Lars Francke]

Hi Pradeep,

thank you for getting back to me and pointing me at your issue.
As a newcomer to a project, I agree that it'd be a tremendous help if
things were a bit more consistent.

Cheers,
Lars

On Wed, Jan 22, 2020 at 6:15 AM PradeeP AgrawaL <
pradeepagrawal8...@gmail.com> wrote:

> Hi Lars,
>
> There is no specific code conventions and checkstyle being followed
> currently, but important reviews are given during patch review process. I
> created one JIRA RANGER-2588 to implement checkstyle. I might need
> suggestion on that from community.
> This might force me to make changes in all the files if I would do so then
> it might affect other developers as well.
>
> Below are few guidelines which I received earlier from a community member.
> I accept that I also don't follow all of them always.
>
>
>1. Indentation: suggest to use 4 spaces; 2 spaces works as well – but be
>consistent! No tabs please!
>2. Spaces:
>   1. a single space before “(“, “{“
>   2. a single space after “)”
>   3. a single space around operators
>   4. Example: “if (a == b && x != y) {“; avoid “if(a==b&!=y){“
>3. Blocks: “{“, at the end of the line that starts the block, end the
>block with “}” in a separate line
>4. Statements: earlier restrictions on line length (of 80/132) were
>likely for printing of the code on paper. Most of us shouldn’t be
> printing
>source code on paper now. So, a statement need not be broken into
> multiple
>lines – even if it is few hundred characters long.
>5. Class member declaration in the following order:
>
>
>1. static members: public, protected, private; within each accessor,
>   ‘final’ members precede non-final
>   2. instance members: public, protected, private; within each
>   accessor, ‘final’ members precede non-final
>   3. Vertically align start of: typeName, variableName, assignment
>   operator
>   4. methods: public, protected, private; within each accessor, static
>   methods precede non-static
>   5. methods: all constructors immediately after ‘public static’
> methods
>   6. methods: all getters/setters immediately after constructors
>   7. methods: all @Override methods immediately after getter/setters
>   8. methods: use a consistent name for variable holding the return
>   value. Suggested name ‘ret’
>   9. methods: use a single return for each method (as much as possible)
>   10. methods: avoid methods with large number of lines; suggested
>   length 24 (from good old days!)
>   11. Blank line before and after each ‘for/while/do/if-else’ block
>   12. Blank line after each variable declaration block
>   13. Separate assignment lines from the rest
>
>
> On Tue, 21 Jan 2020 at 15:49, Lars Francke  wrote:
>
> > Hi Ranger devs,
> >
> > I've stumbled through the Ranger code base in the last few days and might
> > have a few things to contribute but I'm wondering if you have any kind of
> > code conventions?
> >
> > I'm asking because there's tabs being used as well as spaces and that's
> > just one of the things that struck me. It seems to be "all over the
> place"
> > which doesn't make it easier to get started. I see some references to
> > checkstyle but no config for it.
> >
> > So, two questions:
> > 1) Do you have any code conventions?
> >
> > 2) Would you accept whitespace/style only patches to "clean" it up a bit?
> > At least to be consistent within a single file.
> >
> > Cheers,
> > Lars
> >
>

Re: Review Request 71999: RANGER-2697: Usersync and Ranger admin changes to support retriving additional user/group attributes from LDAP/AD

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71999/
---

(Updated Jan. 22, 2020, 7:02 a.m.)


Review request for ranger.


Changes
---

Made few changes to Ranger User Store information


Bugs: RANGER-2697
https://issues.apache.org/jira/browse/RANGER-2697


Repository: ranger


Description
---

Modified Usersync code to retrieve configurable user and group attributes from 
AD/LDAP. Also added rest API support in ranger admin to dowmload user and group 
information including these additional attributes.


Diffs (updated)
-

  agents-common/src/main/java/org/apache/ranger/plugin/model/GroupInfo.java 
PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPluginInfo.java
 90367fe04 
  agents-common/src/main/java/org/apache/ranger/plugin/model/UserInfo.java 
PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerUserStore.java 
PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerUserStoreUtil.java
 PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 0f4488861 
  security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java 04596dc65 
  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 
ccda6abb0 
  security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java 3045eaf9a 
  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 037c591e8 
  
security-admin/src/main/java/org/apache/ranger/common/RangerUserStoreCache.java 
PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/db/XXGlobalStateDao.java 
2e462bd3a 
  security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 1e8a09379 
  security-admin/src/main/java/org/apache/ranger/service/XGroupServiceBase.java 
1a701bbfb 
  security-admin/src/main/java/org/apache/ranger/service/XUserServiceBase.java 
1004952a9 
  
ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapDeltaUserGroupBuilder.java
 ca50f098a 
  
ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapPolicyMgrUserGroupBuilder.java
 b469e9245 
  
ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
 8efa1613a 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
 1d4e37fcf 
  ugsync/src/main/java/org/apache/ranger/unixusersync/model/MUserInfo.java 
841bac64a 
  ugsync/src/main/java/org/apache/ranger/unixusersync/model/XGroupInfo.java 
cbe0eb02b 
  ugsync/src/main/java/org/apache/ranger/unixusersync/model/XUserInfo.java 
e2f36b2c5 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
 f08c5117b 
  ugsync/src/main/java/org/apache/ranger/usergroupsync/UserGroupSink.java 
77bd016fd 
  
ugsync/src/test/java/org/apache/ranger/usergroupsync/LdapPolicyMgrUserGroupBuilderTest.java
 99bc2b44e 
  
ugsync/src/test/java/org/apache/ranger/usergroupsync/PolicyMgrUserGroupBuilderTest.java
 2bc395180 


Diff: https://reviews.apache.org/r/71999/diff/2/

Changes: https://reviews.apache.org/r/71999/diff/1-2/


Testing
---

1. Verified all the existing unit tests are run successfully.
2. Patched a cluster and ran tests to sync various user/group attributes from 
test AD
3. Also verified new rest API to download user and group information with 
addition attributes.


Thanks,

Sailaja Polavarapu

Re: Code conventions

[PradeeP AgrawaL]

Hi Lars,

There is no specific code conventions and checkstyle being followed
currently, but important reviews are given during patch review process. I
created one JIRA RANGER-2588 to implement checkstyle. I might need
suggestion on that from community.
This might force me to make changes in all the files if I would do so then
it might affect other developers as well.

Below are few guidelines which I received earlier from a community member.
I accept that I also don't follow all of them always.


   1. Indentation: suggest to use 4 spaces; 2 spaces works as well – but be
   consistent! No tabs please!
   2. Spaces:
  1. a single space before “(“, “{“
  2. a single space after “)”
  3. a single space around operators
  4. Example: “if (a == b && x != y) {“; avoid “if(a==b&!=y){“
   3. Blocks: “{“, at the end of the line that starts the block, end the
   block with “}” in a separate line
   4. Statements: earlier restrictions on line length (of 80/132) were
   likely for printing of the code on paper. Most of us shouldn’t be printing
   source code on paper now. So, a statement need not be broken into multiple
   lines – even if it is few hundred characters long.
   5. Class member declaration in the following order:


   1. static members: public, protected, private; within each accessor,
  ‘final’ members precede non-final
  2. instance members: public, protected, private; within each
  accessor, ‘final’ members precede non-final
  3. Vertically align start of: typeName, variableName, assignment
  operator
  4. methods: public, protected, private; within each accessor, static
  methods precede non-static
  5. methods: all constructors immediately after ‘public static’ methods
  6. methods: all getters/setters immediately after constructors
  7. methods: all @Override methods immediately after getter/setters
  8. methods: use a consistent name for variable holding the return
  value. Suggested name ‘ret’
  9. methods: use a single return for each method (as much as possible)
  10. methods: avoid methods with large number of lines; suggested
  length 24 (from good old days!)
  11. Blank line before and after each ‘for/while/do/if-else’ block
  12. Blank line after each variable declaration block
  13. Separate assignment lines from the rest


On Tue, 21 Jan 2020 at 15:49, Lars Francke  wrote:

> Hi Ranger devs,
>
> I've stumbled through the Ranger code base in the last few days and might
> have a few things to contribute but I'm wondering if you have any kind of
> code conventions?
>
> I'm asking because there's tabs being used as well as spaces and that's
> just one of the things that struck me. It seems to be "all over the place"
> which doesn't make it easier to get started. I see some references to
> checkstyle but no config for it.
>
> So, two questions:
> 1) Do you have any code conventions?
>
> 2) Would you accept whitespace/style only patches to "clean" it up a bit?
> At least to be consistent within a single file.
>
> Cheers,
> Lars
>

Re: Review Request 71993: RANGER-2699 : JVM metrics for Ranger usersync and Ranger tagsync

[Abhay Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71993/#review219347
---




tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncConfig.java
Lines 110 (patched)


Please consider inserting the unit (milliseconds) in the property name here 
and in usersync property name.



tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncMetricsProducer.java
Lines 45 (patched)


Please consider removing this check if the thread is not started at all 
when metrics collection is disabled.



tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java
Lines 141 (patched)


If the metrics collection is disabled, is there any reason to create 
TagSyncMetricsProducer object and run it in the thread?



tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java
Lines 143 (patched)


Please ensure that the thread is a damon thread. Please see 
TagAdminRESTSink.start() for reference.


- Abhay Kulkarni


On Jan. 21, 2020, 7:14 a.m., Dineshkumar Yadav wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71993/
> ---
> 
> (Updated Jan. 21, 2020, 7:14 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Gautam Borad, Mehul Parikh, and 
> Pradeep Agrawal.
> 
> 
> Bugs: RANGER-2699
> https://issues.apache.org/jira/browse/RANGER-2699
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Added metrics information in usersync and tagsync services
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerMetricsUtil.java
>  PRE-CREATION 
>   tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncConfig.java 
> 6d27b02cd 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncMetricsProducer.java
>  PRE-CREATION 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java 
> 8806c7469 
>   tagsync/src/main/resources/ranger-tagsync-site.xml 9a14c1cc7 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
>  1d4e37fcf 
>   
> ugsync/src/main/java/org/apache/ranger/usergroupsync/UserSyncMetricsProducer.java
>  PRE-CREATION 
>   ugsync/src/test/resources/ranger-ugsync-site.xml 0a1a86da0 
>   
> unixauthservice/src/main/java/org/apache/ranger/authentication/UnixAuthenticationService.java
>  1ee5e21ba 
> 
> 
> Diff: https://reviews.apache.org/r/71993/diff/5/
> 
> 
> Testing
> ---
> 
> Validated metrics which are getting generated at configured interval when 
> usersync and tagsync services start. 
> I have kept below parameter configurable
> 1. Metrics Name (Default value metrics.json)
> 2. Metrics Directory (must have write permission and default value /tmp/)
> 3. frequency at metrics get updated (default value: 10 sec)
> 4. metrics enabled or disabled 
> Checked for metrics.json being regularly updated based on JVM resources 
> utilization. Validated for usersync as well as tagsync process.
> 
> 
> Thanks,
> 
> Dineshkumar Yadav
> 
>

Re: Review Request 72010: RANGER-2688: Make cookie name configurable

[Abhay Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72010/#review219346
---


Ship it!




Ship It!

- Abhay Kulkarni


On Jan. 18, 2020, 6:08 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72010/
> ---
> 
> (Updated Jan. 18, 2020, 6:08 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay 
> Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh 
> Mani, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2688
> https://issues.apache.org/jira/browse/RANGER-2688
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** Current cookie name 'RANGERADMINSESSIONID' is 
> hardcoded in the ranger code at many places. It should be configurable so 
> that hard coding can be avoided.
> 
> **Proposed Solution:** 
> 
> added a property "ranger.admin.cookie.name" in the ranger-admin-site.xml
> added a property "ranger.usersync.dest.ranger.session.cookie.name" in the 
> ranger-usersync-site.xml
> added a property "ranger.tagsync.dest.ranger.session.cookie.name" in the 
> ranger-tagsync-site.xml
> 
> **Note**
> Before starting these modules user should provide same cookie name value in 
> above mentioned configs file and web.xml file. default value shall be 
> "RANGERADMINSESSIONID" only.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerCommonConstants.java
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/common/PropertiesUtil.java 
> ee8ce8dfb 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKrbFilter.java
>  f2856d3f6 
>   security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 2e584a7a8 
>   security-admin/src/main/resources/conf.dist/security-applicationContext.xml 
> 2e7a8910a 
>   tagsync/conf/templates/ranger-tagsync-template.xml 41aacbfd4 
>   tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncConfig.java 
> 6d27b02cd 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/sink/tagadmin/TagAdminRESTSink.java
>  5d32cc04c 
>   tagsync/src/main/resources/ranger-tagsync-default.xml 08afc4266 
>   tagsync/src/main/resources/ranger-tagsync-site.xml 9a14c1cc7 
>   
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapPolicyMgrUserGroupBuilder.java
>  b469e9245 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
>  1d4e37fcf 
>   unixauthservice/conf.dist/ranger-ugsync-default.xml e2e014b00 
>   unixauthservice/scripts/templates/ranger-ugsync-template.xml 0c2d1fcac 
> 
> 
> Diff: https://reviews.apache.org/r/72010/diff/2/
> 
> 
> Testing
> ---
> 
> **Steps to test the patch:**
> 
> build ranger 
> untar the ranger admin and install the ranger-admin.
> open web.xml and change the  --->  --->  
> property value with desired cookie name. 
> save and close the web.xml file
> open ranger-admin-site.xml and provide the cookie name in the property 
> "ranger.admin.cookie.name"
> save and close the ranger-admin-site.xml file
> start ranger-admin and make a REST call; in response ranger will return new 
> cookie name.
> 
> 
> untar and install the ranger usersync.
> open ranger-ugsync-site.xml and update the cookie name for the property 
> "ranger.usersync.dest.ranger.session.cookie.name"
> start the usersync and usersync utility should able to sync users.
> 
> 
> untar and install the ranger tagsync.
> open ranger-tagsync-site.xml and update the cookie name for the property 
> "ranger.tagsync.dest.ranger.session.cookie.name"
> start the usersync and usersync utility should able to sync users.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Commented] (RANGER-2689) Support multiple versions of Hive

[Andrew Charneski (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020400#comment-17020400
 ] 

Andrew Charneski commented on RANGER-2689:
--

My largest concern with that approach is that it will end up forking a 
significant amount of code, while the modifications for backwards-compatibility 
are fairly isolated as demonstrated in the PRs linked in the description. This 
will produce an ongoing technical burden when making changes to the Hive 
plugin(s).

> Support multiple versions of Hive
> -
>
> Key: RANGER-2689
> URL: https://issues.apache.org/jira/browse/RANGER-2689
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andrew Charneski
>Priority: Major
>
> Currently Ranger supports the latest version of Hive, 3.1.2. Unfortunately, 
> there are large segments of the big data community that still rely on older 
> versions of Hive. Two major examples:
> # Spark SQL uses a forked version of Hive 1.2.1 
> (https://spark.apache.org/docs/latest/sql-migration-guide-hive-compatibility.html)
> # EMR provides Hive only up to 2.3.5 
> (https://docs.aws.amazon.com/emr/latest/ReleaseGuide/Hive-release-history.html)
> In order to support these internally, my organization has prepared two 
> modifications of Ranger to link against these versions. These are illustrated 
> in the PRs https://github.com/acharneski/ranger/pull/4 and 
> https://github.com/acharneski/ranger/pull/5
> We would like to eliminate the need for entirely separate builds of Ranger to 
> support this, and integrate these variants into the main Ranger codebase. We 
> are willing to do the bulk of the implementation but would first like to 
> discuss the architecture of this change so as to build it in a way the Ranger 
> committers would be amenable to adopting. 
> My initial thought is to split the `hive-agent` module into something like 
> `hive-agent-base`, `hive-agent-1`, `hive-agent-2`, and `hive-agent-3`. This 
> would allow us to explicitly link to each major version of Hive while 
> minimizing the duplication of code. Thoughts? 
> Thank you!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Call for presentations for ApacheCon North America 2020 now open

[Rich Bowen]

Dear Apache enthusiast,

(You’re receiving this message because you are subscribed to one or more 
project mailing lists at the Apache Software Foundation.)


The call for presentations for ApacheCon North America 2020 is now open 
at https://apachecon.com/acna2020/cfp


ApacheCon will be held at the Sheraton, New Orleans, September 28th 
through October 2nd, 2020.


As in past years, ApacheCon will feature tracks focusing on the various 
technologies within the Apache ecosystem, and so the call for 
presentations will ask you to select one of those tracks, or “General” 
if the content falls outside of one of our already-organized tracks. 
These tracks are:


Karaf
Internet of Things
Fineract
Community
Content Delivery
Solr/Lucene (Search)
Gobblin/Big Data Integration
Ignite
Observability
Cloudstack
Geospatial
Graph
Camel/Integration
Flagon
Tomcat
Cassandra
Groovy
Web/httpd
General/Other

The CFP will close Friday, May 1, 2020 8:00 AM (America/New_York time).

Submit early, submit often, at https://apachecon.com/acna2020/cfp

Rich, for the ApacheCon Planners

[jira] [Updated] (RANGER-2694) TagSync fails to start due to NoClassDefFoundError

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2694:
-
Fix Version/s: 2.1.0

> TagSync fails to start due to NoClassDefFoundError
> --
>
> Key: RANGER-2694
> URL: https://issues.apache.org/jira/browse/RANGER-2694
> Project: Ranger
>  Issue Type: Bug
>  Components: tagsync
>Affects Versions: 2.0.0
>Reporter: Toshihiko Uchida
>Assignee: Madhan Neethiraj
>Priority: Minor
> Fix For: 2.1.0, 2.0.1
>
> Attachments: RANGER-2694.patch
>
>
> TagSync cannot start due to NoClassDefFoundError on 
> org.apache.commons.configuration.PropertiesConfiguration.
> {code:java}
> Exception in thread "main" java.lang.NoClassDefFoundError: 
> org/apache/commons/configuration/PropertiesConfiguration
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
> at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at 
> org.apache.atlas.kafka.NotificationProvider.get(NotificationProvider.java:33)
> at 
> org.apache.ranger.tagsync.source.atlas.AtlasTagSource.initialize(AtlasTagSource.java:103)
> at 
> org.apache.ranger.tagsync.process.TagSynchronizer.initializeTagSources(TagSynchronizer.java:262)
> at 
> org.apache.ranger.tagsync.process.TagSynchronizer.initialize(TagSynchronizer.java:113)
> at 
> org.apache.ranger.tagsync.process.TagSynchronizer.main(TagSynchronizer.java:65)
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.commons.configuration.PropertiesConfiguration
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 17 more
> {code}
> It happens since Ranger 2.0.0 uses commons-configuration2, while Atlas 2.0.0 
> uses commons-configuration.
>  - Ranger: 
> [https://github.com/apache/ranger/blob/89c5ac6/tagsync/pom.xml#L72-L76]
>  - Atlas: 
> [https://github.com/apache/atlas/blob/35ec91a/notification/pom.xml#L119-L123]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2665) Policy engine for delegate-admin processing is not built correctly when policy-deltas are enabled and a zone policy is updated

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2665:
-
Fix Version/s: 2.1.0

> Policy engine for delegate-admin processing is not built correctly when 
> policy-deltas are enabled and a zone policy is updated
> --
>
> Key: RANGER-2665
> URL: https://issues.apache.org/jira/browse/RANGER-2665
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: master, 2.1.0
>
>
> When policy-deltas are enabled, and the only change is to a zone policy, then 
> the policy-engine used for delegate-admin processing fails to build. This may 
> lead to incorrect processing of delegate-admin privileges when the affected 
> policy has delegate-admin policy-item(s).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2666) Rename matchType and matchScope enumerators to more descriptive names

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2666:
-
Fix Version/s: 2.1.0

> Rename matchType and matchScope enumerators to more descriptive names
> -
>
> Key: RANGER-2666
> URL: https://issues.apache.org/jira/browse/RANGER-2666
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: master
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: master, 2.1.0
>
>
> Rename match-type and match-scope enumerators as follows:
> ANCESTOR_WITH_WILDCARDS ==> SELF_AND_ALL_DESCENDANTS



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2668) Remove tag service linking for any service created in Ranger KMS

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2668:
-
Fix Version/s: 2.1.0

> Remove tag service linking for any service created in Ranger KMS
> 
>
> Key: RANGER-2668
> URL: https://issues.apache.org/jira/browse/RANGER-2668
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Dhaval B. SHAH
>Assignee: Dhaval B. SHAH
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: RANGER-2668-01.patch, RANGER-2668-02.patch
>
>
> Ranger KMS shouldn't be having Tag based service hence with regards to this 
> jira.
>  1.) No Tag based service will be associated with RangerKMS default service.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2667) Running the disable-presto-plugin.sh script will always get stuck

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2667:
-
Fix Version/s: 2.1.0

> Running the disable-presto-plugin.sh script will always get stuck
> -
>
> Key: RANGER-2667
> URL: https://issues.apache.org/jira/browse/RANGER-2667
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 2.0.0
>Reporter: Jiayi Liu
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 0001-RANGER-2667.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Running the disable-presto-plugin.sh script will always get stuck. Because 
> the script sets the 
> controlName to an empty string and passes it to the function 
> addOrUpdatePropertyToFile when disable the presto plugin. If controlName is 
> an empty string, addOrUpdatePropertyToFile will ignore this parameter and let 
> fn to be the second parameter. So $3 passed to checkPropertyInFile will be 
> empty. In checkPropertyInFile, if the second parameter is empty, the sed 
> command will never return, and the disable-presto-plugin.sh script will 
> always get stuck.
> We should pass the default value of access-control.name and set controlName 
> to be allow-all when disable presto plugin. 
> The default value of access-control.name can be seen here: 
> https://prestodb.io/docs/current/security/built-in-system-access-control.html
> The pull request: https://github.com/apache/ranger/pull/45



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2671) Running the setup.sh in usersync a second time will change the permissions and owner of hadoop core-site.xml

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2671:
-
Fix Version/s: 2.1.0

> Running the setup.sh in usersync a second time will change the permissions 
> and owner of hadoop core-site.xml
> 
>
> Key: RANGER-2671
> URL: https://issues.apache.org/jira/browse/RANGER-2671
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 2.0.0, 1.2.0
>Reporter: Jiayi Liu
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 0001-RANGER-2671.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> If we run setup.sh in usersync a second time, the setup.py in usersync folder 
> will change the permissions to 0750 and owner to ranger:ranger of hadoop 
> core-site.xml. This will affect other software that needs to read 
> core-site.xml, for example, hiveserver2 will fail to start because it does 
> not have permission to read core-site.xml. Ranger should never change the 
> permission or ownership of core-site.xml in hadoop conf dir. 
> The reason why the permissions and owner of core-site.xml are modified is 
> because the following code in unixauthservice/scripts/setup.py
> {code:python}
> for dir in fixPermList:
> for root, dirs, files in os.walk(dir):
> os.chown(root, ownerId, groupId)
> os.chmod(root, 0755)
> for obj in dirs:
> dn = join(root, obj)
> os.chown(dn, ownerId, groupId)
> os.chmod(dn, 0755)
> for obj in files:
> fn = join(root, obj)
> os.chown(fn, ownerId, groupId)
> os.chmod(fn, 0750)
> {code}
> If we run setup.sh in usersync a second time, there will be a soft link of 
> core-site.xml in /etc/ranger/usersync/conf. In the for loop, it will traverse 
> to /etc/ranger/usersync/conf/core-site.xml, and use os.chown and os.chmod to 
> change the permisson and ownership. We should unlink the soft link of 
> core-site.xml before this for loop. 
> pr: https://github.com/apache/ranger/pull/46



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2674) Allow service admins to manage tag policies

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2674:
-
Fix Version/s: 2.1.0

> Allow service admins to manage tag policies
> ---
>
> Key: RANGER-2674
> URL: https://issues.apache.org/jira/browse/RANGER-2674
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Dhaval B. SHAH
>Assignee: Dhaval B. SHAH
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: RANGER-2674.patch
>
>
> User is able to manage tab based policies once they are marked as service 
> admin in tag service _but only after that particular user is given permission 
> of tab "Tag based policies"._
> As part of this Jira we will extend the functionality in such a way like,
> _Once any particular user is marked as service admin while creating / 
> updating tag based service then we need to provide permission of tab "Tag 
> based policies" to that user._



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2678) RangerUgSyncRESTClient sets password as username for basic auth

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2678:
-
Fix Version/s: 2.1.0

> RangerUgSyncRESTClient sets password as username for basic auth
> ---
>
> Key: RANGER-2678
> URL: https://issues.apache.org/jira/browse/RANGER-2678
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: master
>Reporter: Krzysztof Adamski
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 2.1.0
>
>
> RANGER-2591 introduced changes to usersync (unix/ldap).
> The following line is however broken
> [https://github.com/apache/ranger/blame/master/ugsync/src/main/java/org/apache/ranger/unixusersync/process/RangerUgSyncRESTClient.java#L83]
> getClient().addFilter(new HTTPBasicAuthFilter(getPassword(), getPassword()));
> it should be 
> getClient().addFilter(new HTTPBasicAuthFilter(getUsername(), getPassword()));



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2676) Before obtaining AuthzAuditEvent Object should check xasecure.audit.destination.xx value

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2676:
-
Fix Version/s: (was: 2.0.0)

> Before obtaining AuthzAuditEvent Object should check 
> xasecure.audit.destination.xx value
> 
>
> Key: RANGER-2676
> URL: https://issues.apache.org/jira/browse/RANGER-2676
> Project: Ranger
>  Issue Type: Improvement
>  Components: audit, plugins
>Affects Versions: 1.0.0, 1.1.0, 2.0.0, 1.2.0
>Reporter: Haihui Xu
>Assignee: Haihui Xu
>Priority: Critical
>  Labels: patch
> Attachments: RANGER-2676.patch
>
>
> When after evalute policy and then process the auditlog，need to obtain 
> AuditHandler Object and AuthzAuditEvent object
> (1) AuditHandler object obtained when init plugin according to the value of 
> xasecure.audit.destination.db/solr/hdfs/...
> (2) So, obtaining AuthzAuditEvent should also according to the value of 
> xasecure.audit.destination.db/solr/hdfs/...,or when AuditHandler is null or 
> dummyHandler, constructing AuthzAuditEvent object is useless which is very 
> frequently called and thus result in the  performance decline of service(hdfs 
> namenode, kakfa broker,hiveserver2...)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2679) upgrade spring framework version to 4.3.23 to be compatible with current spring security version(4.2.13)

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2679:
-
Fix Version/s: 2.1.0

> upgrade spring framework version to 4.3.23 to be compatible with current 
> spring security version(4.2.13)
> 
>
> Key: RANGER-2679
> URL: https://issues.apache.org/jira/browse/RANGER-2679
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Nikhil Purbhe
>Assignee: Nikhil Purbhe
>Priority: Major
> Fix For: 2.1.0
>
>
> upgrade spring framework version to 4.3.23 to be compatible with current 
> spring security version(4.2.13)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2680) Ranger Usersync and Tagsync is not updating session cookie until restart

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2680:
-
Fix Version/s: 2.1.0

> Ranger Usersync and Tagsync is not updating session cookie until restart
> 
>
> Key: RANGER-2680
> URL: https://issues.apache.org/jira/browse/RANGER-2680
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Nikhil Purbhe
>Assignee: Nikhil Purbhe
>Priority: Major
> Fix For: master, 2.1.0
>
>
> Ranger Usersync and Tagsync is not updating session cookie until restart, 
> this is generating unnecessary login sessions in ranger admin from usersync 
> and tagsync.
>  
> Ranger clients should update the invalid session cookie and replace it with 
> cookie which is valid and avoid unnecessary login session count increment.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (RANGER-2681) CVE-2019-12397: Apache Ranger cross site scripting issue

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy resolved RANGER-2681.
--
Resolution: Information Provided

> CVE-2019-12397: Apache Ranger cross site scripting issue
> 
>
> Key: RANGER-2681
> URL: https://issues.apache.org/jira/browse/RANGER-2681
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Reporter: Liu Qilei
>Priority: Major
> Attachments: The corresponding Vulnerabilies.PNG
>
>
> which commit is correspond to this vulnerabily?
> And there is another vulnerabily *CVE-2018-11778*, I can not find the 
> corresponding ranger commit either.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (RANGER-2681) CVE-2019-12397: Apache Ranger cross site scripting issue

[Velmurugan Periasamy (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020252#comment-17020252
 ] 

Velmurugan Periasamy commented on RANGER-2681:
--

https://issues.apache.org/jira/browse/RANGER-2172 fixed this issue. 

> CVE-2019-12397: Apache Ranger cross site scripting issue
> 
>
> Key: RANGER-2681
> URL: https://issues.apache.org/jira/browse/RANGER-2681
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Reporter: Liu Qilei
>Priority: Major
> Attachments: The corresponding Vulnerabilies.PNG
>
>
> which commit is correspond to this vulnerabily?
> And there is another vulnerabily *CVE-2018-11778*, I can not find the 
> corresponding ranger commit either.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2685) set_globals.sh will throw an exception "cp ews/logs/* failed"

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2685:
-
Fix Version/s: 2.1.0

> set_globals.sh will throw an exception "cp ews/logs/* failed"
> -
>
> Key: RANGER-2685
> URL: https://issues.apache.org/jira/browse/RANGER-2685
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Reporter: Jiayi Liu
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 0001-RANGER-2685.patch
>
>
> set_globals.sh try to copy files in ews/logs although there are no files in 
> ews/logs, and the cp command will throw an exception "cp: cannot stat 
> ‘ews/logs/*’: No such file or directory". We need to check if there are files 
> in the folder.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2692) RangerKafkaAuthorizer support for ConsumerGroup resource for authorization

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2692:
-
Fix Version/s: 2.1.0

> RangerKafkaAuthorizer support for ConsumerGroup resource for authorization
> --
>
> Key: RANGER-2692
> URL: https://issues.apache.org/jira/browse/RANGER-2692
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Minor
> Fix For: 2.1.0
>
> Attachments: 
> 0001-RANGER-2692-RangerKafkaAuthorizer-support-for-Consum.patch, 
> 0001-RANGER-2692-RangerKafkaAuthorizer-support-for-Consum.patch2
>
>
> RangerKafkaAuthorizer support for ConsumerGroup resource for authorization



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2691) Upgrade with MsSQL fails because of invalid syntax

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2691:
-
Fix Version/s: 2.1.0

> Upgrade with MsSQL fails because of invalid syntax
> --
>
> Key: RANGER-2691
> URL: https://issues.apache.org/jira/browse/RANGER-2691
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Reporter: László Dénes Terjéki
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: 0001-RANGER-2691-Fix-MsSQL-syntax-in-patch.patch
>
>
>  
> [https://github.com/apache/ranger/blob/ac4ef50b01fe35b858fe8ef45a8b5f896e44662b/security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql]
>  contains invalid SQL for MsSQL which can cause upgrade issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2702) Upgrade Kafka Version in Ranger to 2.4

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2702:
-
Fix Version/s: 2.1.0

> Upgrade Kafka Version in Ranger to 2.4
> --
>
> Key: RANGER-2702
> URL: https://issues.apache.org/jira/browse/RANGER-2702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 2.1.0
>
>
> Upgrade Kafka Version in Ranger to 2.4.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2701) Improve Logging mechanism for Ranger KMS

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2701:
-
Fix Version/s: 2.1.0

> Improve Logging mechanism for Ranger KMS
> 
>
> Key: RANGER-2701
> URL: https://issues.apache.org/jira/browse/RANGER-2701
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Dhaval B. SHAH
>Assignee: Dhaval B. SHAH
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: RANGER-2701-01.patch
>
>
>  Change the logging level from INFO to DEBUG of certain logs in Ranger KMS.
> E.G. 
> {code:java}
> // code placeholder
> 2019-12-13 14:45:25,844 INFO KMS - Entering decryptEncryptedKey method.
> 2019-12-13 14:45:25,866 INFO KMS - Exiting handleEncryptedKeyOp method.{code}
> This logs are useful while debugging the issue.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2703) Grant operation succeeds even though there is no delegate admin permission

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2703:
-
Fix Version/s: 2.1.0

> Grant operation succeeds even though there is no delegate admin permission
> --
>
> Key: RANGER-2703
> URL: https://issues.apache.org/jira/browse/RANGER-2703
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: master, 2.1.0
>
>
> Grant or revoke operations in Kerberized environment create or update Ranger 
> policy  even when the grantor does not have delegated admin access on the 
> resource being granted access to.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2696) Fix unit test failure in TestPolicyEngine.testValiditySchedulerApplicable

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2696:
-
Fix Version/s: 2.1.0

> Fix unit test failure in TestPolicyEngine.testValiditySchedulerApplicable
> -
>
> Key: RANGER-2696
> URL: https://issues.apache.org/jira/browse/RANGER-2696
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master, ranger-2.0, ranger-1.2
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Minor
> Fix For: master, 2.1.0, ranger-2.0, ranger-1.2
>
>
> Test data contains a specific date which is now past. Fix is to update it to 
> date far into future.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2695) Default displayName for ServiceDef

[Velmurugan Periasamy (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2695:
-
Fix Version/s: 2.1.0

> Default displayName for ServiceDef
> --
>
> Key: RANGER-2695
> URL: https://issues.apache.org/jira/browse/RANGER-2695
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Kishor Gollapalliwar
>Assignee: Kishor Gollapalliwar
>Priority: Major
> Fix For: 2.1.0
>
>
> Ranger creates ServiceDef(s) as part of initial setup. While creating these 
> ServiceDef(s) it refers respective JSON file. If display name for a 
> ServiceDef is not provided OR it is empty, ServiceDef's name should be used 
> as DEFAULT displayName for that ServiceDef.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster

[Leandro Loos (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020237#comment-17020237
 ] 

Leandro Loos commented on RANGER-2621:
--

[~susidev33] were you able to work the issue out? I'm facing the same problem 
on a nearly equal scenario (but with my own kerberos servers also on different 
EC2 instances)

> Ranger Policy Update fails on Kerberized Cluster
> 
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster 
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate 
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine.. 
> When EMR is kerberized, policy download does not work anymore...
>  
> We see below error:
> +*Access Log:*+ 
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET 
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
>  HTTP/1.1" 401 52 "-" "curl/7.61.1"
>  
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient 
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting 
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS), 
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication 
> Failed"}, serviceName=hivedev
>  
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show 
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>  
>  
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status:  Enabled
>  
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net 
> Password :   
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver 
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net 
> Common Name for Certificate: 
> Add New Configurations 
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>  
>  
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation 
> around the installation and configuration, we are all handicapped when it 
> comes to using. Appreciate if some of you add good documentation, it helps us 
> appreciate the amount of work done by you ... Right now, we are only shooting 
> in the DARK.*{color} 
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (RANGER-2708) Ranger public group check uses wrong comparison for equality

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke reassigned RANGER-2708:


Assignee: Lars Francke

> Ranger public group check uses wrong comparison for equality
> 
>
> Key: RANGER-2708
> URL: https://issues.apache.org/jira/browse/RANGER-2708
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Major
> Attachments: RANGER-2708.1.patch
>
>
> In {{RangerBizUtil#checkUsrPermForPolicy}} we can find this piece of code:
> {code:java}
> matchFound = (publicGroupId != null && publicGroupId == permMap.getGroupId()) 
> || isGroupInList(permMap.getGroupId(), userGroups);
> {code}
> {{publicGroupId}} and {{permMap.getGroupId()}} are both Long so testing 
> equality using "{{==}}" is wrong and only works "by accident" or not at all 
> depending on the JVM.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (RANGER-2706) usermapsyncenabled can be removed from code

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke reassigned RANGER-2706:


Assignee: Lars Francke

> usermapsyncenabled can be removed from code
> ---
>
> Key: RANGER-2706
> URL: https://issues.apache.org/jira/browse/RANGER-2706
> Project: Ranger
>  Issue Type: Improvement
>  Components: usersync
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Minor
> Attachments: RANGER-2706.1.patch
>
>
> RANGER-742 removed the need for the usermapsyncenabled property but it's 
> still being read, printed in the logs and I believe it can even be configured 
> in Ambari but all it does is confuse so this should be removed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (RANGER-2707) ranger.usersync.enabled not respected on initial ugsync startup

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke reassigned RANGER-2707:


Assignee: Lars Francke

> ranger.usersync.enabled not respected on initial ugsync startup
> ---
>
> Key: RANGER-2707
> URL: https://issues.apache.org/jira/browse/RANGER-2707
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Reporter: Lars Francke
>Assignee: Lars Francke
>Priority: Major
> Attachments: RANGER-2707.1.patch
>
>
> ranger.usersync.enabled can be used to control whether usersync is enabled or 
> not.
> Unfortunately UserGroupSync unconditionally does run a sync when it initially 
> starts no matter what this setting is set to.
> This is annoying if Ambari automatically starts usersync



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Add to contributor list

[Lars Francke]

Thank you!

On Tue, Jan 21, 2020 at 1:58 PM Velmurugan Periasamy  wrote:

> Hi - welcome to Ranger community. I have added you as a contributor.
> Thanks.
>
> On Tue, Jan 21, 2020 at 5:50 AM Lars Francke 
> wrote:
> Hi,
>
> could someone add me "larsfrancke" to the contributor group so I can assign
> myself issues?
>
> Thank you,
> Lars

[jira] [Comment Edited] (RANGER-2689) Support multiple versions of Hive

[Velmurugan Periasamy (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020214#comment-17020214
 ] 

Velmurugan Periasamy edited comment on RANGER-2689 at 1/21/20 1:03 PM:
---

Instead of splitting the hive-agent module, consider creating separate modules 
for supporting specific hive versions. Default hive-agent would support the 
latest version and can create separate modules for older versions? CC [~abhay] 
/ [~rmani]


was (Author: vperiasamy):
Instead of splitting the hive-agent module, consider creating separate modules 
for supporting specific hive versions. Default hive-agent would support the 
latest version and can create separate modules for older versions? 

> Support multiple versions of Hive
> -
>
> Key: RANGER-2689
> URL: https://issues.apache.org/jira/browse/RANGER-2689
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andrew Charneski
>Priority: Major
>
> Currently Ranger supports the latest version of Hive, 3.1.2. Unfortunately, 
> there are large segments of the big data community that still rely on older 
> versions of Hive. Two major examples:
> # Spark SQL uses a forked version of Hive 1.2.1 
> (https://spark.apache.org/docs/latest/sql-migration-guide-hive-compatibility.html)
> # EMR provides Hive only up to 2.3.5 
> (https://docs.aws.amazon.com/emr/latest/ReleaseGuide/Hive-release-history.html)
> In order to support these internally, my organization has prepared two 
> modifications of Ranger to link against these versions. These are illustrated 
> in the PRs https://github.com/acharneski/ranger/pull/4 and 
> https://github.com/acharneski/ranger/pull/5
> We would like to eliminate the need for entirely separate builds of Ranger to 
> support this, and integrate these variants into the main Ranger codebase. We 
> are willing to do the bulk of the implementation but would first like to 
> discuss the architecture of this change so as to build it in a way the Ranger 
> committers would be amenable to adopting. 
> My initial thought is to split the `hive-agent` module into something like 
> `hive-agent-base`, `hive-agent-1`, `hive-agent-2`, and `hive-agent-3`. This 
> would allow us to explicitly link to each major version of Hive while 
> minimizing the duplication of code. Thoughts? 
> Thank you!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (RANGER-2689) Support multiple versions of Hive

[Velmurugan Periasamy (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020214#comment-17020214
 ] 

Velmurugan Periasamy commented on RANGER-2689:
--

Instead of splitting the hive-agent module, consider creating separate modules 
for supporting specific hive versions. Default hive-agent would support the 
latest version and can create separate modules for older versions? 

> Support multiple versions of Hive
> -
>
> Key: RANGER-2689
> URL: https://issues.apache.org/jira/browse/RANGER-2689
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Andrew Charneski
>Priority: Major
>
> Currently Ranger supports the latest version of Hive, 3.1.2. Unfortunately, 
> there are large segments of the big data community that still rely on older 
> versions of Hive. Two major examples:
> # Spark SQL uses a forked version of Hive 1.2.1 
> (https://spark.apache.org/docs/latest/sql-migration-guide-hive-compatibility.html)
> # EMR provides Hive only up to 2.3.5 
> (https://docs.aws.amazon.com/emr/latest/ReleaseGuide/Hive-release-history.html)
> In order to support these internally, my organization has prepared two 
> modifications of Ranger to link against these versions. These are illustrated 
> in the PRs https://github.com/acharneski/ranger/pull/4 and 
> https://github.com/acharneski/ranger/pull/5
> We would like to eliminate the need for entirely separate builds of Ranger to 
> support this, and integrate these variants into the main Ranger codebase. We 
> are willing to do the bulk of the implementation but would first like to 
> discuss the architecture of this change so as to build it in a way the Ranger 
> committers would be amenable to adopting. 
> My initial thought is to split the `hive-agent` module into something like 
> `hive-agent-base`, `hive-agent-1`, `hive-agent-2`, and `hive-agent-3`. This 
> would allow us to explicitly link to each major version of Hive while 
> minimizing the duplication of code. Thoughts? 
> Thank you!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Add to contributor list

[Velmurugan Periasamy]

Hi - welcome to Ranger community. I have added you as a contributor. Thanks.

On Tue, Jan 21, 2020 at 5:50 AM Lars Francke  wrote:
Hi,

could someone add me "larsfrancke" to the contributor group so I can assign
myself issues?

Thank you,
Lars

Review Request 72032: RANGER-2708 Ranger public group check uses wrong comparison for equality

[Lars Francke]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72032/
---

Review request for ranger.


Bugs: RANGER-2708
https://issues.apache.org/jira/browse/RANGER-2708


Repository: ranger


Description
---

In RangerBizUtil#checkUsrPermForPolicy we can find this piece of code:

matchFound = (publicGroupId != null && publicGroupId == 
permMap.getGroupId()) || isGroupInList(permMap.getGroupId(), userGroups);

publicGroupId and permMap.getGroupId() are both Long so testing equality using 
"==" is wrong and only works "by accident" or not at all depending on the JVM.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
21308b1e7 


Diff: https://reviews.apache.org/r/72032/diff/1/


Testing
---


Thanks,

Lars Francke

[jira] [Updated] (RANGER-2708) Ranger public group check uses wrong comparison for equality

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2708:
-
Attachment: RANGER-2708.1.patch

> Ranger public group check uses wrong comparison for equality
> 
>
> Key: RANGER-2708
> URL: https://issues.apache.org/jira/browse/RANGER-2708
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Lars Francke
>Priority: Major
> Attachments: RANGER-2708.1.patch
>
>
> In {{RangerBizUtil#checkUsrPermForPolicy}} we can find this piece of code:
> {code:java}
> matchFound = (publicGroupId != null && publicGroupId == permMap.getGroupId()) 
> || isGroupInList(permMap.getGroupId(), userGroups);
> {code}
> {{publicGroupId}} and {{permMap.getGroupId()}} are both Long so testing 
> equality using "{{==}}" is wrong and only works "by accident" or not at all 
> depending on the JVM.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (RANGER-2708) Ranger public group check uses wrong comparison for equality

[Lars Francke (Jira)]

Lars Francke created RANGER-2708:


 Summary: Ranger public group check uses wrong comparison for 
equality
 Key: RANGER-2708
 URL: https://issues.apache.org/jira/browse/RANGER-2708
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Lars Francke


In {{RangerBizUtil#checkUsrPermForPolicy}} we can find this piece of code:
{code:java}
matchFound = (publicGroupId != null && publicGroupId == permMap.getGroupId()) 
|| isGroupInList(permMap.getGroupId(), userGroups);
{code}
{{publicGroupId}} and {{permMap.getGroupId()}} are both Long so testing 
equality using "{{==}}" is wrong and only works "by accident" or not at all 
depending on the JVM.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Add to contributor list

[Lars Francke]

Hi,

could someone add me "larsfrancke" to the contributor group so I can assign
myself issues?

Thank you,
Lars

Review Request 72031: RANGER-2707 ranger.usersync.enabled not respected on initial ugsync startup

[Lars Francke]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72031/
---

Review request for ranger.


Bugs: RANGER-2707
https://issues.apache.org/jira/browse/RANGER-2707


Repository: ranger


Description
---

ranger.usersync.enabled can be used to control whether usersync is enabled or 
not.

Unfortunately UserGroupSync unconditionally does run a sync when it initially 
starts no matter what this setting is set to.

This is annoying if Ambari automatically starts usersync


I took this opportunity to clean up the UGSync class a bit


Diffs
-

  ugsync/src/main/java/org/apache/ranger/usergroupsync/UserGroupSync.java 
2befe33ea 


Diff: https://reviews.apache.org/r/72031/diff/1/


Testing
---


Thanks,

Lars Francke

[jira] [Updated] (RANGER-2707) ranger.usersync.enabled not respected on initial ugsync startup

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2707:
-
Attachment: RANGER-2707.1.patch

> ranger.usersync.enabled not respected on initial ugsync startup
> ---
>
> Key: RANGER-2707
> URL: https://issues.apache.org/jira/browse/RANGER-2707
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Reporter: Lars Francke
>Priority: Major
> Attachments: RANGER-2707.1.patch
>
>
> ranger.usersync.enabled can be used to control whether usersync is enabled or 
> not.
> Unfortunately UserGroupSync unconditionally does run a sync when it initially 
> starts no matter what this setting is set to.
> This is annoying if Ambari automatically starts usersync



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (RANGER-2707) ranger.usersync.enabled not respected on initial ugsync startup

[Lars Francke (Jira)]

Lars Francke created RANGER-2707:


 Summary: ranger.usersync.enabled not respected on initial ugsync 
startup
 Key: RANGER-2707
 URL: https://issues.apache.org/jira/browse/RANGER-2707
 Project: Ranger
  Issue Type: Bug
  Components: usersync
Reporter: Lars Francke


ranger.usersync.enabled can be used to control whether usersync is enabled or 
not.

Unfortunately UserGroupSync unconditionally does run a sync when it initially 
starts no matter what this setting is set to.

This is annoying if Ambari automatically starts usersync



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Code conventions

[Lars Francke]

Hi Ranger devs,

I've stumbled through the Ranger code base in the last few days and might
have a few things to contribute but I'm wondering if you have any kind of
code conventions?

I'm asking because there's tabs being used as well as spaces and that's
just one of the things that struck me. It seems to be "all over the place"
which doesn't make it easier to get started. I see some references to
checkstyle but no config for it.

So, two questions:
1) Do you have any code conventions?

2) Would you accept whitespace/style only patches to "clean" it up a bit?
At least to be consistent within a single file.

Cheers,
Lars

Review Request 72030: RANGER-2706 usermapsyncenabled can be removed from code

[Lars Francke]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72030/
---

Review request for ranger.


Bugs: RANGER-2706
https://issues.apache.org/jira/browse/RANGER-2706


Repository: ranger


Description
---

RANGER-742 removed the need for the usermapsyncenabled property but it's still 
being read, printed in the logs and I believe it can even be configured in 
Ambari but all it does is confuse so this should be removed.


Diffs
-

  migration-util/ambari2.1-hdp2.3-ranger0.50/bin/import_ranger_to_ambari.py 
e25b8ae89 
  ugsync/ldapconfigchecktool/ldapconfigcheck/conf/input.properties 449ee6f44 
  
ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
 920ebf0b3 
  
ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapDeltaUserGroupBuilder.java
 ca50f098a 
  
ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
 8efa1613a 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
 1d4e37fcf 
  ugsync/src/test/resources/ranger-ugsync-site.xml 0a1a86da0 
  unixauthservice/scripts/templates/installprop2xml.properties fa342fbff 
  unixauthservice/scripts/templates/ranger-ugsync-template.xml 0c2d1fcac 


Diff: https://reviews.apache.org/r/72030/diff/1/


Testing
---


Thanks,

Lars Francke

[jira] [Updated] (RANGER-2706) usermapsyncenabled can be removed from code

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2706:
-
Attachment: RANGER-2706.1.patch

> usermapsyncenabled can be removed from code
> ---
>
> Key: RANGER-2706
> URL: https://issues.apache.org/jira/browse/RANGER-2706
> Project: Ranger
>  Issue Type: Improvement
>  Components: usersync
>Reporter: Lars Francke
>Priority: Minor
> Attachments: RANGER-2706.1.patch
>
>
> RANGER-742 removed the need for the usermapsyncenabled property but it's 
> still being read, printed in the logs and I believe it can even be configured 
> in Ambari but all it does is confuse so this should be removed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2705) Group sync does does not parse DNs properly

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2705:
-
Description: 
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

 Our DN/CN looks like this:
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma-separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug. It should use the {{LdapName}} class instead and/or parse according to the 
RFC 2253 but maybe even that is wrong what it really should probably do is the 
same as user sync?

 This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 

  was:
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

 Our DN/CN looks like this:
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma-separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug. It should use the {{LdapName}} class instead and/or parse according to the 
RFC 2253.

 This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 


> Group sync does does not parse DNs properly
> ---
>
> Key: RANGER-2705
> URL: https://issues.apache.org/jira/browse/RANGER-2705
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Reporter: Lars Francke
>Priority: Major
>
> When we have enabled user & group search 
> ({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
> get the groups and its members and compare them to what already exists.
>  Our DN/CN looks like this:
> {code:java}
>  CN=Francke\, Lars,OU=bla bla. 
> {code}
> Our CN contains a comma but the {{getShortUserName}} method in 
> {{LdapDeltaUserGroupBuilder}} has this piece of code:
> {code:java}
> StringTokenizer stc = new StringTokenizer(longUserName, ",");
> String firstToken = stc.nextToken();{code}
> The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first 
> part of the comma-separated DN) but that doesn't work if that contains a 
> comma itself. It is escaped but Ranger just splits at the comma. That's 
> definitely a bug. It should use the {{LdapName}} class instead and/or parse 
> according to the RFC 2253 but maybe even that is wrong what it really should 
> probably do is the same as user sync?
>  This way we currently cannot use group sync at all because if we do we don't 
> get any groups at all as the user search doesn't take its own groups when 
> group sync is also enabled (this was another surprise).
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (RANGER-2706) usermapsyncenabled can be removed from code

[Lars Francke (Jira)]

Lars Francke created RANGER-2706:


 Summary: usermapsyncenabled can be removed from code
 Key: RANGER-2706
 URL: https://issues.apache.org/jira/browse/RANGER-2706
 Project: Ranger
  Issue Type: Improvement
  Components: usersync
Reporter: Lars Francke


RANGER-742 removed the need for the usermapsyncenabled property but it's still 
being read, printed in the logs and I believe it can even be configured in 
Ambari but all it does is confuse so this should be removed.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2705) Group sync does does not parse DNs properly

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2705:
-
Description: 
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

 Our DN/CN looks like this:
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma-separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug. It should use the {{LdapName}} class instead and/or parse according to the 
RFC 2253.

 This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 

  was:
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
which returns something along the lines of "AB12" for us (user Ids). The 
members' list that is returned from the groups, however, includes a full DN for 
each member:

 
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
 

There are two problems here:

1. Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:

 
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug. It should use the `LdapName` class instead and/or parse according to the 
RFC 2253.

 

2. Even if 1. would work there is no user "Francke, Lars" in Ranger. Ranger 
would need to use that DN to look up the user and then apply the 
{{user.nameattribute}} to match it to existing users.

Update: I believe point 2 to be incorrect as it looks up the "fullUserName" in 
the "userNameMap".

 

This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 

 

 

This is made much harder by being all completely underdocumented ;)

I guess it'd make sense to create two issues for this and I can do that 
tomorrow.

 


> Group sync does does not parse DNs properly
> ---
>
> Key: RANGER-2705
> URL: https://issues.apache.org/jira/browse/RANGER-2705
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Reporter: Lars Francke
>Priority: Major
>
> When we have enabled user & group search 
> ({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
> get the groups and its members and compare them to what already exists.
>  Our DN/CN looks like this:
> {code:java}
>  CN=Francke\, Lars,OU=bla bla. 
> {code}
> Our CN contains a comma but the {{getShortUserName}} method in 
> {{LdapDeltaUserGroupBuilder}} has this piece of code:
> {code:java}
> StringTokenizer stc = new StringTokenizer(longUserName, ",");
> String firstToken = stc.nextToken();{code}
> The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first 
> part of the comma-separated DN) but that doesn't work if that contains a 
> comma itself. It is escaped but Ranger just splits at the comma. That's 
> definitely a bug. It should use the {{LdapName}} class instead and/or parse 
> according to the RFC 2253.
>  This way we currently cannot use group sync at all because if we do we don't 
> get any groups at all as the user search doesn't take its own groups when 
> group sync is also enabled (this was another surprise).
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2705) Group sync does does not parse DNs properly

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2705:
-
Summary: Group sync does does not parse DNs properly  (was: Group sync does 
not respect ldap.user.nameattribute)

> Group sync does does not parse DNs properly
> ---
>
> Key: RANGER-2705
> URL: https://issues.apache.org/jira/browse/RANGER-2705
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Reporter: Lars Francke
>Priority: Major
>
> When we have enabled user & group search 
> ({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
> get the groups and its members and compare them to what already exists.
> Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
> which returns something along the lines of "AB12" for us (user Ids). The 
> members' list that is returned from the groups, however, includes a full DN 
> for each member:
>  
> {code:java}
>  CN=Francke\, Lars,OU=bla bla. 
> {code}
>  
> There are two problems here:
> 1. Our CN contains a comma but the {{getShortUserName}} method in 
> {{LdapDeltaUserGroupBuilder}} has this piece of code:
>  
> {code:java}
> StringTokenizer stc = new StringTokenizer(longUserName, ",");
> String firstToken = stc.nextToken();{code}
> The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first 
> part of the comma separated DN) but that doesn't work if that contains a 
> comma itself. It is escaped but Ranger just splits at the comma. That's 
> definitely a bug. It should use the `LdapName` class instead and/or parse 
> according to the RFC 2253.
>  
> 2. Even if 1. would work there is no user "Francke, Lars" in Ranger. Ranger 
> would need to use that DN to look up the user and then apply the 
> {{user.nameattribute}} to match it to existing users.
> Update: I believe point 2 to be incorrect as it looks up the "fullUserName" 
> in the "userNameMap".
>  
> This way we currently cannot use group sync at all because if we do we don't 
> get any groups at all as the user search doesn't take its own groups when 
> group sync is also enabled (this was another surprise).
>  
>  
>  
> This is made much harder by being all completely underdocumented ;)
> I guess it'd make sense to create two issues for this and I can do that 
> tomorrow.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-2705) Group sync does not respect ldap.user.nameattribute

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2705:
-
Description: 
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
which returns something along the lines of "AB12" for us (user Ids). The 
members' list that is returned from the groups, however, includes a full DN for 
each member:

 
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
 

There are two problems here:

1. Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:

 
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug. It should use the `LdapName` class instead and/or parse according to the 
RFC 2253.

 

2. Even if 1. would work there is no user "Francke, Lars" in Ranger. Ranger 
would need to use that DN to look up the user and then apply the 
{{user.nameattribute}} to match it to existing users.

Update: I believe point 2 to be incorrect as it looks up the "fullUserName" in 
the "userNameMap".

 

This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 

 

 

This is made much harder by being all completely underdocumented ;)

I guess it'd make sense to create two issues for this and I can do that 
tomorrow.

 

  was:
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
which returns something along the lines of "AB12" for us (user Ids). The 
members' list that is returned from the groups, however, includes a full DN for 
each member:

 
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
 

There are two problems here:

1. Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:

 
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug.

 

2. Even if 1. would work there is no user "Francke, Lars" in Ranger. Ranger 
would need to use that DN to look up the user and then apply the 
{{user.nameattribute}} to match it to existing users.

Update: I believe point 2 to be incorrect as it looks up the "fullUserName" in 
the "userNameMap".

 

This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 

 

 

This is made much harder by being all completely underdocumented ;)

I guess it'd make sense to create two issues for this and I can do that 
tomorrow.

 


> Group sync does not respect ldap.user.nameattribute
> ---
>
> Key: RANGER-2705
> URL: https://issues.apache.org/jira/browse/RANGER-2705
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Reporter: Lars Francke
>Priority: Major
>
> When we have enabled user & group search 
> ({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
> get the groups and its members and compare them to what already exists.
> Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
> which returns something along the lines of "AB12" for us (user Ids). The 
> members' list that is returned from the groups, however, includes a full DN 
> for each member:
>  
> {code:java}
>  CN=Francke\, Lars,OU=bla bla. 
> {code}
>  
> There are two problems here:
> 1. Our CN contains a comma but the {{getShortUserName}} method in 
> {{LdapDeltaUserGroupBuilder}} has this piece of code:
>  
> {code:java}
> StringTokenizer stc = new StringTokenizer(longUserName, ",");
> String firstToken = stc.nextToken();{code}
> The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first 
> part of the comma separated DN) but that doesn't 

[jira] [Updated] (RANGER-2705) Group sync does not respect ldap.user.nameattribute

[Lars Francke (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Francke updated RANGER-2705:
-
Description: 
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
which returns something along the lines of "AB12" for us (user Ids). The 
members' list that is returned from the groups, however, includes a full DN for 
each member:

 
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
 

There are two problems here:

1. Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:

 
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug.

 

2. Even if 1. would work there is no user "Francke, Lars" in Ranger. Ranger 
would need to use that DN to look up the user and then apply the 
{{user.nameattribute}} to match it to existing users.

Update: I believe point 2 to be incorrect as it looks up the "fullUserName" in 
the "userNameMap".

 

This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 

 

 

This is made much harder by being all completely underdocumented ;)

I guess it'd make sense to create two issues for this and I can do that 
tomorrow.

 

  was:
When we have enabled user & group search 
({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
get the groups and its members and compare them to what already exists.

Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
which returns something along the lines of "AB12" for us (user Ids). The 
members' list that is returned from the groups, however, includes a full DN for 
each member:

 
{code:java}
 CN=Francke\, Lars,OU=bla bla. 
{code}
 

There are two problems here:

1. Our CN contains a comma but the {{getShortUserName}} method in 
{{LdapDeltaUserGroupBuilder}} has this piece of code:

 
{code:java}
StringTokenizer stc = new StringTokenizer(longUserName, ",");
String firstToken = stc.nextToken();{code}
The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first part 
of the comma separated DN) but that doesn't work if that contains a comma 
itself. It is escaped but Ranger just splits at the comma. That's definitely a 
bug.

 

2. Even if 1. would work there is no user "Francke, Lars" in Ranger. Ranger 
would need to use that DN to look up the user and then apply the 
{{user.nameattribute}} to match it to existing users.

This way we currently cannot use group sync at all because if we do we don't 
get any groups at all as the user search doesn't take its own groups when group 
sync is also enabled (this was another surprise).

 

 

 

This is made much harder by being all completely underdocumented ;)

I guess it'd make sense to create two issues for this and I can do that 
tomorrow.

 


> Group sync does not respect ldap.user.nameattribute
> ---
>
> Key: RANGER-2705
> URL: https://issues.apache.org/jira/browse/RANGER-2705
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Reporter: Lars Francke
>Priority: Major
>
> When we have enabled user & group search 
> ({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to 
> get the groups and its members and compare them to what already exists.
> Unfortunately, we have set the user.nameattribute to the usual sAMAccountName 
> which returns something along the lines of "AB12" for us (user Ids). The 
> members' list that is returned from the groups, however, includes a full DN 
> for each member:
>  
> {code:java}
>  CN=Francke\, Lars,OU=bla bla. 
> {code}
>  
> There are two problems here:
> 1. Our CN contains a comma but the {{getShortUserName}} method in 
> {{LdapDeltaUserGroupBuilder}} has this piece of code:
>  
> {code:java}
> StringTokenizer stc = new StringTokenizer(longUserName, ",");
> String firstToken = stc.nextToken();{code}
> The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first 
> part of the comma separated DN) but that doesn't work if that contains a 
> comma itself. It is escaped but Ranger just splits at the comma. That's 
> definitely a bug.
>  
> 2. Even if 1. would work there is no user "Francke, Lars" in