[jira] [Assigned] (RANGER-3682) Unify the ways that rangerkeystore to encapsulate zonekey

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel reassigned RANGER-3682:


Assignee: kirby zhou

> Unify the ways that rangerkeystore to encapsulate zonekey
> -
>
> Key: RANGER-3682
> URL: https://issues.apache.org/jira/browse/RANGER-3682
> Project: Ranger
>  Issue Type: Improvement
>  Components: kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
>
> Unify the ways that rangerkeystore to encapsulate zonekey
> Now we have 2 styles of MasterKeyProvider:
>  # RangerMasterKey, RangerHSM, RangerSafenetKeySecure
>  # RangerAzureKeyVaultKeyGenerator, RangerGoogleCloudHSMProvider, 
> RangerTencentKMSProvider
> Style 1 can get out master key string from provider, Style 2 can not.
> In old, I add a flag KeyVaultEnabled to distinguish them. 
> KeyVaultEnabled=false means style1, true means style2
> RangerKeyStore with  style1 use SecretKeyEntry with SealedObject to store a 
> key and do encryption / decryption by itself.
> RangerKeyStore with  style2 use SecretKeyByteEntry to store a key and let MK 
> provider to encryption / decryption.
> These are ugly and hard to maintain. I refactor it by removing 
> SecretKeyEntry, and let providers of style1 do encryption / decryption. 
> Add a  common base class of RangerMasterKey, RangerHSM andd 
> RangerSafenetKeySecure, named AbstractRangerMasterKey. It provides the common 
> logic of encryptZoneKey and decryptZoneKey. AbstractRangerMasterKey encodes 
> SealedObject into byte[].
> So the new code does not change the actual storage format, and there is no 
> problem in compatibility.
> =
>  
> And, there is no unified method to initialize a master key provider. 
> Duplicate code is distributed in RangerKeyStoreProvider and a bunch of CLI 
> classes.
> I made a new RangerKMSMKIFactory class to unify it.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3687) Password Policy Best Practices for Strong Security

2022-04-07 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519316#comment-17519316
 ] 

Bhavik Patel commented on RANGER-3687:
--

Anyone is interested to resolve issue for FIPS enabled environment ?

> Password Policy Best Practices for Strong Security
> --
>
> Key: RANGER-3687
> URL: https://issues.apache.org/jira/browse/RANGER-3687
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 3.0.0
>Reporter: Bhavik Patel
>Priority: Major
>
> # Password history should be configured to restrict users from reusing their 
> last 4 or 5 passwords.
>  # Forcing users to change passwords every 90-180 days 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3693) Ranger - Upgrade tomcat to 8.5.78

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel reassigned RANGER-3693:


Assignee: Mateen Mansoori

> Ranger - Upgrade tomcat to 8.5.78
> -
>
> Key: RANGER-3693
> URL: https://issues.apache.org/jira/browse/RANGER-3693
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Mateen N Mansoori
>Assignee: Mateen Mansoori
>Priority: Major
>
> Currently ranger is pulling tomcat - 8.5.76, This task is to upgrade tomcat 
> version to 8.5.78.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3696) java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory

2022-04-07 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519315#comment-17519315
 ] 

Bhavik Patel commented on RANGER-3696:
--

yeah, we have to fix ES plugin. As of now I'm not using this plugin so can’t  
provide fix for this.

Anyone is interested in providing the fix?

> java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
> ---
>
> Key: RANGER-3696
> URL: https://issues.apache.org/jira/browse/RANGER-3696
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 2.2.0
> Environment: Apache Ranger ElasticSearch Plugin: 
> ranger-2.2.0-elasticsearch-plugin.tar.gz
> elasticsearch version: 7.6.0 
> OS: Ubuntu 20.04.4
>Reporter: MohdSiddique Bagwan
>Priority: Blocker
>
> Please find the versions I am using 
> *Apache Ranger ElasticSearch Plugin:* ranger-2.2.0-elasticsearch-plugin.tar.gz
> *elasticsearch version:* 7.6.0 
> *OS:* Ubuntu 20.04.4
> I installed the apache ranger elasticsearch plugin on elastic search host, 
> while starting elasticsearch service I am getting below error:
> Note: Without ranger plugin the elasticsearch plugin is working perfect. It 
> would be very helpful if you redirect me to documentation on how to install 
> ranger-2.2.0-elasticsearch-plugin.tar.gz on 7.6.0 & above. 
> {code:java}
> service elasticsearch start
>  * Starting Elasticsearch Server                                              
>                                                                               
>                                                sysctl: setting key 
> "vm.max_map_count", ignoring: Read-only file system
> OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in 
> version 9.0 and will likely be removed in a future release.
>                                                                               
>                                                                               
>                                         [ OK ]
> root@3b8fcbe634f3:~# fatal error in thread [main], exiting
> java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
>         at 
> org.apache.ranger.authorization.elasticsearch.plugin.RangerElasticsearchPlugin.(RangerElasticsearchPlugin.java:52)
>         at 
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>  Method)
>         at 
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>         at 
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at 
> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
>         at 
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
>         at 
> org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:607)
>         at 
> org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556)
>         at 
> org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471)
>         at 
> org.elasticsearch.plugins.PluginsService.(PluginsService.java:163)
>         at org.elasticsearch.node.Node.(Node.java:313)
>         at org.elasticsearch.node.Node.(Node.java:257)
>         at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:221)
>         at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221)
>         at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349)
>         at 
> org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
>         at 
> org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
>         at 
> org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
>         at 
> org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125)
>         at org.elasticsearch.cli.Command.main(Command.java:90)
>         at 
> org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
>         at 
> org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
> Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
>         at 
> java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
>         at 
> java.base/java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:864)
>         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
>         ... 22 more {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3661) Ranger - Upgrade netty to 4.1.72-final

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel reassigned RANGER-3661:


Assignee: Bhavik Patel

> Ranger - Upgrade netty to 4.1.72-final
> --
>
> Key: RANGER-3661
> URL: https://issues.apache.org/jira/browse/RANGER-3661
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Mateen N Mansoori
>Assignee: Bhavik Patel
>Priority: Major
> Fix For: 3.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3661) Ranger - Upgrade netty to 4.1.72-final

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel reassigned RANGER-3661:


Assignee: Mateen Mansoori  (was: Bhavik Patel)

> Ranger - Upgrade netty to 4.1.72-final
> --
>
> Key: RANGER-3661
> URL: https://issues.apache.org/jira/browse/RANGER-3661
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Mateen N Mansoori
>Assignee: Mateen Mansoori
>Priority: Major
> Fix For: 3.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3698) Ranger - Upgrade kylin to 3.1.3

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel reassigned RANGER-3698:


Assignee: Mateen Mansoori

> Ranger - Upgrade kylin to 3.1.3
> ---
>
> Key: RANGER-3698
> URL: https://issues.apache.org/jira/browse/RANGER-3698
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Mateen N Mansoori
>Assignee: Mateen Mansoori
>Priority: Major
>
> Ranger is currently pulling in kylin 2.6.6, This task is to track kylin 
> verison upgrade to 3.1.3



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3702) RANGER - Export policy in excel is failing.

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel reassigned RANGER-3702:


Assignee: Mateen Mansoori

> RANGER - Export policy in excel is failing.
> ---
>
> Key: RANGER-3702
> URL: https://issues.apache.org/jira/browse/RANGER-3702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Mateen N Mansoori
>Assignee: Mateen Mansoori
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Export policy in excel is failing with the below error : 
>  
> {code:java}
> java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
>   at org.apache.poi.POIDocument.(POIDocument.java:65)
>   at 
> org.apache.ranger.biz.ServiceDBStore.writeExcel(ServiceDBStore.java:3991)
>   at 
> org.apache.ranger.biz.ServiceDBStore.getPoliciesInExcel(ServiceDBStore.java:2428)
>   at 
> org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(ServiceREST.java:1985) 
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3699) Ranger - Upgrade poi to 5.2.1+

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel reassigned RANGER-3699:


Assignee: Mateen Mansoori

> Ranger - Upgrade poi to 5.2.1+
> --
>
> Key: RANGER-3699
> URL: https://issues.apache.org/jira/browse/RANGER-3699
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Mateen N Mansoori
>Assignee: Mateen Mansoori
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (RANGER-3677) Update Password Policy validation at WEB-UI

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel resolved RANGER-3677.
--
Resolution: Fixed

> Update Password Policy validation at WEB-UI 
> 
>
> Key: RANGER-3677
> URL: https://issues.apache.org/jira/browse/RANGER-3677
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Bhavik Patel
>Assignee: Dhaval Rajpara
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-RANGER-3677.patch
>
>
> Password validation should be updated at Web-UI as per latest changes 
> RANGER-3624. & [RANGER-3678|https://issues.apache.org/jira/browse/RANGER-3678]
> At-lease one upper-case and one lower-case alphabet must be present in the 
> password. 
> [~Dhaval.Rajpara]  [~nitin.galave] can you please help on this?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3677) Update Password Policy validation at WEB-UI

2022-04-07 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519311#comment-17519311
 ] 

Bhavik Patel commented on RANGER-3677:
--

master branch commit link: 
[https://github.com/apache/ranger/commit/758926e9ba99d3e6c9331cb8bef09058c1a660f0]

ranger-2.3 commit link: 
https://github.com/apache/ranger/commit/de268eaa39cc3770ef5f6603fba8c47212df9823

> Update Password Policy validation at WEB-UI 
> 
>
> Key: RANGER-3677
> URL: https://issues.apache.org/jira/browse/RANGER-3677
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Bhavik Patel
>Assignee: Dhaval Rajpara
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-RANGER-3677.patch
>
>
> Password validation should be updated at Web-UI as per latest changes 
> RANGER-3624. & [RANGER-3678|https://issues.apache.org/jira/browse/RANGER-3678]
> At-lease one upper-case and one lower-case alphabet must be present in the 
> password. 
> [~Dhaval.Rajpara]  [~nitin.galave] can you please help on this?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (RANGER-3677) Update Password Policy validation at WEB-UI

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel updated RANGER-3677:
-
Fix Version/s: 3.0.0
   2.3.0

> Update Password Policy validation at WEB-UI 
> 
>
> Key: RANGER-3677
> URL: https://issues.apache.org/jira/browse/RANGER-3677
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Bhavik Patel
>Assignee: Dhaval Rajpara
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-RANGER-3677.patch
>
>
> Password validation should be updated at Web-UI as per latest changes 
> RANGER-3624. & [RANGER-3678|https://issues.apache.org/jira/browse/RANGER-3678]
> At-lease one upper-case and one lower-case alphabet must be present in the 
> password. 
> [~Dhaval.Rajpara]  [~nitin.galave] can you please help on this?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Review Request 72024: RANGER-2704 : Support browser login using kerberized authentication.

2022-04-07 Thread Mehul Parikh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72024/#review224271
---


Ship it!




Ship It!

- Mehul Parikh


On April 5, 2022, 12:24 p.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72024/
> ---
> 
> (Updated April 5, 2022, 12:24 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, 
> Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Madhan 
> Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2704
> https://issues.apache.org/jira/browse/RANGER-2704
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Need to support browser login using kerberos authentication. Added a logout 
> for an unauthenticated user to redirect to the login page.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKrbFilter.java
>  223a991c76bae7d25f5ce89604d0a8a90d426fe5 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
>  abbf2d983beb30b59e5d3f6429d6fc226f735793 
>   security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml 
> 0a1128613dca50fe67ea3f891261f1ee449c46db 
> 
> 
> Diff: https://reviews.apache.org/r/72024/diff/2/
> 
> 
> Testing
> ---
> 
> Veriried kerberos ticket authentication is working on a kerberized browser.
> 
> 
> Steps to test for a kerberized browser:
> #1) For Kerberized browsers:
> #1> To open Chrome in kerberos enabled mode need to run below command:
>google-chrome --auth-server-whitelist="*ranger.testserver.com"
> #2> For Firefox, need to go to about:configs and then search for 
> negotiate and then add the host domain
> ranger.testserver.com to the property 
> "network.negotiate-auth.trusted-uris"
> #2) Perform kinit with the required user.
> #3) Open the Ranger Admin portal using FQDN of the server host.
> 
> 
> Known Issue: If there is no valid kerberos ticket, user lands on a blank page 
> and a short hack is to either append locallogin to the URL or refresh the 
> browser tab to redirect to the login page.
> P.S: this issue is not observed on Google Chrome browser
> 
> 
> File Attachments
> 
> 
> RANGER-2704.patch
>   
> https://reviews.apache.org/media/uploaded/files/2020/01/17/8c9682ca-1ade-4281-89e7-d43a8af09300__RANGER-2704.patch
> RANGER-2704.02.patch
>   
> https://reviews.apache.org/media/uploaded/files/2022/04/04/6e737bec-e640-4459-922c-353793172b12__RANGER-2704.02.patch
> RANGER-2704.03.patch
>   
> https://reviews.apache.org/media/uploaded/files/2022/04/05/31e52557-051e-40ba-bc34-5dc6418e06f8__RANGER-2704.03.patch
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

[jira] [Commented] (RANGER-3697) Migrate all python scripts in Ranger to Python3

2022-04-07 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519295#comment-17519295
 ] 

Bhavik Patel commented on RANGER-3697:
--

scripts should support both the python version (2 & 3). Any specific reason of 
changing the file name?

> Migrate all python scripts in Ranger to Python3
> ---
>
> Key: RANGER-3697
> URL: https://issues.apache.org/jira/browse/RANGER-3697
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, Ranger
>Reporter: Abhishek Kumar
>Assignee: Abhishek Kumar
>Priority: Major
>
> Python 2 has sunset as of Jan 1, 2020. All scripts using python2 should 
> eventually move to python3.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Resolved] (RANGER-3144) Please add me as a contributor

2022-04-07 Thread Bhavik Patel (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bhavik Patel resolved RANGER-3144.
--
Resolution: Done

> Please add me as a contributor
> --
>
> Key: RANGER-3144
> URL: https://issues.apache.org/jira/browse/RANGER-3144
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger
>Reporter: Zeashan Pappa
>Assignee: Zeashan Pappa
>Priority: Trivial
>
> [~bosco] can you please add me as a contributor so I can push PRs for 
> improvements and fixes against the apache-ranger python client.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3632) Improve ranger logs, RENAME_ON_ROTATE and others

2022-04-07 Thread Pradeep Agrawal (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519292#comment-17519292
 ] 

Pradeep Agrawal commented on RANGER-3632:
-

ranger-2.3 only.

> Improve ranger logs,  RENAME_ON_ROTATE and others
> -
>
> Key: RANGER-3632
> URL: https://issues.apache.org/jira/browse/RANGER-3632
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Currently, the filename of the access-log in use has a timestamp as the 
> suffix. This brings trouble to some log monitoring and analysis programs, 
> such as "tail -f access-log"
> Need to add an option to enable tomcat's RenameOnRotate capability to fix the 
> file name of access-log.
>  
> {code:java}
> // in EmbeddedServer::start()
> valve.setRenameOnRotate(
> EmbeddedServerUtil.getConfig(ACCESS_LOG_RENAME_ON_ROTATE,  false);
> );{code}
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Review Request 73936: RANGER-3695 : Ranger Keystore alias should be configurable

2022-04-07 Thread bhavik patel 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73936/#review224269
---




embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
Line 167 (original), 167 (patched)


default should be "rangeradmin".


- bhavik patel


On April 7, 2022, 4:41 p.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73936/
> ---
> 
> (Updated April 7, 2022, 4:41 p.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, 
> Gautam Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Madhan 
> Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3695
> https://issues.apache.org/jira/browse/RANGER-3695
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger requires keystore alias for TLS, However keystore alias should be  an 
> optional parameter, hence should be only configured
> if provided by the user.
> Fix contains changes to make the keystore alias optional.
> 
> 
> Diffs
> -
> 
>   
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
>  cae9075a7b7726ad5abf2b52f53f612d4223f712 
> 
> 
> Diff: https://reviews.apache.org/r/73936/diff/1/
> 
> 
> Testing
> ---
> 
> Validated changes on a local VM with TLS enabled.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

Re: Review Request 73937: RANGER-3697: Python3 migration of all ranger python scripts

2022-04-07 Thread Kirby Zhou 


> On 四月 7, 2022, 10:14 p.m., Ramesh Mani wrote:
> > security-admin/scripts/change_password_util.py
> > Lines 1 (patched)
> > 
> >
> > Deleting file changepasswordutil.py and creating with a new file name 
> > change_password_util.py may result in failure of the file not to be 
> > included in package or it may be reference in other scripts which has to be 
> > changed. Please refer all those reference and correct it or best to use the 
> > same name.

I suggest that this commit DO NOT change any filename, open a new review to 
change filename.


- Kirby


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73937/#review224265
---


On 四月 7, 2022, 7:11 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73937/
> ---
> 
> (Updated 四月 7, 2022, 7:11 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3697
> https://issues.apache.org/jira/browse/RANGER-3697
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Python 2 has sunset as of Jan 1, 2020. All scripts using python2 should 
> eventually move to python3.
> 
> Initial Review - will be updated to include all scripts.
> 
> Changes include:
> 1. Support only for Python3 and above.
> 2. Conformity with PEP 8 guidelines.
> 3. Code optimizations and performance improvements.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/upgrade-plugin.py d865ee366 
>   agents-common/scripts/upgrade_plugin.py PRE-CREATION 
>   ranger-util/src/scripts/saveVersion.py 51227542d 
>   ranger-util/src/scripts/save_version.py PRE-CREATION 
>   security-admin/scripts/change_password_util.py PRE-CREATION 
>   security-admin/scripts/change_username_util.py PRE-CREATION 
>   security-admin/scripts/changepasswordutil.py e45dab643 
>   security-admin/scripts/changeusernameutil.py 699f945f0 
>   security-admin/scripts/deleteUserGroupUtil.py 1c9f58385 
>   security-admin/scripts/delete_user_group_util.py PRE-CREATION 
>   security-admin/scripts/ranger_credential_helper.py 85f29ac43 
>   security-admin/scripts/restrict_permissions.py b19bafe9b 
>   security-admin/scripts/role_based_user_search_util.py PRE-CREATION 
>   security-admin/scripts/rolebasedusersearchutil.py 612db33df 
>   security-admin/scripts/updateUserAndGroupNamesInJson.py c40ec4406 
>   security-admin/scripts/upgrade_admin.py 85f57b8ce 
>   security-admin/src/bin/ranger_install.py 39b9d1f81 
>   security-admin/src/bin/ranger_usersync.py 4374896c7 
>   security-admin/src/bin/service_start.py ea13b85af 
> 
> 
> Diff: https://reviews.apache.org/r/73937/diff/1/
> 
> 
> Testing
> ---
> 
> Pending.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

Re: Review Request 73937: RANGER-3697: Python3 migration of all ranger python scripts

2022-04-07 Thread Kirby Zhou 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73937/#review224266
---



"#!/usr/bin/python"

should be replace to

"#!/usr/bin/env python3"

for 2 reason

1. python is missing in RHEL-8, only python2 / python3
2. Some system do not install python3 in /usr/

- Kirby Zhou


On 四月 7, 2022, 7:11 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73937/
> ---
> 
> (Updated 四月 7, 2022, 7:11 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3697
> https://issues.apache.org/jira/browse/RANGER-3697
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Python 2 has sunset as of Jan 1, 2020. All scripts using python2 should 
> eventually move to python3.
> 
> Initial Review - will be updated to include all scripts.
> 
> Changes include:
> 1. Support only for Python3 and above.
> 2. Conformity with PEP 8 guidelines.
> 3. Code optimizations and performance improvements.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/upgrade-plugin.py d865ee366 
>   agents-common/scripts/upgrade_plugin.py PRE-CREATION 
>   ranger-util/src/scripts/saveVersion.py 51227542d 
>   ranger-util/src/scripts/save_version.py PRE-CREATION 
>   security-admin/scripts/change_password_util.py PRE-CREATION 
>   security-admin/scripts/change_username_util.py PRE-CREATION 
>   security-admin/scripts/changepasswordutil.py e45dab643 
>   security-admin/scripts/changeusernameutil.py 699f945f0 
>   security-admin/scripts/deleteUserGroupUtil.py 1c9f58385 
>   security-admin/scripts/delete_user_group_util.py PRE-CREATION 
>   security-admin/scripts/ranger_credential_helper.py 85f29ac43 
>   security-admin/scripts/restrict_permissions.py b19bafe9b 
>   security-admin/scripts/role_based_user_search_util.py PRE-CREATION 
>   security-admin/scripts/rolebasedusersearchutil.py 612db33df 
>   security-admin/scripts/updateUserAndGroupNamesInJson.py c40ec4406 
>   security-admin/scripts/upgrade_admin.py 85f57b8ce 
>   security-admin/src/bin/ranger_install.py 39b9d1f81 
>   security-admin/src/bin/ranger_usersync.py 4374896c7 
>   security-admin/src/bin/service_start.py ea13b85af 
> 
> 
> Diff: https://reviews.apache.org/r/73937/diff/1/
> 
> 
> Testing
> ---
> 
> Pending.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

[jira] [Commented] (RANGER-3632) Improve ranger logs, RENAME_ON_ROTATE and others

2022-04-07 Thread kirby zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519250#comment-17519250
 ] 

kirby zhou commented on RANGER-3632:


I will check it, the failure happens on ranger-2.3 or master or both?

> Improve ranger logs,  RENAME_ON_ROTATE and others
> -
>
> Key: RANGER-3632
> URL: https://issues.apache.org/jira/browse/RANGER-3632
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Currently, the filename of the access-log in use has a timestamp as the 
> suffix. This brings trouble to some log monitoring and analysis programs, 
> such as "tail -f access-log"
> Need to add an option to enable tomcat's RenameOnRotate capability to fix the 
> file name of access-log.
>  
> {code:java}
> // in EmbeddedServer::start()
> valve.setRenameOnRotate(
> EmbeddedServerUtil.getConfig(ACCESS_LOG_RENAME_ON_ROTATE,  false);
> );{code}
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3619) REST API should return 403 when authenticated client is not allowed to access API.

2022-04-07 Thread kirby zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519248#comment-17519248
 ] 

kirby zhou commented on RANGER-3619:


Simple test with dual-instance HA done, nothing special happens.

 

> REST API should return 403 when authenticated client is not allowed to access 
> API.
> --
>
> Key: RANGER-3619
> URL: https://issues.apache.org/jira/browse/RANGER-3619
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.2.0
>Reporter: kirby zhou
>Priority: Major
>
> REST API should return 403-Forbidden when authenticated client is not allowed 
> to access API to avoid crash Ranger Clients.
>  
> Now, some API returns 401-Unauthorized instead of 403-Forbidden when client 
> is already passed authentication but now allowed to do something.
> In general, this will not cause any serious problems. However, there is a 
> flaw in the SPNEGO protocol implementation of Java HTTPUrlConnection. It 
> causes the Client to throw an unexpected exception. This will trouble the 
> operators and developers.
>  
> Let me show you how it happens:
>  
> For example:
>  
> The RangerAdminClient inside KMS  want to access API 
> "/service/secure/policies/download", but the principal is not in the 
> allowlist.
>  
>  # RangerAdminClient is based on Jersey-Client
>  # JerseyClient sends a HTTP-request to Ranger Service without authentication 
> information
>  # Tomcat/Spring inside Ranger returns 401 with HTTP header 
> “WWW-Authentication: Neogotiate”
>  # JerseyClient sends request again with Kerberos/SPNEGO authentication 
> tokens.
>  # Tomcat/Spring inside Ranger accept the authentication, then call 
> ServiceRest::getSecureServicePoliciesIfUpdated to reply the API calling.
>  # ServiceRest::getSecureServicePoliciesIfUpdated checks allowlist of “kms 
> service”, and refuse client with 401.
>  # Tomcat/Spring inside Ranger returns 401 with HTTP header 
> “WWW-Authentication: Neogotiate….” for notifying RangerAdminClient the 
> authentication is passed.
>  
> Now, there is a malformed state. HTTP-status code told client authenticate is 
> failed, but HTTP header told client authentication is passed.
>  
> In the RangerAdminClient side, 
>  
>  # sun.net.www.protocol.http.HttpURLConnection.getInputStream0() see the 
> second 401.
>  # 'inNegotiate' = true, so it is in the progress of _Negotiate._
>  # It checks that: if "WWW-Authenticate: Negotiate" exist then disable 
> negotiate for following code to avoid try {_}Negotiate once again{_}.
>  # But "WWW-Authenticate: Negotiate xczsd324…" does not the rule above.
>  # So HttpURLConnection calls AuthenticationInfo.sendHeaders to generate a 
> new request header.
>  # Wow, Null exception happens.
>  # Logs "ERROR RangerAdminRESTClient - Error getting policies; Received NULL 
> response!!. secureMode=true, user=… (auth:KERBEROS), serviceName=kmsdev"
>  # Log of KMS: "ERROR RangerAdminRESTClient - Failed to get response, Error 
> is : java.lang.RuntimeException: java.lang.NullPointerException"
>  
> This log makes admin confused.
>  
>  
> {code:java}
> //ServiceRest::getServicePoliciesIfUpdated
> if (isAllowed) {
> //...
> } else {
>httpCode = HttpServletResponse.SC_UNAUTHORIZED;
> }
>  {code}
> {code:java}
> // sun.net.www.protocol.http.HttpURLConnection.getInputStream0()
> // Read comments labeled "Failed Negotiate" for details.
> boolean dontUseNegotiate = false;
> Iterator iter = responses.multiValueIterator("WWW-Authenticate");
> while (iter.hasNext()) {
> String value = iter.next().trim();
> if (value.equalsIgnoreCase("Negotiate") ||
> value.equalsIgnoreCase("Kerberos")) {
> if (!inNegotiate) {
> inNegotiate = true;
> } else {
> dontUseNegotiate = true;
> doingNTLM2ndStage = false;
> serverAuthentication = null;
> }
> break;
> }
> }
> /**
>  * Failed Negotiate
>  *
>  * In some cases, the Negotiate auth is supported for the
>  * remote host but the negotiate process still fails (For
>  * example, if the web page is located on a backend server
>  * and delegation is needed but fails). The authentication
>  * process will start again, and we need to detect this
>  * kind of failure and do proper fallback (say, to NTLM).
>  *
>  * In order to achieve this, the inNegotiate flag is set
>  * when the first negotiate challenge is met (and reset
>  * if authentication is finished). If a fresh new negotiate
>  * challenge (no parameter) is found while inNegotiate is
>  * set, we know there's a failed auth attempt recently.
>  * Here we'll ignore the header line so that fallback
>  * can be practiced.
>  *
>  * inNegotiateProxy is for proxy authentication.
>  */
>   {code}
>  

Re: Review Request 73937: RANGER-3697: Python3 migration of all ranger python scripts

2022-04-07 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73937/#review224265
---




security-admin/scripts/change_password_util.py
Lines 1 (patched)


Deleting file changepasswordutil.py and creating with a new file name 
change_password_util.py may result in failure of the file not to be included in 
package or it may be reference in other scripts which has to be changed. Please 
refer all those reference and correct it or best to use the same name.


- Ramesh Mani


On April 7, 2022, 7:11 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73937/
> ---
> 
> (Updated April 7, 2022, 7:11 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3697
> https://issues.apache.org/jira/browse/RANGER-3697
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Python 2 has sunset as of Jan 1, 2020. All scripts using python2 should 
> eventually move to python3.
> 
> Initial Review - will be updated to include all scripts.
> 
> Changes include:
> 1. Support only for Python3 and above.
> 2. Conformity with PEP 8 guidelines.
> 3. Code optimizations and performance improvements.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/upgrade-plugin.py d865ee366 
>   agents-common/scripts/upgrade_plugin.py PRE-CREATION 
>   ranger-util/src/scripts/saveVersion.py 51227542d 
>   ranger-util/src/scripts/save_version.py PRE-CREATION 
>   security-admin/scripts/change_password_util.py PRE-CREATION 
>   security-admin/scripts/change_username_util.py PRE-CREATION 
>   security-admin/scripts/changepasswordutil.py e45dab643 
>   security-admin/scripts/changeusernameutil.py 699f945f0 
>   security-admin/scripts/deleteUserGroupUtil.py 1c9f58385 
>   security-admin/scripts/delete_user_group_util.py PRE-CREATION 
>   security-admin/scripts/ranger_credential_helper.py 85f29ac43 
>   security-admin/scripts/restrict_permissions.py b19bafe9b 
>   security-admin/scripts/role_based_user_search_util.py PRE-CREATION 
>   security-admin/scripts/rolebasedusersearchutil.py 612db33df 
>   security-admin/scripts/updateUserAndGroupNamesInJson.py c40ec4406 
>   security-admin/scripts/upgrade_admin.py 85f57b8ce 
>   security-admin/src/bin/ranger_install.py 39b9d1f81 
>   security-admin/src/bin/ranger_usersync.py 4374896c7 
>   security-admin/src/bin/service_start.py ea13b85af 
> 
> 
> Diff: https://reviews.apache.org/r/73937/diff/1/
> 
> 
> Testing
> ---
> 
> Pending.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

[jira] [Resolved] (RANGER-3689) Ranger : ranger-2.3 Port missing commits.

2022-04-07 Thread Pradeep Agrawal (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pradeep Agrawal resolved RANGER-3689.
-
  Assignee: Pradeep Agrawal
Resolution: Done

> Ranger : ranger-2.3 Port missing commits.
> -
>
> Key: RANGER-3689
> URL: https://issues.apache.org/jira/browse/RANGER-3689
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Affects Versions: 2.3.0
>Reporter: Mateen N Mansoori
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 2.3.0
>
>
> The ranger-2.3 branch is approximately 70 commits behind the master branch, 
> using this ticket to track the porting of missing commits. 
> List of commits which are missing from ranger-2.3 : 
> |1|[RANGER-3435: Add unique index on guid and service id column of 
> x_poli…|https://github.com/apache/ranger/commit/ec7e57e284a50773f8106a748b117692e9a81105]|
> |2|[RANGER-3439: REST api to get or delete ranger policy based on guid 
> an…|https://github.com/apache/ranger/commit/db9f9a488e99092b9c0dba125dcb5e5efa2ad9a6]|
> |3|[RANGER-3433: Null Dereference in ServiceREST getPolicyByName 
> method|https://github.com/apache/ranger/commit/1639896aac695057971c7b73a0b91265b3c3e772]|
> |4|[RANGER-3023: Permission tab takes longer time to load with large 
> numb…|https://github.com/apache/ranger/commit/5732060da4c4c88c1fba3c89fa5369ea56c9b942]
>  …|
> |5|[RANGER-3509: updateRoles() REST API updated to permit 
> role-admins|https://github.com/apache/ranger/commit/e2566827e963afe8b939f4f1a22ccab22716ba04]|
> |6|[RANGER-3505: modified code to ignore case while validating a user 
> for…|https://github.com/apache/ranger/commit/5ca622fedeb0db6738ebe4a7628ccdbcc7d22fbd]|
> |7|[RANGER-3510 : Ranger upgrade spring framework version to 
> 5.3.12|https://github.com/apache/ranger/commit/63aeb5285c3259e6669f88ffbe4192aedd833733]|
> |8|[RANGER-3504 : Create framework to execute DB patch dependent on Java 
> …|https://github.com/apache/ranger/commit/dc6dc621fc99f1dbff355c2e2ac00472155a0baf]|
> |9|[RANGER-3516 : J10045 patch is taking more time during 
> upgrade|https://github.com/apache/ranger/commit/8068996e42d79a8c0d9bb56b77bb4ec82bfe4113]|
> |10|[RANGER-3519: Provide an option to optimize space needed by Trie 
> objects|https://github.com/apache/ranger/commit/71888f243d38ae7cff5e0406c7d54a386d269664]|
> |11|[RANGER-3519: Provide an option to optimize space needed by Trie objects 
> - 
> part2|https://github.com/apache/ranger/commit/5852efde1cba728ad580231ad02145ea72861186]|
> |12|[RANGER-3439: Add rest api to get or delete ranger policy based on guid - 
> part2|https://github.com/apache/ranger/commit/000e6351ee4628979a20e2b72ac6f226e6dd1c0e]|
> |13|[RANGER-3507:Handle trailing slash in the ranger Hive URL policy 
> autho…|https://github.com/apache/ranger/commit/d8f674d3fab849aee7daf8e49a21856fdee82c34]|
> |14|[RANGER-3514: Java patch to update sync source on 
> upgrades|https://github.com/apache/ranger/commit/5fb097fda8c51dc9fe671e4105e8b8a7fb5697cd]|
> |15|[RANGER-3515: Enhance Ranger Java client SSL config to be configured 
> u…|https://github.com/apache/ranger/commit/b56aa63a9e1b2020e208c170642a96f5d62cd892]|
> |16|[RANGER-3522: Improve Tagsync authentication error 
> reporting|https://github.com/apache/ranger/commit/3f82858760e01ed186a2b3055c95b9cdd343db4b]|
> |17|[RANGER-3522: Improve Tagsync authentication error reporting - 
> Part-2|https://github.com/apache/ranger/commit/03f6d3f18f8576d710928be4b148143b8a9f8d91]|
> |18|[RANGER-3493: Add unique index on service and resource_signature 
> colum…|https://github.com/apache/ranger/commit/de8f5e197fb93fcb924f7a59a88013b99bd1194b]|
> |19|[RANGER-3511: Create Java patch to update policy resource-signature 
> to…|https://github.com/apache/ranger/commit/4fdb3af5fc21f43ab22b2fb4d0e411b500460cbc]|
> |20|[RANGER-3490: Make policy resource signature is unique in a 
> service|https://github.com/apache/ranger/commit/856571c4348e31725498c0922338339c76ebba02]|
> |21|[RANGER-3276 Remove duplicate code from 
> buildks|https://github.com/apache/ranger/commit/3045345f3dea4fa44cc522df7b171d6fb3bd5303]|
> |22|[RANGER-3518: Limit the query size stored in Audit 
> logs|https://github.com/apache/ranger/commit/a7b527bbd0df8ba86eee7b3fdc65b470bbbc17fa]|
> |23|[RANGER-3528 : Ranger Group creation audit is not shown during 
> service…|https://github.com/apache/ranger/commit/bb9b3cd14d5ebdb5381ca4a03db27b469c2277e1]|
> |24|[RANGER-3468: Fixed an issue where inactivity timeout request is not 
> h…|https://github.com/apache/ranger/commit/6678ef77438d1289e0ade0cc2e7652a6bd836621]|
> |25|[RANGER-3438: Optimized code to extract GroupPrincipals from javax 
> Sub…|https://github.com/apache/ranger/commit/84cdf593423f03c3082db3baee9bb89149205b5a]|
> |26|[RANGER-3435: Add unique index on guid, service and

[jira] [Commented] (RANGER-3697) Migrate all python scripts in Ranger to Python3

2022-04-07 Thread Abhishek Kumar (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519116#comment-17519116
 ] 

Abhishek Kumar commented on RANGER-3697:


[Review Request:https://reviews.apache.org/r/73937/]

> Migrate all python scripts in Ranger to Python3
> ---
>
> Key: RANGER-3697
> URL: https://issues.apache.org/jira/browse/RANGER-3697
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, Ranger
>Reporter: Abhishek Kumar
>Assignee: Abhishek Kumar
>Priority: Major
>
> Python 2 has sunset as of Jan 1, 2020. All scripts using python2 should 
> eventually move to python3.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (RANGER-3697) Migrate all python scripts in Ranger to Python3

2022-04-07 Thread Abhishek Kumar (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519116#comment-17519116
 ] 

Abhishek Kumar edited comment on RANGER-3697 at 4/7/22 7:22 PM:


[Review Request|https://reviews.apache.org/r/73937/]


was (Author: abhi_2110):
[Review Request:https://reviews.apache.org/r/73937/]

> Migrate all python scripts in Ranger to Python3
> ---
>
> Key: RANGER-3697
> URL: https://issues.apache.org/jira/browse/RANGER-3697
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, Ranger
>Reporter: Abhishek Kumar
>Assignee: Abhishek Kumar
>Priority: Major
>
> Python 2 has sunset as of Jan 1, 2020. All scripts using python2 should 
> eventually move to python3.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73937: RANGER-3697: Python3 migration of all ranger python scripts

2022-04-07 Thread Abhishek Kumar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73937/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
Sailaja Polavarapu, and Velmurugan Periasamy.


Bugs: RANGER-3697
https://issues.apache.org/jira/browse/RANGER-3697


Repository: ranger


Description
---

Python 2 has sunset as of Jan 1, 2020. All scripts using python2 should 
eventually move to python3.

Initial Review - will be updated to include all scripts.

Changes include:
1. Support only for Python3 and above.
2. Conformity with PEP 8 guidelines.
3. Code optimizations and performance improvements.


Diffs
-

  agents-common/scripts/upgrade-plugin.py d865ee366 
  agents-common/scripts/upgrade_plugin.py PRE-CREATION 
  ranger-util/src/scripts/saveVersion.py 51227542d 
  ranger-util/src/scripts/save_version.py PRE-CREATION 
  security-admin/scripts/change_password_util.py PRE-CREATION 
  security-admin/scripts/change_username_util.py PRE-CREATION 
  security-admin/scripts/changepasswordutil.py e45dab643 
  security-admin/scripts/changeusernameutil.py 699f945f0 
  security-admin/scripts/deleteUserGroupUtil.py 1c9f58385 
  security-admin/scripts/delete_user_group_util.py PRE-CREATION 
  security-admin/scripts/ranger_credential_helper.py 85f29ac43 
  security-admin/scripts/restrict_permissions.py b19bafe9b 
  security-admin/scripts/role_based_user_search_util.py PRE-CREATION 
  security-admin/scripts/rolebasedusersearchutil.py 612db33df 
  security-admin/scripts/updateUserAndGroupNamesInJson.py c40ec4406 
  security-admin/scripts/upgrade_admin.py 85f57b8ce 
  security-admin/src/bin/ranger_install.py 39b9d1f81 
  security-admin/src/bin/ranger_usersync.py 4374896c7 
  security-admin/src/bin/service_start.py ea13b85af 


Diff: https://reviews.apache.org/r/73937/diff/1/


Testing
---

Pending.


Thanks,

Abhishek  Kumar

[jira] [Comment Edited] (RANGER-3691) Upgrade spring to 5.3.18 CVE-2022-22965

2022-04-07 Thread Ramesh Mani (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519090#comment-17519090
 ] 

Ramesh Mani edited comment on RANGER-3691 at 4/7/22 6:41 PM:
-

[~kirbyzhou]  could you please refer to me the log4j-logback patches which you 
are mentioning?


was (Author: rmani):
[~kirbyzhou]  could you please refer the log4j-logback patches which you are 
mentioning?

> Upgrade spring to 5.3.18 CVE-2022-22965
> ---
>
> Key: RANGER-3691
> URL: https://issues.apache.org/jira/browse/RANGER-3691
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, kms
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Blocker
> Fix For: 3.0.0
>
>
> [https://tanzu.vmware.com/security/cve-2022-22965|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-22965]
> [https://github.com/spring-projects/spring-framework/releases]
>  
> Spring has a new 0day Remote-Code-Execution problem, related to spring-beans 
> and JDK9+
> Fixed at spring 5.3.18 / 5.2.20
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3691) Upgrade spring to 5.3.18 CVE-2022-22965

2022-04-07 Thread Ramesh Mani (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17519090#comment-17519090
 ] 

Ramesh Mani commented on RANGER-3691:
-

[~kirbyzhou]  could you please refer the log4j-logback patches which you are 
mentioning?

> Upgrade spring to 5.3.18 CVE-2022-22965
> ---
>
> Key: RANGER-3691
> URL: https://issues.apache.org/jira/browse/RANGER-3691
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, kms
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Blocker
> Fix For: 3.0.0
>
>
> [https://tanzu.vmware.com/security/cve-2022-22965|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-22965]
> [https://github.com/spring-projects/spring-framework/releases]
>  
> Spring has a new 0day Remote-Code-Execution problem, related to spring-beans 
> and JDK9+
> Fixed at spring 5.3.18 / 5.2.20
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73936: RANGER-3695 : Ranger Keystore alias should be configurable

2022-04-07 Thread Vishal Suvagia via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73936/
---

Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, Gautam 
Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, 
Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
Periasamy.


Bugs: RANGER-3695
https://issues.apache.org/jira/browse/RANGER-3695


Repository: ranger


Description
---

Ranger requires keystore alias for TLS, However keystore alias should be  an 
optional parameter, hence should be only configured
if provided by the user.
Fix contains changes to make the keystore alias optional.


Diffs
-

  
embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
 cae9075a7b7726ad5abf2b52f53f612d4223f712 


Diff: https://reviews.apache.org/r/73936/diff/1/


Testing
---

Validated changes on a local VM with TLS enabled.


Thanks,

Vishal Suvagia

[jira] [Updated] (RANGER-3695) Ranger Keystore alias should be configurable

2022-04-07 Thread Vishal Suvagia (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vishal Suvagia updated RANGER-3695:
---
Attachment: RANGER-3695.patch

> Ranger Keystore alias should be configurable
> 
>
> Key: RANGER-3695
> URL: https://issues.apache.org/jira/browse/RANGER-3695
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin
>Affects Versions: 3.0.0
>Reporter: Vishal Suvagia
>Assignee: Vishal Suvagia
>Priority: Major
> Attachments: RANGER-3695.patch
>
>
> Ranger keystore alias is currently hard-coded, it should be configurable to 
> allow user to provide a custom alias



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73935: RANGER-3669 : Connection to DB fails for MySQL version above 8.0

2022-04-07 Thread Vishal Suvagia via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73935/
---

Review request for ranger, Ankita Sinha, Dhaval Shah, Dineshkumar Yadav, Gautam 
Borad, Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, 
Mehul Parikh, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
Periasamy.


Bugs: RANGER-3669
https://issues.apache.org/jira/browse/RANGER-3669


Repository: ranger


Description
---

Ranger KMS db setup script needs to be updated to support MySql versions 
greater than 8.0
Made changes to allow non-ssl connection with DB for Mysql version greater than 
8.0
made a fix to allow user to define the custom jdbc url which can be used in 
db-setup.
Added missing change for Ranger Admin db-setup in RANGER-3647


Diffs
-

  kms/scripts/db_setup.py 165e30d89443b7e8244ed965c34a5d7219e7d1f3 
  kms/scripts/install.properties 780509dcdd06c13e84f1a860213eb28f3556fa26 
  security-admin/scripts/db_setup.py eaae5c8990724d7ead703d747140a0c3c49289d7 


Diff: https://reviews.apache.org/r/73935/diff/1/


Testing
---

Validated changes locally with available Mysql-8.0 release.


Thanks,

Vishal Suvagia

[jira] [Updated] (RANGER-3669) Connection to DB fails for MySQL version above 8.0

2022-04-07 Thread Vishal Suvagia (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vishal Suvagia updated RANGER-3669:
---
Attachment: RANGER-3669.patch

> Connection to DB fails for MySQL version above 8.0
> --
>
> Key: RANGER-3669
> URL: https://issues.apache.org/jira/browse/RANGER-3669
> Project: Ranger
>  Issue Type: Improvement
>  Components: kms
>Affects Versions: 3.0.0
>Reporter: Vishal Suvagia
>Assignee: Vishal Suvagia
>Priority: Major
> Attachments: RANGER-3669.patch
>
>
> Observed that Ranger KMS DB setup fails when using with MySQL version above 
> 8.0.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[GitHub] [ranger] Wrekkers closed pull request #144: DP-1756: adding add users feature to python client

2022-04-07 Thread GitBox 


Wrekkers closed pull request #144: DP-1756: adding add users feature to python 
client
URL: https://github.com/apache/ranger/pull/144


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Review Request 73934: RANGER-3699 : Upgrade poi to 5.2.2

2022-04-07 Thread Mateen Mansoori 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73934/
---

Review request for ranger, bhavik patel, Mehul Parikh, Pradeep Agrawal, and 
Velmurugan Periasamy.


Bugs: RANGER-3699
https://issues.apache.org/jira/browse/RANGER-3699


Repository: ranger


Description
---

Ranger currently pulling poi 5.1.0, Upgraded to 5.2.2


Diffs
-

  pom.xml 9889685c9 


Diff: https://reviews.apache.org/r/73934/diff/1/


Testing
---

- Build succeeded
- Tested on local VM : policy import - export funtionality.


Thanks,

Mateen Mansoori

Re: Review Request 73933: RANGER-3702 : Export policy in excel is failing

2022-04-07 Thread bhavik patel 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73933/#review224263
---


Ship it!




Ship It!

- bhavik patel


On April 7, 2022, 8:43 a.m., Mateen Mansoori wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73933/
> ---
> 
> (Updated April 7, 2022, 8:43 a.m.)
> 
> 
> Review request for ranger, bhavik patel, Madhan Neethiraj, Mehul Parikh, and 
> Pradeep Agrawal.
> 
> 
> Bugs: RANGER-3702
> https://issues.apache.org/jira/browse/RANGER-3702
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Due to missing dependency on classpath - Export policy in excel is failing 
> with the below error : 
> 
> java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
> at org.apache.poi.POIDocument.(POIDocument.java:65)
> 
> 
> Diffs
> -
> 
>   security-admin/pom.xml a2060e1c2 
> 
> 
> Diff: https://reviews.apache.org/r/73933/diff/1/
> 
> 
> Testing
> ---
> 
> - Build succeeded 
> - Test on local VM - policy export in excel,csv and json all are working fine.
> 
> 
> Thanks,
> 
> Mateen Mansoori
> 
>

[jira] [Commented] (RANGER-3702) RANGER - Export policy in excel is failing.

2022-04-07 Thread Mateen N Mansoori (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518697#comment-17518697
 ] 

Mateen N Mansoori commented on RANGER-3702:
---

Review board : https://reviews.apache.org/r/73933/

> RANGER - Export policy in excel is failing.
> ---
>
> Key: RANGER-3702
> URL: https://issues.apache.org/jira/browse/RANGER-3702
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Mateen N Mansoori
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Export policy in excel is failing with the below error : 
>  
> {code:java}
> java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
>   at org.apache.poi.POIDocument.(POIDocument.java:65)
>   at 
> org.apache.ranger.biz.ServiceDBStore.writeExcel(ServiceDBStore.java:3991)
>   at 
> org.apache.ranger.biz.ServiceDBStore.getPoliciesInExcel(ServiceDBStore.java:2428)
>   at 
> org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(ServiceREST.java:1985) 
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73933: RANGER-3702 : Export policy in excel is failing

2022-04-07 Thread Mateen Mansoori 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73933/
---

Review request for ranger, bhavik patel, Madhan Neethiraj, Mehul Parikh, and 
Pradeep Agrawal.


Bugs: RANGER-3702
https://issues.apache.org/jira/browse/RANGER-3702


Repository: ranger


Description
---

Due to missing dependency on classpath - Export policy in excel is failing with 
the below error : 

java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
at org.apache.poi.POIDocument.(POIDocument.java:65)


Diffs
-

  security-admin/pom.xml a2060e1c2 


Diff: https://reviews.apache.org/r/73933/diff/1/


Testing
---

- Build succeeded 
- Test on local VM - policy export in excel,csv and json all are working fine.


Thanks,

Mateen Mansoori

[jira] [Commented] (RANGER-3691) Upgrade spring to 5.3.18 CVE-2022-22965

2022-04-07 Thread Christian Pfarr (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518678#comment-17518678
 ] 

Christian Pfarr commented on RANGER-3691:
-

yes please... i´ve done this as well with 2.2.0 after the build and everything 
worked well.

> Upgrade spring to 5.3.18 CVE-2022-22965
> ---
>
> Key: RANGER-3691
> URL: https://issues.apache.org/jira/browse/RANGER-3691
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, kms
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Blocker
> Fix For: 3.0.0
>
>
> [https://tanzu.vmware.com/security/cve-2022-22965|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-22965]
> [https://github.com/spring-projects/spring-framework/releases]
>  
> Spring has a new 0day Remote-Code-Execution problem, related to spring-beans 
> and JDK9+
> Fixed at spring 5.3.18 / 5.2.20
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3619) REST API should return 403 when authenticated client is not allowed to access API.

2022-04-07 Thread kirby zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518659#comment-17518659
 ] 

kirby zhou commented on RANGER-3619:


I have not verified with HA, but the problem seem not related to HA.

> REST API should return 403 when authenticated client is not allowed to access 
> API.
> --
>
> Key: RANGER-3619
> URL: https://issues.apache.org/jira/browse/RANGER-3619
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.2.0
>Reporter: kirby zhou
>Priority: Major
>
> REST API should return 403-Forbidden when authenticated client is not allowed 
> to access API to avoid crash Ranger Clients.
>  
> Now, some API returns 401-Unauthorized instead of 403-Forbidden when client 
> is already passed authentication but now allowed to do something.
> In general, this will not cause any serious problems. However, there is a 
> flaw in the SPNEGO protocol implementation of Java HTTPUrlConnection. It 
> causes the Client to throw an unexpected exception. This will trouble the 
> operators and developers.
>  
> Let me show you how it happens:
>  
> For example:
>  
> The RangerAdminClient inside KMS  want to access API 
> "/service/secure/policies/download", but the principal is not in the 
> allowlist.
>  
>  # RangerAdminClient is based on Jersey-Client
>  # JerseyClient sends a HTTP-request to Ranger Service without authentication 
> information
>  # Tomcat/Spring inside Ranger returns 401 with HTTP header 
> “WWW-Authentication: Neogotiate”
>  # JerseyClient sends request again with Kerberos/SPNEGO authentication 
> tokens.
>  # Tomcat/Spring inside Ranger accept the authentication, then call 
> ServiceRest::getSecureServicePoliciesIfUpdated to reply the API calling.
>  # ServiceRest::getSecureServicePoliciesIfUpdated checks allowlist of “kms 
> service”, and refuse client with 401.
>  # Tomcat/Spring inside Ranger returns 401 with HTTP header 
> “WWW-Authentication: Neogotiate….” for notifying RangerAdminClient the 
> authentication is passed.
>  
> Now, there is a malformed state. HTTP-status code told client authenticate is 
> failed, but HTTP header told client authentication is passed.
>  
> In the RangerAdminClient side, 
>  
>  # sun.net.www.protocol.http.HttpURLConnection.getInputStream0() see the 
> second 401.
>  # 'inNegotiate' = true, so it is in the progress of _Negotiate._
>  # It checks that: if "WWW-Authenticate: Negotiate" exist then disable 
> negotiate for following code to avoid try {_}Negotiate once again{_}.
>  # But "WWW-Authenticate: Negotiate xczsd324…" does not the rule above.
>  # So HttpURLConnection calls AuthenticationInfo.sendHeaders to generate a 
> new request header.
>  # Wow, Null exception happens.
>  # Logs "ERROR RangerAdminRESTClient - Error getting policies; Received NULL 
> response!!. secureMode=true, user=… (auth:KERBEROS), serviceName=kmsdev"
>  # Log of KMS: "ERROR RangerAdminRESTClient - Failed to get response, Error 
> is : java.lang.RuntimeException: java.lang.NullPointerException"
>  
> This log makes admin confused.
>  
>  
> {code:java}
> //ServiceRest::getServicePoliciesIfUpdated
> if (isAllowed) {
> //...
> } else {
>httpCode = HttpServletResponse.SC_UNAUTHORIZED;
> }
>  {code}
> {code:java}
> // sun.net.www.protocol.http.HttpURLConnection.getInputStream0()
> // Read comments labeled "Failed Negotiate" for details.
> boolean dontUseNegotiate = false;
> Iterator iter = responses.multiValueIterator("WWW-Authenticate");
> while (iter.hasNext()) {
> String value = iter.next().trim();
> if (value.equalsIgnoreCase("Negotiate") ||
> value.equalsIgnoreCase("Kerberos")) {
> if (!inNegotiate) {
> inNegotiate = true;
> } else {
> dontUseNegotiate = true;
> doingNTLM2ndStage = false;
> serverAuthentication = null;
> }
> break;
> }
> }
> /**
>  * Failed Negotiate
>  *
>  * In some cases, the Negotiate auth is supported for the
>  * remote host but the negotiate process still fails (For
>  * example, if the web page is located on a backend server
>  * and delegation is needed but fails). The authentication
>  * process will start again, and we need to detect this
>  * kind of failure and do proper fallback (say, to NTLM).
>  *
>  * In order to achieve this, the inNegotiate flag is set
>  * when the first negotiate challenge is met (and reset
>  * if authentication is finished). If a fresh new negotiate
>  * challenge (no parameter) is found while inNegotiate is
>  * set, we know there's a failed auth attempt recently.
>  * Here we'll ignore the header line so that fallback
>  * can be practiced.
>  *
>  * inNegotiateProxy is for proxy authentication.
>  */
>   {code}
>  

[GitHub] [ranger] akatona84 commented on pull request #133: RANGER-3231: Ranger-Kafka-Plugin implementing Kafka Authorizer from KIP-504

2022-04-07 Thread GitBox 


akatona84 commented on PR #133:
URL: https://github.com/apache/ranger/pull/133#issuecomment-1091189356

   merged to master


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [ranger] akatona84 closed pull request #133: RANGER-3231: Ranger-Kafka-Plugin implementing Kafka Authorizer from KIP-504

2022-04-07 Thread GitBox 


akatona84 closed pull request #133: RANGER-3231: Ranger-Kafka-Plugin 
implementing Kafka Authorizer from KIP-504
URL: https://github.com/apache/ranger/pull/133


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (RANGER-3702) RANGER - Export policy in excel is failing.

2022-04-07 Thread Mateen N Mansoori (Jira) 
Mateen N Mansoori created RANGER-3702:
-

 Summary: RANGER - Export policy in excel is failing.
 Key: RANGER-3702
 URL: https://issues.apache.org/jira/browse/RANGER-3702
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Affects Versions: 3.0.0, 2.3.0
Reporter: Mateen N Mansoori
 Fix For: 3.0.0, 2.3.0


Export policy in excel is failing with the below error : 

 
{code:java}
java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
at org.apache.poi.POIDocument.(POIDocument.java:65)
at 
org.apache.ranger.biz.ServiceDBStore.writeExcel(ServiceDBStore.java:3991)
at 
org.apache.ranger.biz.ServiceDBStore.getPoliciesInExcel(ServiceDBStore.java:2428)
at 
org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(ServiceREST.java:1985) 
{code}
 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3698) Ranger - Upgrade kylin to 3.1.3

2022-04-07 Thread Mateen N Mansoori (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518632#comment-17518632
 ] 

Mateen N Mansoori commented on RANGER-3698:
---

Review Board - https://reviews.apache.org/r/73932/

> Ranger - Upgrade kylin to 3.1.3
> ---
>
> Key: RANGER-3698
> URL: https://issues.apache.org/jira/browse/RANGER-3698
> Project: Ranger
>  Issue Type: Task
>  Components: Ranger
>Reporter: Mateen N Mansoori
>Priority: Major
>
> Ranger is currently pulling in kylin 2.6.6, This task is to track kylin 
> verison upgrade to 3.1.3



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73932: RANGER-3698 : Upgrade kylin to 3.1.3

2022-04-07 Thread Mateen Mansoori 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73932/
---

Review request for ranger and Pradeep Agrawal.


Bugs: RANGER-3698
https://issues.apache.org/jira/browse/RANGER-3698


Repository: ranger


Description
---

Ranger is currently pulling in kylin 2.6.6, This task is to track kylin verison 
upgrade to 3.1.3


Diffs
-

  plugin-kylin/pom.xml 563e53a87 
  pom.xml 9889685c9 
  ranger-kylin-plugin-shim/pom.xml 53e567467 


Diff: https://reviews.apache.org/r/73932/diff/1/


Testing
---

- Build Succeeded
- All Unit tests are passed.


Thanks,

Mateen Mansoori

[jira] [Commented] (RANGER-3569) Support Ranger KMS integration with Google cloud HSM

2022-04-07 Thread Mateen N Mansoori (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518618#comment-17518618
 ] 

Mateen N Mansoori commented on RANGER-3569:
---

ranger-2.3 : 
[https://github.com/apache/ranger/commit/1803188a72788870f5d8b3dbf2766a60e9d5dd5b]

 

> Support Ranger KMS integration with Google cloud HSM
> 
>
> Key: RANGER-3569
> URL: https://issues.apache.org/jira/browse/RANGER-3569
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms
>Reporter: Mateen N Mansoori
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Task for tracking Ranger KMS integration with google cloud HSM



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3619) REST API should return 403 when authenticated client is not allowed to access API.

2022-04-07 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17518608#comment-17518608
 ] 

Bhavik Patel commented on RANGER-3619:
--

[~kirbyzhou] have you verified the HA mode?

> REST API should return 403 when authenticated client is not allowed to access 
> API.
> --
>
> Key: RANGER-3619
> URL: https://issues.apache.org/jira/browse/RANGER-3619
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 3.0.0, 2.2.0
>Reporter: kirby zhou
>Priority: Major
>
> REST API should return 403-Forbidden when authenticated client is not allowed 
> to access API to avoid crash Ranger Clients.
>  
> Now, some API returns 401-Unauthorized instead of 403-Forbidden when client 
> is already passed authentication but now allowed to do something.
> In general, this will not cause any serious problems. However, there is a 
> flaw in the SPNEGO protocol implementation of Java HTTPUrlConnection. It 
> causes the Client to throw an unexpected exception. This will trouble the 
> operators and developers.
>  
> Let me show you how it happens:
>  
> For example:
>  
> The RangerAdminClient inside KMS  want to access API 
> "/service/secure/policies/download", but the principal is not in the 
> allowlist.
>  
>  # RangerAdminClient is based on Jersey-Client
>  # JerseyClient sends a HTTP-request to Ranger Service without authentication 
> information
>  # Tomcat/Spring inside Ranger returns 401 with HTTP header 
> “WWW-Authentication: Neogotiate”
>  # JerseyClient sends request again with Kerberos/SPNEGO authentication 
> tokens.
>  # Tomcat/Spring inside Ranger accept the authentication, then call 
> ServiceRest::getSecureServicePoliciesIfUpdated to reply the API calling.
>  # ServiceRest::getSecureServicePoliciesIfUpdated checks allowlist of “kms 
> service”, and refuse client with 401.
>  # Tomcat/Spring inside Ranger returns 401 with HTTP header 
> “WWW-Authentication: Neogotiate….” for notifying RangerAdminClient the 
> authentication is passed.
>  
> Now, there is a malformed state. HTTP-status code told client authenticate is 
> failed, but HTTP header told client authentication is passed.
>  
> In the RangerAdminClient side, 
>  
>  # sun.net.www.protocol.http.HttpURLConnection.getInputStream0() see the 
> second 401.
>  # 'inNegotiate' = true, so it is in the progress of _Negotiate._
>  # It checks that: if "WWW-Authenticate: Negotiate" exist then disable 
> negotiate for following code to avoid try {_}Negotiate once again{_}.
>  # But "WWW-Authenticate: Negotiate xczsd324…" does not the rule above.
>  # So HttpURLConnection calls AuthenticationInfo.sendHeaders to generate a 
> new request header.
>  # Wow, Null exception happens.
>  # Logs "ERROR RangerAdminRESTClient - Error getting policies; Received NULL 
> response!!. secureMode=true, user=… (auth:KERBEROS), serviceName=kmsdev"
>  # Log of KMS: "ERROR RangerAdminRESTClient - Failed to get response, Error 
> is : java.lang.RuntimeException: java.lang.NullPointerException"
>  
> This log makes admin confused.
>  
>  
> {code:java}
> //ServiceRest::getServicePoliciesIfUpdated
> if (isAllowed) {
> //...
> } else {
>httpCode = HttpServletResponse.SC_UNAUTHORIZED;
> }
>  {code}
> {code:java}
> // sun.net.www.protocol.http.HttpURLConnection.getInputStream0()
> // Read comments labeled "Failed Negotiate" for details.
> boolean dontUseNegotiate = false;
> Iterator iter = responses.multiValueIterator("WWW-Authenticate");
> while (iter.hasNext()) {
> String value = iter.next().trim();
> if (value.equalsIgnoreCase("Negotiate") ||
> value.equalsIgnoreCase("Kerberos")) {
> if (!inNegotiate) {
> inNegotiate = true;
> } else {
> dontUseNegotiate = true;
> doingNTLM2ndStage = false;
> serverAuthentication = null;
> }
> break;
> }
> }
> /**
>  * Failed Negotiate
>  *
>  * In some cases, the Negotiate auth is supported for the
>  * remote host but the negotiate process still fails (For
>  * example, if the web page is located on a backend server
>  * and delegation is needed but fails). The authentication
>  * process will start again, and we need to detect this
>  * kind of failure and do proper fallback (say, to NTLM).
>  *
>  * In order to achieve this, the inNegotiate flag is set
>  * when the first negotiate challenge is met (and reset
>  * if authentication is finished). If a fresh new negotiate
>  * challenge (no parameter) is found while inNegotiate is
>  * set, we know there's a failed auth attempt recently.
>  * Here we'll ignore the header line so that fallback
>  * can be practiced.
>  *
>  * inNegotiateProxy is for proxy authentication.
>  */
>   {code}
>  
>  
>  
>  



--

Re: Review Request 73881: RANGER-3619: REST API returns 403 when authed user has no permission.

2022-04-07 Thread bhavik patel 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73881/#review224262
---


Ship it!




Ship It!

- bhavik patel


On April 7, 2022, 6:21 a.m., Kirby Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73881/
> ---
> 
> (Updated April 7, 2022, 6:21 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-3619
> https://issues.apache.org/jira/browse/RANGER-3619
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> REST API should return 403-Forbidden when authenticated client is not allowed 
> to access API to avoid crash Ranger Clients.
> 
>  
> Now, some API returns 401-Unauthorized instead of 403-Forbidden when client 
> is already passed authentication but not allowed to do something.
> In general, this will not cause any serious problems, but there is a flaw in 
> the SPNEGO protocol implementation of Java HTTPUrlConnection. It causes the 
> Client to throw an unexpected exception. This will trouble the operators and 
> developers.
>  
> Let me show you how it happens:
>  
> For example:
>  
> The RangerAdminClient inside KMS  want to access API 
> "/service/secure/policies/download", but the principal is not in the 
> allowlist.
>  
> RangerAdminClient is based on Jersey-Client
> JerseyClient sends a HTTP-request to Ranger Service without authentication 
> information
> Tomcat/Spring inside Ranger returns 401 with HTTP header “WWW-Authentication: 
> Neogotiate”
> JerseyClient sends request again with Kerberos/SPNEGO authentication tokens.
> Tomcat/Spring inside Ranger accept the authentication, then call 
> ServiceRest::getSecureServicePoliciesIfUpdated to reply the API calling.
> ServiceRest::getSecureServicePoliciesIfUpdated checks allowlist of “kms 
> service”, and refuse client with 401.
> Tomcat/Spring inside Ranger returns 401 with HTTP header “WWW-Authentication: 
> Neogotiate….” for notifying RangerAdminClient the authentication is passed.
>  
> Now, there is a malformed state. HTTP-status code told client authenticate is 
> failed, but HTTP header told client authentication is passed.
>  
> In the RangerAdminClient side, 
>  
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0() see the second 
> 401.
> 'inNegotiate' = true, so it is in the progress of Negotiate.
> It checks that: if "WWW-Authenticate: Negotiate" exist then disable negotiate 
> for following code to avoid try Negotiate once again.
> But "WWW-Authenticate: Negotiate xczsd324…" does not the rule above.
> So HttpURLConnection calls AuthenticationInfo.sendHeaders to generate a new 
> request header.
> Wow, Null exception happens.
> 
> Logs "ERROR RangerAdminRESTClient - Error getting policies; Received NULL 
> response!!. secureMode=true, user=… (auth:KERBEROS), serviceName=kmsdev"
> 
> Log of KMS: "ERROR RangerAdminRESTClient - Failed to get response, Error is : 
> java.lang.RuntimeException: java.lang.NullPointerException"
>  
> 
> This log makes admin confused, and no not know how to fix it.
> 
> My patch fixes the return code of http, thus avoiding these problems.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
> 1ec1df0a3d09577c52e503532d5aea87ad6cd72d 
>   security-admin/src/main/java/org/apache/ranger/biz/XAuditMgr.java 
> 935435044624a38ce7b0b9c7401e3f3dbacc0f65 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 
> 8109968e4d55de9e7875fb56590e50522fba32cb 
>   security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java 
> e3cdef1c2ba6411cf4d4a26cd49e56e9017f3e93 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> 098188e3b9f1f837727c7d279a4fab1f0aa84e34 
>   security-admin/src/main/java/org/apache/ranger/rest/TagREST.java 
> 10f91e037180a50287b8d0b0fa0ea3eec0d7f415 
>   security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 
> 451805321d050dda06a0f2b66a9b945411632e2f 
>   
> security-admin/src/main/java/org/apache/ranger/security/context/RangerPreAuthSecurityHandler.java
>  5d7cbdc679c010a7b88c85324e6f9912cba29fe6 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKrbFilter.java
>  223a991c76bae7d25f5ce89604d0a8a90d426fe5 
> 
> 
> Diff: https://reviews.apache.org/r/73881/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kirby Zhou
> 
>

Re: Review Request 73881: RANGER-3619: REST API returns 403 when authed user has no permission.

2022-04-07 Thread Kirby Zhou 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73881/
---

(Updated 四月 7, 2022, 6:21 a.m.)


Review request for ranger.


Bugs: RANGER-3619
https://issues.apache.org/jira/browse/RANGER-3619


Repository: ranger


Description (updated)
---

REST API should return 403-Forbidden when authenticated client is not allowed 
to access API to avoid crash Ranger Clients.

 
Now, some API returns 401-Unauthorized instead of 403-Forbidden when client is 
already passed authentication but not allowed to do something.
In general, this will not cause any serious problems, but there is a flaw in 
the SPNEGO protocol implementation of Java HTTPUrlConnection. It causes the 
Client to throw an unexpected exception. This will trouble the operators and 
developers.
 
Let me show you how it happens:
 
For example:
 
The RangerAdminClient inside KMS  want to access API 
"/service/secure/policies/download", but the principal is not in the allowlist.
 
RangerAdminClient is based on Jersey-Client
JerseyClient sends a HTTP-request to Ranger Service without authentication 
information
Tomcat/Spring inside Ranger returns 401 with HTTP header “WWW-Authentication: 
Neogotiate”
JerseyClient sends request again with Kerberos/SPNEGO authentication tokens.
Tomcat/Spring inside Ranger accept the authentication, then call 
ServiceRest::getSecureServicePoliciesIfUpdated to reply the API calling.
ServiceRest::getSecureServicePoliciesIfUpdated checks allowlist of “kms 
service”, and refuse client with 401.
Tomcat/Spring inside Ranger returns 401 with HTTP header “WWW-Authentication: 
Neogotiate….” for notifying RangerAdminClient the authentication is passed.
 
Now, there is a malformed state. HTTP-status code told client authenticate is 
failed, but HTTP header told client authentication is passed.
 
In the RangerAdminClient side, 
 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0() see the second 
401.
'inNegotiate' = true, so it is in the progress of Negotiate.
It checks that: if "WWW-Authenticate: Negotiate" exist then disable negotiate 
for following code to avoid try Negotiate once again.
But "WWW-Authenticate: Negotiate xczsd324…" does not the rule above.
So HttpURLConnection calls AuthenticationInfo.sendHeaders to generate a new 
request header.
Wow, Null exception happens.

Logs "ERROR RangerAdminRESTClient - Error getting policies; Received NULL 
response!!. secureMode=true, user=… (auth:KERBEROS), serviceName=kmsdev"

Log of KMS: "ERROR RangerAdminRESTClient - Failed to get response, Error is : 
java.lang.RuntimeException: java.lang.NullPointerException"
 

This log makes admin confused, and no not know how to fix it.

My patch fixes the return code of http, thus avoiding these problems.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
1ec1df0a3d09577c52e503532d5aea87ad6cd72d 
  security-admin/src/main/java/org/apache/ranger/biz/XAuditMgr.java 
935435044624a38ce7b0b9c7401e3f3dbacc0f65 
  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 
8109968e4d55de9e7875fb56590e50522fba32cb 
  security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java 
e3cdef1c2ba6411cf4d4a26cd49e56e9017f3e93 
  security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
098188e3b9f1f837727c7d279a4fab1f0aa84e34 
  security-admin/src/main/java/org/apache/ranger/rest/TagREST.java 
10f91e037180a50287b8d0b0fa0ea3eec0d7f415 
  security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 
451805321d050dda06a0f2b66a9b945411632e2f 
  
security-admin/src/main/java/org/apache/ranger/security/context/RangerPreAuthSecurityHandler.java
 5d7cbdc679c010a7b88c85324e6f9912cba29fe6 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKrbFilter.java
 223a991c76bae7d25f5ce89604d0a8a90d426fe5 


Diff: https://reviews.apache.org/r/73881/diff/1/


Testing
---


Thanks,

Kirby Zhou

[jira] [Created] (RANGER-3701) Establish plug-in system for KMS MasterKeyProvider

2022-04-07 Thread kirby zhou (Jira) 
kirby zhou created RANGER-3701:
--

 Summary: Establish plug-in system for KMS MasterKeyProvider
 Key: RANGER-3701
 URL: https://issues.apache.org/jira/browse/RANGER-3701
 Project: Ranger
  Issue Type: Improvement
  Components: kms
Affects Versions: 3.0.0, 2.3.0
Reporter: kirby zhou


At present, RangerKMS has six different MasterKey Provider. Among them, three 
types can access MK, and KMS can complete the encryption and decryption of 
ZoneKey by itself, and three types can only entrust the encryption and 
decryption of ZoneKey to MasterKey Provider. 

Except the built-in JDBC-based RangerMasterKey class, other provider have more 
or less introduced a large number of dependencies. This makes the dependence of 
KMS quite complicated and confusing. In the future, these dependencies may 
conflict. Therefore, it is necessary to refine MasterKey Provider into a 
plug-in mechanism, similar to plugin shim of Ranger Admin.

 

A preliminary idea,  we can define a MKProviderFactory interface which can 
create instance of RangerKMSMKI from a URL. Then we use 
ServiceLoader to create MK Provider at runtime.  The 
dependencies of actual MK Provider is hidden by plugin class loader.

 

URL schema can like "mkp-azure://conffile/keyprefix", 
"mkp-jdbc://connectionstring", ...

 

At last we can unify the way of key import / export / migration CLI utilities.

 

Task Blocked on: https://issues.apache.org/jira/browse/RANGER-3682

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)