Re: Review Request 73807: Support Ranger KMS integration with TencentKMS

2022-02-15 Thread Kirby Zhou 


> On 二月 15, 2022, 12:15 p.m., Dhaval Shah wrote:
> > Hi Kirby Zhou,
> > 
> > Facing PMD Violation issue.
> > 
> > [INFO] PMD version: 6.29.0
> > [INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:20 
> > Rule:UnusedImports Priority:4 Avoid unused imports such as 
> > 'com.microsoft.azure.keyvault.KeyVaultClient'.
> > [INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:45 
> > Rule:UnusedImports Priority:4 Avoid unused imports such as 
> > 'org.apache.commons.lang.StringUtils'.
> > [INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:55 
> > Rule:UnusedImports Priority:4 Avoid unused imports such as 
> > 'com.tencentcloudapi.kms.v20190118.KmsClient'.
> > [INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:80 
> > Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
> > 'AZURE_KEYVAULT_SSL_ENABLED'..
> > [INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:81 
> > Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
> > 'AZURE_CLIENT_ID'..
> > [INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:84 
> > Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
> > 'AZURE_KEYVAULT_CERTIFICATE_PATH'..
> > [INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:85 
> > Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
> > 'AZURE_KEYVAULT_CERTIFICATE_PASSWORD'..
> > 
> > 
> > Request ypu please upload the patch after successful build using command 
> > "mvn clean install"

% mvn clean install
...
[INFO] Reactor Summary for ranger 3.0.0-SNAPSHOT:
[INFO] 
[INFO] ranger . SUCCESS [  5.724 s]
...

[INFO] Apache Ranger Distribution . SUCCESS [01:07 min]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time:  16:19 min
[INFO] Finished at: 2022-02-16T15:44:57+08:00
[INFO] 

is it OK?


- Kirby


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/#review224050
---


On 二月 15, 2022, 8:48 a.m., Kirby Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73807/
> ---
> 
> (Updated 二月 15, 2022, 8:48 a.m.)
> 
> 
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, 
> Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Mehul 
> Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3580
> https://issues.apache.org/jira/browse/RANGER-3580
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger KMS integration with TencentKMS
> - This task is to integrate the RANGER KMS Service with TencentKMS.
> - To Configure RANGER KMS Service with TencentKMS below configurations need 
> to be added in install.properties file bfore running the setup.sh
> 
> ```
> # Do you use Tencent Cloud KMS? 
> TENCENT_KMS_ENABLED=true 
> # MasterKeyID on Tencent Cloud
> TENCENT_MASTERKEY_ID=YourKeyID
> # Login ID
> TENCENT_CLIENT_ID=YourClientLoginId
> # Login password
> TENCENT_CLIENT_SECRET=YourClientLoginSecret
> # Tencent Cloud area, see Tencent Cloud SDK for details. 
> TENCENT_CLIENT_REGION=ap-beijing
> ```
> 
> Run the setup.sh, It will add the below configs in dbks-site.xml
> ```
> 
> 
> ranger.kms.tencentkms.enabled
> false
> Flag for Tencent KMS
> 
> 
> ranger.kms.tencent.client.id
> 
> Tencent Client Id
> 
> 
> ranger.kms.tencent.client.secret
> 
> Tencent Client Secret
> 
> 
> ranger.kms.tencent.client.secret.alias
> ranger.ks.tencent.client.secret
> Tencent Client Secret Alias
> 
> 
> ranger.kms.tencent.client.region
> ap-beijing
> Tencent Client Id
> 
> 
> ranger.kms.tencent.masterkey.id
> 
> Tencent master key name
> 
> 
> ```
> 
> Generally, we don't want the account bound by KMS to have the right to create 
> a Key in TencentKMS. So we have to create Master Key on TencentKMS web 
> console at first.
> Start the kms service, On start Master Key from TencentKMS should be used.
> 
> 
> Diffs
> -
> 
>   distro/src/main/assembly/kms.xml 983a43e5938ecc6a02e918f587d7a8913678087e 
>   kms/config/kms-webapp/dbks-site.xml 
> 07de4d494b5d72609b47752109fc40a9e016f6ab 
>   kms/pom.xml

[jira] [Commented] (RANGER-3624) Update Ranger services Password Policy

2022-02-15 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493010#comment-17493010
 ] 

Bhavik Patel commented on RANGER-3624:
--

[~pradeepagrawal8184] [~dhavalshah9131] [~madhan] [~rmani] can you guys please 
review..

> Update Ranger services Password Policy
> --
>
> Key: RANGER-3624
> URL: https://issues.apache.org/jira/browse/RANGER-3624
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin
>Affects Versions: 3.0.0
>Reporter: Bhavik Patel
>Assignee: Bhavik Patel
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 
> 0001-RANGER-3624-Update-Ranger-services-Password-Policy.patch
>
>
> Current Password policies(validation) is not strong enough as it expect the 
> {*}"minimum 8 characters with minimum one alphabet and one numeric"{*}.
> In this improvement Jira will enhance the password policies to *"minimum 8 
> characters, at least one uppercase letter, one lowercase letter and one 
> numeric"*



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3623) Add ability to enable anonymous download of policy/role/tag

2022-02-15 Thread kirby zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492928#comment-17492928
 ] 

kirby zhou commented on RANGER-3623:


No,  I want to pull policies from ranger in third-party services without any 
authentication.

At present, if I enable anonymous/unauthenticated pulling/downloading, the 
unauthenticated policy modification will also be enabled. 

The ability I want to add is to allow anonymous downloads independently.

 

> Add ability to enable anonymous download of policy/role/tag
> ---
>
> Key: RANGER-3623
> URL: https://issues.apache.org/jira/browse/RANGER-3623
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Priority: Major
> Attachments: add-downloadonly-option.patch
>
>
> Currently, we have an option ranger.admin.allow.unauthenticated.access to 
> allow unauthenticated clients to perform a series of API operations. This 
> option allows the client to perform both dangerous grant/revoke permission 
> operation and relatively safe download operation.
> In many cases, allowing anonymous downloading of policy is not a serious risk 
> problem. On the contrary, the complicated kerberos and SSL settings make it 
> difficult for ranger plugin embedded in third-party services to complete the 
> task of refreshing policy, which may be a bigger problem. In particular, 
> refresh failure often has no obvious features for administrators to discover.
> Therefore, I suggest that ranger increase the ability to allow client to 
> download policy/tag/roles anonymously.
> There are two ways to achieve it.
>  
> 1. Just limit the ability of  "ranger.admin.allow.unauthenticated.access=true"
> which needs to modify 
> "security-admin/src/main/resources/conf.dist/security-applicationContext.xml" 
> to remove dangerous operations from '
> security="none"'.
>  
> 2. Add a candidate value "downloadonly" to 
> "ranger.admin.allow.unauthenticated.access"
> Which needs modify ServiceRest.Java and BizUtil.java to implement the 
> enhanced checking logic. 
>  
> I have a patch for method2



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Commented] (RANGER-3627) Implement Spark extension

2022-02-15 Thread kirby zhou (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492924#comment-17492924
 ] 

kirby zhou commented on RANGER-3627:


Compared with reusing Hive plugin [Hadoop SQL], what is your highlight of a new 
Spark extension?

> Implement Spark extension
> -
>
> Key: RANGER-3627
> URL: https://issues.apache.org/jira/browse/RANGER-3627
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Jakub Leś
>Priority: Major
>
> Hi 
> I would like to implement Ranger Spark SQL. Is this ok ?
>  
> Best regards,
> Jakub
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Review Request 73848: RANGER-3621 Optimise Tag iterator

2022-02-15 Thread Abhay Kulkarni 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73848/#review224055
---


Fix it, then Ship it!





security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java
Lines 272 (patched)


Please consider similar optimization when building a service-policies 
cache, if applicable.


- Abhay Kulkarni


On Feb. 15, 2022, 12:57 p.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73848/
> ---
> 
> (Updated Feb. 15, 2022, 12:57 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.
> 
> 
> Bugs: RANGER-3621
> https://issues.apache.org/jira/browse/RANGER-3621
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Optimise the logic of Tag iterator
> 
> Problem Statement:
> Tags in the iterator are not freed till the whole processing of all Tags are 
> done. So, in the case where there are lot of tag, the performance can be 
> impacted as the large number of tags are held by iterator for long time. This 
> causes performance degradation while retrieving the tags from db.
> 
> Solution:
> The individual tag as soon as it is processed should be removed from the 
> iterator.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java 
> 1b7e8b272 
> 
> 
> Diff: https://reviews.apache.org/r/73848/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Manually by using bulk tags.
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

Re: Review Request 73848: RANGER-3621 Optimise Tag iterator

2022-02-15 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73848/#review224054
---


Ship it!




Ship It!

- Madhan Neethiraj


On Feb. 15, 2022, 12:57 p.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73848/
> ---
> 
> (Updated Feb. 15, 2022, 12:57 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.
> 
> 
> Bugs: RANGER-3621
> https://issues.apache.org/jira/browse/RANGER-3621
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Optimise the logic of Tag iterator
> 
> Problem Statement:
> Tags in the iterator are not freed till the whole processing of all Tags are 
> done. So, in the case where there are lot of tag, the performance can be 
> impacted as the large number of tags are held by iterator for long time. This 
> causes performance degradation while retrieving the tags from db.
> 
> Solution:
> The individual tag as soon as it is processed should be removed from the 
> iterator.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java 
> 1b7e8b272 
> 
> 
> Diff: https://reviews.apache.org/r/73848/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Manually by using bulk tags.
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

Re: Review Request 73849: RANGER-3625 Update isDebugEnable condition in RangerHiveAuthorizer

2022-02-15 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73849/#review224053
---


Ship it!




Ship It!

- Madhan Neethiraj


On Feb. 15, 2022, 1:19 p.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73849/
> ---
> 
> (Updated Feb. 15, 2022, 1:19 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.
> 
> 
> Bugs: RANGER-3625
> https://issues.apache.org/jira/browse/RANGER-3625
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The condition "if (!LOG.isDebugEnabled())" should be replaced by "if 
> (LOG.isDebugEnabled())" 
> so that the debug.level log inside this condition is logged.
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  e6cb50796 
> 
> 
> Diff: https://reviews.apache.org/r/73849/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Manually and checked the debug log getting logged if the Debug is 
> enable.
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

[jira] [Created] (RANGER-3627) Implement Spark extension

2022-02-15 Thread Jira 
Jakub Leś created RANGER-3627:
-

 Summary: Implement Spark extension
 Key: RANGER-3627
 URL: https://issues.apache.org/jira/browse/RANGER-3627
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Jakub Leś


Hi 

I would like to implement Ranger Spark SQL. Is this ok ?

 

Best regards,

Jakub

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (RANGER-3626) Class cast exception org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory cannot be cast to org.apache.hadoop.hive.ql.security.authorization.plugi

2022-02-15 Thread Jira 
Michał Wieleba created RANGER-3626:
--

 Summary: Class cast exception 
org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory 
cannot be cast to 
org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerFactory
 Key: RANGER-3626
 URL: https://issues.apache.org/jira/browse/RANGER-3626
 Project: Ranger
  Issue Type: Bug
  Components: plugins
Affects Versions: 2.2.0
Reporter: Michał Wieleba


HiveServer2 fails when ranger hive plugin is enabled.

Apache Ranger 2.2.0

Apache Hive 3.1.2

2022-02-15 16:09:02,990 ERROR authorizer.RangerHiveAuthorizerFactory: Error 
Enabling RangerHivePlugin
java.lang.ClassCastException: 
org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory 
cannot be cast to 
org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerFactory
        at 
org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory.init(RangerHiveAuthorizerFactory.java:72)
        at 
org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory.(RangerHiveAuthorizerFactory.java:51)
        at sun.reflect.GeneratedConstructorAccessor26.newInstance(Unknown 
Source)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:135)
        at 
org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthorizerFactory(HiveUtils.java:392)
        at 
org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:914)
        at 
org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1893)
        at 
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:131)
        at org.apache.hive.service.cli.CLIService.init(CLIService.java:115)
        at 
org.apache.hive.service.CompositeService.init(CompositeService.java:59)
        at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:230)
        at 
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1036)
        at 
org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140)
        at 
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1305)
        at 
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1149)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Review Request 73842: RANGER-3620 : Ranger - Upgrade tomcat to 8.5.75

2022-02-15 Thread bhavik patel 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73842/#review224052
---


Ship it!




Ship It!

- bhavik patel


On Feb. 14, 2022, 1:21 p.m., Mateen Mansoori wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73842/
> ---
> 
> (Updated Feb. 14, 2022, 1:21 p.m.)
> 
> 
> Review request for ranger, Dhaval Shah, Mehul Parikh, Pradeep Agrawal, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3620
> https://issues.apache.org/jira/browse/RANGER-3620
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Upgraded embedded-tomcat to 8.5.75
> 
> 
> Diffs
> -
> 
>   pom.xml 8a19c2de4 
> 
> 
> Diff: https://reviews.apache.org/r/73842/diff/1/
> 
> 
> Testing
> ---
> 
> - Build succeeded - mvn clean compile test verify install
> - Verified service, user CRUD on local VM.
> 
> 
> Thanks,
> 
> Mateen Mansoori
> 
>

[jira] [Commented] (RANGER-3623) Add ability to enable anonymous download of policy/role/tag

2022-02-15 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17492619#comment-17492619
 ] 

Bhavik Patel commented on RANGER-3623:
--

[~kirbyzhou] As you mentioned "ranger plugin embedded in third-party services 
to complete the task of refreshing policy" ==> You mean to say if we want to 
update the ranger default policies from third-party services then it create 
problem?  
 If it's true than that's the correct behaviour, because if allow them then 
anybody can update policies from there service and get the permissions for the 
unwanted resource.

> Add ability to enable anonymous download of policy/role/tag
> ---
>
> Key: RANGER-3623
> URL: https://issues.apache.org/jira/browse/RANGER-3623
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Priority: Major
> Attachments: add-downloadonly-option.patch
>
>
> Currently, we have an option ranger.admin.allow.unauthenticated.access to 
> allow unauthenticated clients to perform a series of API operations. This 
> option allows the client to perform both dangerous grant/revoke permission 
> operation and relatively safe download operation.
> In many cases, allowing anonymous downloading of policy is not a serious risk 
> problem. On the contrary, the complicated kerberos and SSL settings make it 
> difficult for ranger plugin embedded in third-party services to complete the 
> task of refreshing policy, which may be a bigger problem. In particular, 
> refresh failure often has no obvious features for administrators to discover.
> Therefore, I suggest that ranger increase the ability to allow client to 
> download policy/tag/roles anonymously.
> There are two ways to achieve it.
>  
> 1. Just limit the ability of  "ranger.admin.allow.unauthenticated.access=true"
> which needs to modify 
> "security-admin/src/main/resources/conf.dist/security-applicationContext.xml" 
> to remove dangerous operations from '
> security="none"'.
>  
> 2. Add a candidate value "downloadonly" to 
> "ranger.admin.allow.unauthenticated.access"
> Which needs modify ServiceRest.Java and BizUtil.java to implement the 
> enhanced checking logic. 
>  
> I have a patch for method2



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Review Request 73849: RANGER-3625 Update isDebugEnable condition in RangerHiveAuthorizer

2022-02-15 Thread bhavik patel 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73849/#review224051
---




hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
Line 1192 (original), 1192 (patched)


we can remove this condition as we are using slf4j logging. 

Based on the log level configuration, slf4j Itself evaluate logger 
statement. We just have update the debug statement format to placehold format


- bhavik patel


On Feb. 15, 2022, 1:19 p.m., Ankita Sinha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73849/
> ---
> 
> (Updated Feb. 15, 2022, 1:19 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.
> 
> 
> Bugs: RANGER-3625
> https://issues.apache.org/jira/browse/RANGER-3625
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The condition "if (!LOG.isDebugEnabled())" should be replaced by "if 
> (LOG.isDebugEnabled())" 
> so that the debug.level log inside this condition is logged.
> 
> 
> Diffs
> -
> 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  e6cb50796 
> 
> 
> Diff: https://reviews.apache.org/r/73849/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Manually and checked the debug log getting logged if the Debug is 
> enable.
> 
> 
> Thanks,
> 
> Ankita Sinha
> 
>

[jira] [Created] (RANGER-3625) Update isDebugEnable condition in RangerHiveAuthorizer

2022-02-15 Thread Ankita Sinha (Jira) 
Ankita Sinha created RANGER-3625:


 Summary: Update isDebugEnable condition in RangerHiveAuthorizer
 Key: RANGER-3625
 URL: https://issues.apache.org/jira/browse/RANGER-3625
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Affects Versions: 2.2.0
Reporter: Ankita Sinha
Assignee: Ankita Sinha


{+}_Problem Statement_{+}: 
In RangerHiveAuthorizer, the Logger isDebugEnabled condition needs to be updated


{code:java}
else if (!result.getIsAllowed()) {
if (!LOG.isDebugEnabled()) {
String path = 
resource.getAsString();

LOG.debug(String.format("filterListCmdObjects: Permission denied: user [%s] 
does not have [%s] privilege on [%s]. resource[%s], request[%s], result[%s]",
user, 
request.getHiveAccessType().name(), path, resource, request, result));
} {code}
here in above code the condition check is "!LOG.isDebugEnabled()" and log is 
logged at Debug level which will never get log.

{+}_Solution_{+}:
The condition "{*}if (!LOG.isDebugEnabled()){*}" should be replaced by *"if 
(LOG.isDebugEnabled())"*

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (RANGER-3625) Update isDebugEnable condition in RangerHiveAuthorizer

2022-02-15 Thread Ankita Sinha (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankita Sinha updated RANGER-3625:
-
Attachment: RANGER-3625-Update-isDebugEnable-condition-in-Ranger.patch

> Update isDebugEnable condition in RangerHiveAuthorizer
> --
>
> Key: RANGER-3625
> URL: https://issues.apache.org/jira/browse/RANGER-3625
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.2.0
>Reporter: Ankita Sinha
>Assignee: Ankita Sinha
>Priority: Trivial
> Attachments: 
> RANGER-3625-Update-isDebugEnable-condition-in-Ranger.patch
>
>
> {+}_Problem Statement_{+}: 
> In RangerHiveAuthorizer, the Logger isDebugEnabled condition needs to be 
> updated
> {code:java}
> else if (!result.getIsAllowed()) {
>   if (!LOG.isDebugEnabled()) {
>   String path = 
> resource.getAsString();
>   
> LOG.debug(String.format("filterListCmdObjects: Permission denied: user [%s] 
> does not have [%s] privilege on [%s]. resource[%s], request[%s], result[%s]",
>   user, 
> request.getHiveAccessType().name(), path, resource, request, result));
>   } {code}
> here in above code the condition check is "!LOG.isDebugEnabled()" and log is 
> logged at Debug level which will never get log.
> {+}_Solution_{+}:
> The condition "{*}if (!LOG.isDebugEnabled()){*}" should be replaced by *"if 
> (LOG.isDebugEnabled())"*
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73849: RANGER-3625 Update isDebugEnable condition in RangerHiveAuthorizer

2022-02-15 Thread Ankita Sinha 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73849/
---

Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.


Bugs: RANGER-3625
https://issues.apache.org/jira/browse/RANGER-3625


Repository: ranger


Description
---

The condition "if (!LOG.isDebugEnabled())" should be replaced by "if 
(LOG.isDebugEnabled())" 
so that the debug.level log inside this condition is logged.


Diffs
-

  
hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
 e6cb50796 


Diff: https://reviews.apache.org/r/73849/diff/1/


Testing
---

Tested Manually and checked the debug log getting logged if the Debug is enable.


Thanks,

Ankita Sinha

[jira] [Updated] (RANGER-3621) Optimise Tag iterator

2022-02-15 Thread Ankita Sinha (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankita Sinha updated RANGER-3621:
-
Attachment: RANGER-3621-Optimise-Tag-iterator.patch

> Optimise Tag iterator  
> ---
>
> Key: RANGER-3621
> URL: https://issues.apache.org/jira/browse/RANGER-3621
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 2.2.0
>Reporter: Ankita Sinha
>Assignee: Ankita Sinha
>Priority: Major
> Attachments: RANGER-3621-Optimise-Tag-iterator.patch
>
>
> Optimise the logic of Tag iterator
> {+}_Problem Statement_{+}:
> Tags in the iterator are not freed till the whole processing of all Tags are 
> done. So, in the case where there are lot of tag, the performance can be 
> impacted as the large number of tags are held by iterator for long time. This 
> causes performance degradation while retrieving the tags from db.
> {+}_Solution_{+}:
> The individual tag as soon as it is processed should be removed from the 
> iterator.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73848: RANGER-3621 Optimise Tag iterator

2022-02-15 Thread Ankita Sinha 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73848/
---

Review request for ranger, Madhan Neethiraj and Subhrat Chaudhary.


Bugs: RANGER-3621
https://issues.apache.org/jira/browse/RANGER-3621


Repository: ranger


Description
---

Optimise the logic of Tag iterator

Problem Statement:
Tags in the iterator are not freed till the whole processing of all Tags are 
done. So, in the case where there are lot of tag, the performance can be 
impacted as the large number of tags are held by iterator for long time. This 
causes performance degradation while retrieving the tags from db.

Solution:
The individual tag as soon as it is processed should be removed from the 
iterator.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java 
1b7e8b272 


Diff: https://reviews.apache.org/r/73848/diff/1/


Testing
---

Tested Manually by using bulk tags.


Thanks,

Ankita Sinha

Re: Review Request 73807: Support Ranger KMS integration with TencentKMS

2022-02-15 Thread Dhaval Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/#review224050
---



Hi Kirby Zhou,

Facing PMD Violation issue.

[INFO] PMD version: 6.29.0
[INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:20 
Rule:UnusedImports Priority:4 Avoid unused imports such as 
'com.microsoft.azure.keyvault.KeyVaultClient'.
[INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:45 
Rule:UnusedImports Priority:4 Avoid unused imports such as 
'org.apache.commons.lang.StringUtils'.
[INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:55 
Rule:UnusedImports Priority:4 Avoid unused imports such as 
'com.tencentcloudapi.kms.v20190118.KmsClient'.
[INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:80 
Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
'AZURE_KEYVAULT_SSL_ENABLED'..
[INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:81 
Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
'AZURE_CLIENT_ID'..
[INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:84 
Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
'AZURE_KEYVAULT_CERTIFICATE_PATH'..
[INFO] PMD Failure: org.apache.hadoop.crypto.key.RangerKeyStoreProvider:85 
Rule:UnusedPrivateField Priority:3 Avoid unused private fields such as 
'AZURE_KEYVAULT_CERTIFICATE_PASSWORD'..


Request ypu please upload the patch after successful build using command "mvn 
clean install"

- Dhaval Shah


On Feb. 15, 2022, 8:48 a.m., Kirby Zhou wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73807/
> ---
> 
> (Updated Feb. 15, 2022, 8:48 a.m.)
> 
> 
> Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, 
> Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Mehul 
> Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3580
> https://issues.apache.org/jira/browse/RANGER-3580
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger KMS integration with TencentKMS
> - This task is to integrate the RANGER KMS Service with TencentKMS.
> - To Configure RANGER KMS Service with TencentKMS below configurations need 
> to be added in install.properties file bfore running the setup.sh
> 
> ```
> # Do you use Tencent Cloud KMS? 
> TENCENT_KMS_ENABLED=true 
> # MasterKeyID on Tencent Cloud
> TENCENT_MASTERKEY_ID=YourKeyID
> # Login ID
> TENCENT_CLIENT_ID=YourClientLoginId
> # Login password
> TENCENT_CLIENT_SECRET=YourClientLoginSecret
> # Tencent Cloud area, see Tencent Cloud SDK for details. 
> TENCENT_CLIENT_REGION=ap-beijing
> ```
> 
> Run the setup.sh, It will add the below configs in dbks-site.xml
> ```
> 
> 
> ranger.kms.tencentkms.enabled
> false
> Flag for Tencent KMS
> 
> 
> ranger.kms.tencent.client.id
> 
> Tencent Client Id
> 
> 
> ranger.kms.tencent.client.secret
> 
> Tencent Client Secret
> 
> 
> ranger.kms.tencent.client.secret.alias
> ranger.ks.tencent.client.secret
> Tencent Client Secret Alias
> 
> 
> ranger.kms.tencent.client.region
> ap-beijing
> Tencent Client Id
> 
> 
> ranger.kms.tencent.masterkey.id
> 
> Tencent master key name
> 
> 
> ```
> 
> Generally, we don't want the account bound by KMS to have the right to create 
> a Key in TencentKMS. So we have to create Master Key on TencentKMS web 
> console at first.
> Start the kms service, On start Master Key from TencentKMS should be used.
> 
> 
> Diffs
> -
> 
>   distro/src/main/assembly/kms.xml 983a43e5938ecc6a02e918f587d7a8913678087e 
>   kms/config/kms-webapp/dbks-site.xml 
> 07de4d494b5d72609b47752109fc40a9e016f6ab 
>   kms/pom.xml 7a4f98df7a2244a2ae4158b32b047d77db01b0f2 
>   kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102 
>   kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java
>  f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054 
>   kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java 
> bacc928570283708daef7a2573707fddd7ca096e 
>   kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 
> 4324439ba66f9f0fb68d570f1964ed6caa8c07bd 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 
> 5234dc7422793b3b88dcc4574fafcf34556fa33f 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 
>

Re: Review Request 73526: RANGER-3373: Ranger Hbase plugin not compatible with Hbase 2.3.4

2022-02-15 Thread bhavik patel 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73526/#review224049
---


Ship it!




Ship It!

- bhavik patel


On Aug. 27, 2021, 4:32 a.m., Shivam Garg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73526/
> ---
> 
> (Updated Aug. 27, 2021, 4:32 a.m.)
> 
> 
> Review request for ranger, bhavik patel, Madhan Neethiraj, Ramesh Mani, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3373
> https://issues.apache.org/jira/browse/RANGER-3373
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger is incompatible with Hbase 2.3.4 because AccessControlLists class has 
> been changed to PermissionStorage
> 
> https://issues.apache.org/jira/browse/HBASE-22084
> 
> 
> Diffs
> -
> 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>  9be691485 
>   pom.xml 8d81988d4 
> 
> 
> Diff: https://reviews.apache.org/r/73526/diff/1/
> 
> 
> Testing
> ---
> 
> Ranger installation and setup successful.
> Validated Ranger Policies for Hbase Cluster.
> 
> 
> File Attachments
> 
> 
> RANGER-3373-001.patch
>   
> https://reviews.apache.org/media/uploaded/files/2021/08/18/d55c3e1c-cfed-4b4e-bf57-c47a21d09f62__RANGER-3373-001.patch
> 
> 
> Thanks,
> 
> Shivam Garg
> 
>

Re: Review Request 73807: Support Ranger KMS integration with TencentKMS

2022-02-15 Thread Kirby Zhou 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73807/
---

(Updated 二月 15, 2022, 8:48 a.m.)


Review request for ranger, Dhaval Shah, Dineshkumar Yadav, Gautam Borad, 
Jayendra Parab, Kishor Gollapalliwar, Abhay Kulkarni, Mateen Mansoori, Mehul 
Parikh, Pradeep Agrawal, VaradreawiZTV VaradreawiZTV, Vishal Suvagia, and 
Velmurugan Periasamy.


Bugs: RANGER-3580
https://issues.apache.org/jira/browse/RANGER-3580


Repository: ranger


Description
---

Ranger KMS integration with TencentKMS
- This task is to integrate the RANGER KMS Service with TencentKMS.
- To Configure RANGER KMS Service with TencentKMS below configurations need to 
be added in install.properties file bfore running the setup.sh

```
# Do you use Tencent Cloud KMS? 
TENCENT_KMS_ENABLED=true 
# MasterKeyID on Tencent Cloud
TENCENT_MASTERKEY_ID=YourKeyID
# Login ID
TENCENT_CLIENT_ID=YourClientLoginId
# Login password
TENCENT_CLIENT_SECRET=YourClientLoginSecret
# Tencent Cloud area, see Tencent Cloud SDK for details. 
TENCENT_CLIENT_REGION=ap-beijing
```

Run the setup.sh, It will add the below configs in dbks-site.xml
```


ranger.kms.tencentkms.enabled
false
Flag for Tencent KMS


ranger.kms.tencent.client.id

Tencent Client Id


ranger.kms.tencent.client.secret

Tencent Client Secret


ranger.kms.tencent.client.secret.alias
ranger.ks.tencent.client.secret
Tencent Client Secret Alias


ranger.kms.tencent.client.region
ap-beijing
Tencent Client Id


ranger.kms.tencent.masterkey.id

Tencent master key name


```

Generally, we don't want the account bound by KMS to have the right to create a 
Key in TencentKMS. So we have to create Master Key on TencentKMS web console at 
first.
Start the kms service, On start Master Key from TencentKMS should be used.


Diffs (updated)
-

  distro/src/main/assembly/kms.xml 983a43e5938ecc6a02e918f587d7a8913678087e 
  kms/config/kms-webapp/dbks-site.xml 07de4d494b5d72609b47752109fc40a9e016f6ab 
  kms/pom.xml 7a4f98df7a2244a2ae4158b32b047d77db01b0f2 
  kms/scripts/install.properties 31143d3426565a338c308dc1a7ea8304f3f4e102 
  kms/scripts/setup.sh 2051df59a8bb0be11ba7a54e547f78cf5a0dca36 
  
kms/src/main/java/org/apache/hadoop/crypto/key/AzureKeyVaultClientAuthenticator.java
 f96cbb7561b2c1a29b7f42c9fb3ed810b05b5054 
  kms/src/main/java/org/apache/hadoop/crypto/key/DBToAzureKeyVault.java 
bacc928570283708daef7a2573707fddd7ca096e 
  kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 
4324439ba66f9f0fb68d570f1964ed6caa8c07bd 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 
5234dc7422793b3b88dcc4574fafcf34556fa33f 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 
74c54a7a6f50878ce0f226d72a5e2c5554a0d4e5 
  
kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyVaultKeyGenerator.java 
c661268c3c25362e428884a3bb34d88d827e7f31 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerTencentKMSProvider.java 
PRE-CREATION 
  pom.xml 8a19c2de42f4ae7acff3ee9b2e399b870ef406f3 


Diff: https://reviews.apache.org/r/73807/diff/6/

Changes: https://reviews.apache.org/r/73807/diff/5-6/


Testing
---

+ mvn clean compile test verify 
+ Fresh setup


File Attachments


0001-add-TencentKMS-as-MasterKeyProvider.patch
  
https://reviews.apache.org/media/uploaded/files/2022/01/19/c0ec963d-95f0-4e77-823d-b7de9d5d54e6__0001-add-TencentKMS-as-MasterKeyProvider.patch


Thanks,

Kirby Zhou