[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189341#comment-17189341
]
Jiayi Liu commented on RANGER-2621:
---
When setting the rest url of ranger admin in ranger-hive-security.xml, we must
use hostname instead of ip address so that we can pass kerberos SPNEGO
authentication. The reason for the previous unsuccess is that I have been using
the ip address. After modifying it to hostname, the policy can be downloaded
successfully.
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189280#comment-17189280
]
Jiayi Liu commented on RANGER-2621:
---
I encountered the same problem. It is ok in the same cluster. Authentication
fails when the plugin and admin are not in the same cluster.
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023200#comment-17023200
]
Ramesh Mani commented on RANGER-2621:
-
[~susidev33] Do you see error in ranger admin log when the policy download call
comes in?
Also does the ranger-admin have the core-site.xml in the class path for the
necessary auth_to_local conversion happening for
"*{color:#de350b}hive/i...@domain.net"{color}*
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17022550#comment-17022550
]
Neeraj Verma commented on RANGER-2621:
--
I am also facing issue with Ranger 2.0.0 Ranger hive plugin . Our EMR cluster
is not Kerberoized .. any help really appriciated
{code:java}
Caused by: java.lang.NoSuchFieldError: REPLLOAD
at
org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.checkPrivileges(RangerHiveAuthorizer.java:694)
~[ranger-hive-plugin-2.0.0.jar:2.0.0]
at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:974)
~[hive-exec-2.3.5-amzn-1.jar:2.3.5-amzn-1]
{code}
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020237#comment-17020237
]
Leandro Loos commented on RANGER-2621:
--
[~susidev33] were you able to work the issue out? I'm facing the same problem
on a nearly equal scenario (but with my own kerberos servers also on different
EC2 instances)
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989411#comment-16989411
]
Sunil Kumar S commented on RANGER-2621:
---
@[~vel] [~susidev33]
I am also trying to implement Ranger on EMR.
Ranger on EMR with out Kerberos is working fine with 0.7.1 Ranger version.
When I try to install Ranger on EMR with Kerberos(Ranger on master node and
ranger on EC2 instance) both fails.
You mentioned that Ranger on Master node with Kerberos worked fine, can you
please let me know what Ranger version did you use and also point me with the
path/steps for Installation.
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957148#comment-16957148
]
Susi Dev commented on RANGER-2621:
--
[^hive-plugin.txt]
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt, hive-plugin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957146#comment-16957146
]
Susi Dev commented on RANGER-2621:
--
[^Ranger-admin.txt]
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
> Attachments: Ranger-admin.txt
>
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16957145#comment-16957145
]
Susi Dev commented on RANGER-2621:
--
[~vel] :
There is some additional setting that Ranger is expecting/missing. Let me break
it down, so we know what is configured as per doc and what is missing...
By following the documentation we have below configuration;
{color:#0747a6}*Ranger Admin (Attached full file):*{color}
{color:#00875a}authentication_method={color:#172b4d}UNIX{color}{color}
{color:#00875a}remoteLoginEnabled={color:#172b4d}true{color}{color}
{color:#00875a}authServiceHostName={color:#172b4d}localhost{color}{color}
{color:#00875a}authServicePort={color:#172b4d}5151{color}{color}
{color:#00875a}# Kerberos Config -{color}
{color:#00875a}spnego_principal={color:#172b4d}HTTP/ip-10-6-62-...@example.net{color}{color}
{color:#00875a}spnego_keytab={color:#172b4d}/usr/local/ranger-admin/keytabs/spnego.service.keytab{color}{color}
{color:#00875a}token_valid=30{color}
{color:#00875a}cookie_domain=i{color:#172b4d}p-10-6-62-150{color}{color}
{color:#00875a}cookie_path=/{color}
{color:#00875a}admin_principal={color:#172b4d}rangeradmin/ip-10-6-62-...@example.net{color}{color}
{color:#00875a}admin_keytab={color:#172b4d}/usr/local/ranger-admin/keytabs/rangeradmin.keytab{color}{color}
{color:#00875a}lookup_principal={color:#172b4d}rangerlookup/ip-10-6-62-...@example.net{color}{color}
{color:#00875a}lookup_keytab={color:#172b4d}/usr/local/ranger-admin/keytabs/rangerlookup.keytab{color}{color}
{color:#00875a}hadoop_conf={color:#172b4d}/etc/hadoop/conf{color}{color}
*## Note:*
* Is hadoop_conf parameter is referring to localhost? because there is no
hadoop installed in Ranger Admin Server, its a Vanilla RHEL node.
* all the principals exists in KDC Server in EMR Master Node, which is
reachable, the krb5.conf is updated properly at Ranger Server host and able to
authenticate via keytabs.
{color:#0747a6}*Hive-Plugin(In Ranger UI):*{color}
{color:#ff}Plugin Config:{color}
Service Name : hivedev
Active Status: Enabled
{color:#ff}Config Properties :{color}
Username : rangeradmin/_hostn...@example.net
Password :
jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
Common Name for Certificate:
{color:#de350b}Add New Configurations: (Tried all three values individually by
replacing the val everytime){color}
||Name||Value||
|policy.download.auth.users | rangeradmin/hostn...@domain.net | |
||Name||Value||
|policy.download.auth.users | hive/hostn...@domain.net | |
||Name||Value||
|policy.download.auth.users | hive| |
*From EMR Master Node:*
Enable Hive-plugin(install.properties):
POLICY_MGR_URL=[http://ip-10-6-62-186:6080|http://ip-10-6-62-186:6080/]
REPOSITORY_NAME=hivedev
[^Ranger-admin.txt][^hive-plugin.txt]
When we enable the hive plugin, it is trying perform the REST call to get the
policies and update the cache file, but there is no configuration mentioned
about which user does the enable pluging script uses to authenticate against
Ranger.
This is the error we get..
+*Hive Server 2 log:*+
2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
(RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
policies. secureMode=true, *{color:#de350b}user=hive/i...@domain.net{color}*
{color:#de350b}(*auth:KERBEROS*{color}),
response={"httpStatusCode":401,"statusCode":401,{color:#de350b}"msgDesc":"Authentication
Failed"{color}}, serviceName=hivedev
Our question is how to make sure the REST call go through without
authentication or how to configure that?
If I run the curl statement with admin:Admin@123 credential, the policy gets
downloaded. Now sure, how to make enable hive plugin use these credentials to
download policies?
Ironically, this issue goes away when Ranger and Kerberos servers are in the
same host.
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
>
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956400#comment-16956400
]
Velmurugan Periasamy commented on RANGER-2621:
--
[~susidev33] - based on your description, it looks like kerberos configuration
issue, not a ranger issue. In kerberized env, plugins download policies using
kerberos principal that the host component (for example hiveserver2 in case of
hive plugin) is configured with. If Ranger admin cannot trust these kerberos
identities, that would be the issue.
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956207#comment-16956207
]
Susi Dev commented on RANGER-2621:
--
[~vel]
Thank you for giving some insights..
1) Yes, we tried different combinations... We created local users with the
principal name given here, changed it to hive principal as well. Yet, it won't
go through. The crucial information here is that ... *Ranger* is installed on a
*standalone EC2* whereas *Kerberos* server is present in *EMR Master Node*. If
Ranger server is also installed on EMR Master Node, then the policy download
works just fine. Only if we place the *Ranger Server* on a *different host*
than the *Kerberos* server, we are running into this issue. So I assume that
it is trying to authenticate with some user account but not sure which one it
is using and how to configure that.. Perhaps, that is the only missing piece in
getting this work. Please throw some light if there are any pre-reqs here.
2) Yes, We are running latest Ranger version that was built recently from the
git master branch. I hope it has all the latest break-fixes.
h2. {color:#4c9aff}Your timely help is very much appreciated. Thanks again.
{color}
CC [~rmani] / [~mehul] / [~abhayk]
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2621) Ranger Policy Update fails on Kerberized Cluster
[
https://issues.apache.org/jira/browse/RANGER-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953796#comment-16953796
]
Velmurugan Periasamy commented on RANGER-2621:
--
1] Regarding error in kerberized env, policy.download.auth.users should be
configured as the right user that is getting passed after the auth-to-rules
translation. Could you please verify that?
2] Regarding plugin error, verify if hive service def is updated (See
https://issues.apache.org/jira/browse/RANGER-2389). Did you upgrade the old
cluster?
CC [~rmani] / [~mehul] / [~abhayk]
> Ranger Policy Update fails on Kerberized Cluster
>
>
> Key: RANGER-2621
> URL: https://issues.apache.org/jira/browse/RANGER-2621
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Affects Versions: 2.0.0
>Reporter: Susi Dev
>Priority: Major
>
> {color:#4c9aff}Can someone help configuring RANGER for KERBERIZED cluster
> ??{color}
> We have Ranger 2.0 installed on separate EC2 node, while trying to integrate
> with EMR cluster.
> When the EMR cluster is not kerberized, the policy sync works just fine..
> When EMR is kerberized, policy download does not work anymore...
>
> We see below error:
> +*Access Log:*+
> 10.23.123.150 - - [14/Oct/2019:20:07:09 +] "GET
> /service/plugins/secure/policies/download/hadoopdev?supportsPolicyDeltas=false
> HTTP/1.1" 401 52 "-" "curl/7.61.1"
>
> +*Hive Server 2 log:*+
> 2019-10-14T20:03:34,353 WARN [Thread-8([])]: client.RangerAdminRESTClient
> (RangerAdminRESTClient.java:getServicePoliciesIfUpdated(186)) - Error getting
> policies. secureMode=true, user=hive/i...@domain.net (auth:KERBEROS),
> response=\{"httpStatusCode":401,"statusCode":401,"msgDesc":"Authentication
> Failed"}, serviceName=hivedev
>
> +*Plugin Error(Test Connection):*+
> org.apache.ranger.plugin.client.HadoopException: Unable to execute SQL [show
> databases like "*"]..
> Unable to execute SQL [show databases like "*"]..
> Error running query: java.lang.NoSuchFieldError: REPLLOAD.
> REPLLOAD.
>
>
> {color:#FF}Plugin Config:{color}
> Service Name : hivedev
> Active Status: Enabled
>
> {color:#FF}Config Properties :{color}
> Username : Rangeradmin/_hostn...@domain.net
> Password :
> jdbc.driverClassName: org.apache.hive.jdbc.HiveDriver
> jdbc.url: jdbc:hive2://hostname:1/;principal=hive/hostn...@domain.net
> Common Name for Certificate:
> Add New Configurations
> ||Name||Value||
> |policy.download.auth.users | rangeradmin/hostn...@domain.net | |
>
>
> {color:#FF}*Ranger 2.0 looks great but with not enough documentation
> around the installation and configuration, we are all handicapped when it
> comes to using. Appreciate if some of you add good documentation, it helps us
> appreciate the amount of work done by you ... Right now, we are only shooting
> in the DARK.*{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
