Re: Review Request 46614: [RU] Hbase is losing table access permissions during upgrade.

2016-04-25 Thread Mahadev Konar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46614/#review130450
---


Ship it!




Ship It!

- Mahadev Konar


On April 25, 2016, 2:32 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46614/
> ---
> 
> (Updated April 25, 2016, 2:32 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-16084
> https://issues.apache.org/jira/browse/AMBARI-16084
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> So here's where we are:
> 
> - https://reviews.apache.org/r/45890/diff/2#index_header removed hard-coded 
> python code which was trying to figure out which coprocessor to use. With 
> these values gone, it's now up to Ambari to set the _right_ value in the 
> configurations.
> 
> - However, after removing the above code, we never change the ranger 
> configurations to reflect the proper value
> 
> - To fix this, we need to fix the values on Ambari upgrade. When upgrading 
> Ambari from 2.0, we'll need to do the following:
> -- If Ranger is disabled; do nothing
> -- If Ranger is enabled and the cluster's current stack is HDP 2.2, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_regionserver_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> {code}
> -- If Ranger is enabled and the cluster's current stack is HDP 2.3, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor "
> hbase_coprocessor_regionserver_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> {code}
> 
> Now HDP stack upgrades will work since we look for 
> {{XaSecureAuthorizationCoprocessor}} and change it to 
> {{RangerAuthorizationCoprocessor}} as needed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  b7665ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
>  0738d70 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  31a3ce5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  4302349 
> 
> Diff: https://reviews.apache.org/r/46614/diff/
> 
> 
> Testing
> ---
> 
> ran unit tests
> 
> Checked ambari upgrade on live cluster. Could not perform EU because Hbase 
> Regionserver started to fail after enabling Ranger HBASE Plugin
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46614: [RU] Hbase is losing table access permissions during upgrade.

2016-04-25 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46614/
---

(Updated April 25, 2016, 5:32 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Velmurugan Periasamy.


Changes
---

Fixed unit test, fixed comment


Bugs: AMBARI-16084
https://issues.apache.org/jira/browse/AMBARI-16084


Repository: ambari


Description
---

So here's where we are:

- https://reviews.apache.org/r/45890/diff/2#index_header removed hard-coded 
python code which was trying to figure out which coprocessor to use. With these 
values gone, it's now up to Ambari to set the _right_ value in the 
configurations.

- However, after removing the above code, we never change the ranger 
configurations to reflect the proper value

- To fix this, we need to fix the values on Ambari upgrade. When upgrading 
Ambari from 2.0, we'll need to do the following:
-- If Ranger is disabled; do nothing
-- If Ranger is enabled and the cluster's current stack is HDP 2.2, then set 
the master/region coprocessors to
{code}
hbase_coprocessor_master_classes = 
"com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
hbase_coprocessor_regionserver_classes = 
"com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
hbase_coprocessor_region_classes = 
"org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
{code}
-- If Ranger is enabled and the cluster's current stack is HDP 2.3, then set 
the master/region coprocessors to
{code}
hbase_coprocessor_master_classes = 
"org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor "
hbase_coprocessor_regionserver_classes = 
"org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
hbase_coprocessor_region_classes = 
"org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
{code}

Now HDP stack upgrades will work since we look for 
{{XaSecureAuthorizationCoprocessor}} and change it to 
{{RangerAuthorizationCoprocessor}} as needed.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 b7665ce 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog220.java
 0738d70 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 31a3ce5 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
 4302349 

Diff: https://reviews.apache.org/r/46614/diff/


Testing (updated)
---

ran unit tests

Checked ambari upgrade on live cluster. Could not perform EU because Hbase 
Regionserver started to fail after enabling Ranger HBASE Plugin


Thanks,

Dmitro Lisnichenko

Re: Review Request 46614: [RU] Hbase is losing table access permissions during upgrade.

2016-04-25 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46614/#review130403
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 (lines 205 - 206)


This is pretty generic and can be named isConfigEnabled(cluster, 
configType, property)

I agree that it's a bit odd that we have service specific calls in the base 
class, but it's not terrible either.  Maybe see how isRangerKnoxPluginEnabled() 
is used and switch to the more generic function if it's easy enough.


- Nate Cole


On April 24, 2016, 2:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46614/
> ---
> 
> (Updated April 24, 2016, 2:55 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-16084
> https://issues.apache.org/jira/browse/AMBARI-16084
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> So here's where we are:
> 
> - https://reviews.apache.org/r/45890/diff/2#index_header removed hard-coded 
> python code which was trying to figure out which coprocessor to use. With 
> these values gone, it's now up to Ambari to set the _right_ value in the 
> configurations.
> 
> - However, after removing the above code, we never change the ranger 
> configurations to reflect the proper value
> 
> - To fix this, we need to fix the values on Ambari upgrade. When upgrading 
> Ambari from 2.0, we'll need to do the following:
> -- If Ranger is disabled; do nothing
> -- If Ranger is enabled and the cluster's current stack is HDP 2.2, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_regionserver_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> {code}
> -- If Ranger is enabled and the cluster's current stack is HDP 2.3, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor "
> hbase_coprocessor_regionserver_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> {code}
> 
> Now HDP stack upgrades will work since we look for 
> {{XaSecureAuthorizationCoprocessor}} and change it to 
> {{RangerAuthorizationCoprocessor}} as needed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  b7665ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  31a3ce5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  4302349 
> 
> Diff: https://reviews.apache.org/r/46614/diff/
> 
> 
> Testing
> ---
> 
> unit tests in progress
> 
> live cluster check in progress
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46614: [RU] Hbase is losing table access permissions during upgrade.

2016-04-25 Thread Dmytro Grinenko 


> On April 25, 2016, 7:53 a.m., Dmytro Grinenko wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java,
> >  line 197
> > 
> >
> > Why to create new function in AbstractUpradeHelper instead of create 
> > one function for all ranger plugins with parameter for config  type?
> 
> Dmitro Lisnichenko wrote:
> Because this function would acccept few string parameters with cryptic 
> names (config type and property name) and it would be very error-prone to 
> copy-paste them in multiple places. Sooner or later, somebody will fail to 
> edit all parameters during copy-paste and get a non-obvious bug.

Probably you didn't understand, i wouldn't wana to see a function 
isRanger[SERVICE_NAME]PluginEnabled in a base class method. As we have a lot of 
services which support and would be in the future support Ranger, list of these 
function would be grows up. Moreover, i didn't see here any cryptic name passed 
here. All what you need, check property in responsible catalog. For this would 
perfectly pass enum, and this is more error less method that create 10  
functions for Ranger plugin and support them in the future separatelym which 
can create more bug in the upgrade


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46614/#review130359
---


On April 24, 2016, 6:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46614/
> ---
> 
> (Updated April 24, 2016, 6:55 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-16084
> https://issues.apache.org/jira/browse/AMBARI-16084
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> So here's where we are:
> 
> - https://reviews.apache.org/r/45890/diff/2#index_header removed hard-coded 
> python code which was trying to figure out which coprocessor to use. With 
> these values gone, it's now up to Ambari to set the _right_ value in the 
> configurations.
> 
> - However, after removing the above code, we never change the ranger 
> configurations to reflect the proper value
> 
> - To fix this, we need to fix the values on Ambari upgrade. When upgrading 
> Ambari from 2.0, we'll need to do the following:
> -- If Ranger is disabled; do nothing
> -- If Ranger is enabled and the cluster's current stack is HDP 2.2, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_regionserver_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> {code}
> -- If Ranger is enabled and the cluster's current stack is HDP 2.3, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor "
> hbase_coprocessor_regionserver_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> {code}
> 
> Now HDP stack upgrades will work since we look for 
> {{XaSecureAuthorizationCoprocessor}} and change it to 
> {{RangerAuthorizationCoprocessor}} as needed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  b7665ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  31a3ce5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  4302349 
> 
> Diff: https://reviews.apache.org/r/46614/diff/
> 
> 
> Testing
> ---
> 
> unit tests in progress
> 
> live cluster check in progress
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46614: [RU] Hbase is losing table access permissions during upgrade.

2016-04-25 Thread Dmitro Lisnichenko 


> On April 25, 2016, 10:53 a.m., Dmytro Grinenko wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java,
> >  line 197
> > 
> >
> > Why to create new function in AbstractUpradeHelper instead of create 
> > one function for all ranger plugins with parameter for config  type?

Because this function would acccept few string parameters with cryptic names 
(config type and property name) and it would be very error-prone to copy-paste 
them in multiple places. Sooner or later, somebody will fail to edit all 
parameters during copy-paste and get a non-obvious bug.


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46614/#review130359
---


On April 24, 2016, 9:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46614/
> ---
> 
> (Updated April 24, 2016, 9:55 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-16084
> https://issues.apache.org/jira/browse/AMBARI-16084
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> So here's where we are:
> 
> - https://reviews.apache.org/r/45890/diff/2#index_header removed hard-coded 
> python code which was trying to figure out which coprocessor to use. With 
> these values gone, it's now up to Ambari to set the _right_ value in the 
> configurations.
> 
> - However, after removing the above code, we never change the ranger 
> configurations to reflect the proper value
> 
> - To fix this, we need to fix the values on Ambari upgrade. When upgrading 
> Ambari from 2.0, we'll need to do the following:
> -- If Ranger is disabled; do nothing
> -- If Ranger is enabled and the cluster's current stack is HDP 2.2, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_regionserver_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> {code}
> -- If Ranger is enabled and the cluster's current stack is HDP 2.3, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor "
> hbase_coprocessor_regionserver_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> {code}
> 
> Now HDP stack upgrades will work since we look for 
> {{XaSecureAuthorizationCoprocessor}} and change it to 
> {{RangerAuthorizationCoprocessor}} as needed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  b7665ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  31a3ce5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  4302349 
> 
> Diff: https://reviews.apache.org/r/46614/diff/
> 
> 
> Testing
> ---
> 
> unit tests in progress
> 
> live cluster check in progress
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46614: [RU] Hbase is losing table access permissions during upgrade.

2016-04-25 Thread Dmytro Grinenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46614/#review130359
---




ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 (line 197)


Why to create new function in AbstractUpradeHelper instead of create one 
function for all ranger plugins with parameter for config  type?


- Dmytro Grinenko


On April 24, 2016, 6:55 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46614/
> ---
> 
> (Updated April 24, 2016, 6:55 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-16084
> https://issues.apache.org/jira/browse/AMBARI-16084
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> So here's where we are:
> 
> - https://reviews.apache.org/r/45890/diff/2#index_header removed hard-coded 
> python code which was trying to figure out which coprocessor to use. With 
> these values gone, it's now up to Ambari to set the _right_ value in the 
> configurations.
> 
> - However, after removing the above code, we never change the ranger 
> configurations to reflect the proper value
> 
> - To fix this, we need to fix the values on Ambari upgrade. When upgrading 
> Ambari from 2.0, we'll need to do the following:
> -- If Ranger is disabled; do nothing
> -- If Ranger is enabled and the cluster's current stack is HDP 2.2, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_regionserver_classes = 
> "com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor"
> {code}
> -- If Ranger is enabled and the cluster's current stack is HDP 2.3, then set 
> the master/region coprocessors to
> {code}
> hbase_coprocessor_master_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor "
> hbase_coprocessor_regionserver_classes = 
> "org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> hbase_coprocessor_region_classes = 
> "org.apache.hadoop.hbase.security.token.TokenProvider,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint,org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor"
> {code}
> 
> Now HDP stack upgrades will work since we look for 
> {{XaSecureAuthorizationCoprocessor}} and change it to 
> {{RangerAuthorizationCoprocessor}} as needed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  b7665ce 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  31a3ce5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  4302349 
> 
> Diff: https://reviews.apache.org/r/46614/diff/
> 
> 
> Testing
> ---
> 
> unit tests in progress
> 
> live cluster check in progress
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>