Review Request 61877: No Hosts are Targeted For MAINT VDF If Available Services is Empty

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61877/
---

Review request for Ambari, Dmytro Grinenko and Dmitro Lisnichenko.


Bugs: AMBARI-21801
https://issues.apache.org/jira/browse/AMBARI-21801


Repository: ambari


Description
---

When trying to distribute a MAINT repository which doesn't have available 
services, no hosts are targeted and the installation doesn't do anything.

The fix is to use the {{ClusterSummary}} from the VDF which includes a mashup 
of the cluster's services and those listed in the VDF's manifest and available 
services structure.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 3fb45ab3ff 


Diff: https://reviews.apache.org/r/61877/diff/1/


Testing
---

PENDING


Thanks,

Jonathan Hurley

Re: Review Request 61875: Oozie server crashes post migration after regen kerberos keytabs and restart all services

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61875/#review183695
---


Ship it!




Ship It!

- Sid Wagle


On Aug. 24, 2017, 1:54 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61875/
> ---
> 
> (Updated Aug. 24, 2017, 1:54 a.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Di Li, Eugene 
> Chekanskiy, Laszlo Puskas, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21798
> https://issues.apache.org/jira/browse/AMBARI-21798
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This is specifically for the IOP *4.2.0 *to HDP migration
> 
> Start with an IOP 4.2.0 cluster with both Oozie and Knox installed, enable 
> Kerberos then run thru the migration. Once EU is finalized, regen Kerberos 
> keytabs via Ambari web UI and check the checkbox to make Ambari restart all 
> services as part of the regen process.
> 
> Notice Oozie crashes after a successful restart with following error in the 
> oozie.log file
> {noformat}
> 2017-08-23 12:39:48,480 FATAL Services:514 - SERVER[umiak2.fyre.ibm.com] 
> E0550: Could not normalize host name [${host}], Could not resolve host 
> [${host}], ${host}: Name or service not known 
> org.apache.oozie.service.ServiceException: E0550: Could not normalize host 
> name [${host}], Could not resolve host [${host}], ${host}: Name or service 
> not known
> {noformat}
> This is because Knox kerberos.json in IOP 4.2.0 has oozie proxy setting as 
> {{"oozie.service.ProxyUserService.proxyuser.knox.hosts": "${hosts}",}}
> 
> In HDP 2.6.2, the same property  is now 
> {{"oozie.service.ProxyUserService.proxyuser.${knox-env/knox_user}.hosts": 
> "${clusterHostInfo/knox_gateway_hosts}"}}, where the ${hosts} placeholder is 
> changed and Knox params.py no longer has the logic to calculate it .
> 
> *A manual fix* is to change Kerboers config for Knox - update 
> {{"oozie.service.ProxyUserService.proxyuser.knox.hosts"}} to have value 
> {{"${clusterHostInfo/knox_gateway_hosts}"}}  ( done via UI, this will kick 
> off regen keytabs and restart services)
> 
> # Solution
> Find and remove configuration specifications for the following configurations 
> in KNOX/KNOX_GATEWAY since they are invalid due to static "knox" embedded in 
> the property name and the values like ${host}:
>  - oozie-site/oozie.service.ProxyUserService.proxyuser.knox.groups
>  - oozie-site/oozie.service.ProxyUserService.proxyuser.knox.hosts
>  - webhcat-site/webhcat.proxyuser.knox.groups
>  - webhcat-site/webhcat.proxyuser.knox.hosts
>  - core-site/hadoop.proxyuser.knox.groups
>  - core-site/hadoop.proxyuser.knox.hosts
>  - 
> falcon-runtime.properties/*.falcon.service.ProxyUserService.proxyuser.knox.groups
>  - 
> falcon-runtime.properties/*.falcon.service.ProxyUserService.proxyuser.knox.hosts
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
>  b83ba643ec 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java
>  d7df68c122 
> 
> 
> Diff: https://reviews.apache.org/r/61875/diff/1/
> 
> 
> Testing
> ---
> 
> manually tested
> 
> # Local test results: PENDING
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 61876: Some services had issue coming up after express upgrade on IOP clusters

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61876/
---

Review request for Ambari, Jonathan Hurley, Robert Levas, and Sumit Mohanty.


Bugs: AMBARI-21800
https://issues.apache.org/jira/browse/AMBARI-21800


Repository: ambari


Description
---

Some services had an issue coming up after express upgrade on IOP clusters. 
Express upgrade was successful as per Ambari UI.

It seems like something is wrong with RCO post-upgrade where the older stack 
version RCO gets recached.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
 234ef26 


Diff: https://reviews.apache.org/r/61876/diff/1/


Testing
---


Thanks,

Sid Wagle

Re: Review Request 61876: Some services had issue coming up after express upgrade on IOP clusters

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61876/#review183694
---


Ship it!




Ship It!

- Jonathan Hurley


On Aug. 23, 2017, 10:30 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61876/
> ---
> 
> (Updated Aug. 23, 2017, 10:30 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Robert Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21800
> https://issues.apache.org/jira/browse/AMBARI-21800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Some services had an issue coming up after express upgrade on IOP clusters. 
> Express upgrade was successful as per Ambari UI.
> 
> It seems like something is wrong with RCO post-upgrade where the older stack 
> version RCO gets recached.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  234ef26 
> 
> 
> Diff: https://reviews.apache.org/r/61876/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 61875: Oozie server crashes post migration after regen kerberos keytabs and restart all services

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61875/
---

(Updated Aug. 23, 2017, 9:54 p.m.)


Review request for Ambari, Attila Magyar, Balázs Bence Sári, Di Li, Eugene 
Chekanskiy, Laszlo Puskas, Sebastian Toader, and Sid Wagle.


Bugs: AMBARI-21798
https://issues.apache.org/jira/browse/AMBARI-21798


Repository: ambari


Description (updated)
---

This is specifically for the IOP *4.2.0 *to HDP migration

Start with an IOP 4.2.0 cluster with both Oozie and Knox installed, enable 
Kerberos then run thru the migration. Once EU is finalized, regen Kerberos 
keytabs via Ambari web UI and check the checkbox to make Ambari restart all 
services as part of the regen process.

Notice Oozie crashes after a successful restart with following error in the 
oozie.log file
{noformat}
2017-08-23 12:39:48,480 FATAL Services:514 - SERVER[umiak2.fyre.ibm.com] E0550: 
Could not normalize host name [${host}], Could not resolve host [${host}], 
${host}: Name or service not known 
org.apache.oozie.service.ServiceException: E0550: Could not normalize host name 
[${host}], Could not resolve host [${host}], ${host}: Name or service not known
{noformat}
This is because Knox kerberos.json in IOP 4.2.0 has oozie proxy setting as 
{{"oozie.service.ProxyUserService.proxyuser.knox.hosts": "${hosts}",}}

In HDP 2.6.2, the same property  is now 
{{"oozie.service.ProxyUserService.proxyuser.${knox-env/knox_user}.hosts": 
"${clusterHostInfo/knox_gateway_hosts}"}}, where the ${hosts} placeholder is 
changed and Knox params.py no longer has the logic to calculate it .

*A manual fix* is to change Kerboers config for Knox - update 
{{"oozie.service.ProxyUserService.proxyuser.knox.hosts"}} to have value 
{{"${clusterHostInfo/knox_gateway_hosts}"}}  ( done via UI, this will kick off 
regen keytabs and restart services)

# Solution
Find and remove configuration specifications for the following configurations 
in KNOX/KNOX_GATEWAY since they are invalid due to static "knox" embedded in 
the property name and the values like ${host}:
 - oozie-site/oozie.service.ProxyUserService.proxyuser.knox.groups
 - oozie-site/oozie.service.ProxyUserService.proxyuser.knox.hosts
 - webhcat-site/webhcat.proxyuser.knox.groups
 - webhcat-site/webhcat.proxyuser.knox.hosts
 - core-site/hadoop.proxyuser.knox.groups
 - core-site/hadoop.proxyuser.knox.hosts
 - 
falcon-runtime.properties/*.falcon.service.ProxyUserService.proxyuser.knox.groups
 - 
falcon-runtime.properties/*.falcon.service.ProxyUserService.proxyuser.knox.hosts


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 b83ba643ec 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java
 d7df68c122 


Diff: https://reviews.apache.org/r/61875/diff/1/


Testing
---

manually tested

# Local test results: PENDING

# Jenkins test results: PENDING


Thanks,

Robert Levas

Review Request 61875: Oozie server crashes post migration after regen kerberos keytabs and restart all services

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61875/
---

Review request for Ambari, Attila Magyar, Balázs Bence Sári, Di Li, Eugene 
Chekanskiy, Laszlo Puskas, Sebastian Toader, and Sid Wagle.


Bugs: AMBARI-21798
https://issues.apache.org/jira/browse/AMBARI-21798


Repository: ambari


Description
---

This is specifically for the IOP *4.2.0 *to HDP migration

Start with an IOP 4.2.0 cluster with both Oozie and Knox installed, enable 
Kerberos then run thru the migration. Once EU is finalized, regen Kerberos 
keytabs via Ambari web UI and check the checkbox to make Ambari restart all 
services as part of the regen process.

Notice Oozie crashes after a successful restart with following error in the 
oozie.log file
{noformat}
2017-08-23 12:39:48,480 FATAL Services:514 - SERVER[umiak2.fyre.ibm.com] E0550: 
Could not normalize host name [${host}], Could not resolve host [${host}], 
${host}: Name or service not known 
org.apache.oozie.service.ServiceException: E0550: Could not normalize host name 
[${host}], Could not resolve host [${host}], ${host}: Name or service not known
{noformat}
This is because Knox kerberos.json in IOP 4.2.0 has oozie proxy setting as 
{{"oozie.service.ProxyUserService.proxyuser.knox.hosts": "${hosts}",}}

In HDP 2.6.2, the same property  is now 
{{"oozie.service.ProxyUserService.proxyuser.${knox-env/knox_user}.hosts": 
"${clusterHostInfo/knox_gateway_hosts}"}}, where the ${hosts} placeholder is 
changed and Knox params.py no longer has the logic to calculate it .

*A manual fix* is to change Kerboers config for Knox - update 
{{"oozie.service.ProxyUserService.proxyuser.knox.hosts"}} to have value 
{{"${clusterHostInfo/knox_gateway_hosts}"}}  ( done via UI, this will kick off 
regen keytabs and restart services)


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java
 b83ba643ec 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog252Test.java
 d7df68c122 


Diff: https://reviews.apache.org/r/61875/diff/1/


Testing
---

manually tested

# Local test results: PENDING

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 61856: Auto fix service config mapping issue discovered during ambari-upgrade

[Vitalyi Brodetskyi]

> On Сер. 23, 2017, 6:59 після полудня, Sid Wagle wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
> > Lines 160 (patched)
> > 
> >
> > Why doesn't this check is service exists or not?

I think that this query will be enough: "SELECT clusterConfig FROM 
ClusterConfigEntity clusterConfig WHERE clusterConfig.serviceConfigEntities IS 
EMPTY AND clusterConfig.type != 'cluster-env'". Using it we will get all 
configs from clusterconfig table, which are not linked ti any service config. I 
think it's safe. Anyway we are using same query in db consistency check and 
they will be removed according to auto fix. One more, if we will just check 
configs without service, there are some cases which will not be covered. Like 
this one (for example): 1) Deploy cluster with hbase 2) Remove hbase. Configs 
will stay. 3) Add hbase. Old configs will become orphaned, and hbase will 
create and use new one. So, to my mind query that i used is correct/safe and 
will cover all needed cases.


> On Сер. 23, 2017, 6:59 після полудня, Sid Wagle wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
> > Lines 168 (patched)
> > 
> >
> > Is this within a transaction boundary?

Sorry, missed that. Done.


> On Сер. 23, 2017, 6:59 після полудня, Sid Wagle wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
> > Lines 48 (patched)
> > 
> >
> > Make sure to run checkstyle on this.

done.


- Vitalyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61856/#review183641
---


On Сер. 23, 2017, 9:14 після полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61856/
> ---
> 
> (Updated Сер. 23, 2017, 9:14 після полудня)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
> 
> 
> Bugs: AMBARI-21795
> https://issues.apache.org/jira/browse/AMBARI-21795
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If user has deleted services, after upgrade db consistency check will throw 
> warnings about unmapped configs.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  40049c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigEntity.java
>  34f3034 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
> 319bf53 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
>  7fc392d 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 9cdf8e8 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql d616e77 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0530651 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 1d16eb6 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql da23e95 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e410555 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
>  44b5d91 
> 
> 
> Diff: https://reviews.apache.org/r/61856/diff/3/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 61856: Auto fix service config mapping issue discovered during ambari-upgrade

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61856/
---

(Updated Сер. 23, 2017, 9:14 після полудня)


Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.


Bugs: AMBARI-21795
https://issues.apache.org/jira/browse/AMBARI-21795


Repository: ambari


Description
---

If user has deleted services, after upgrade db consistency check will throw 
warnings about unmapped configs.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 40049c0 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigEntity.java
 34f3034 
  ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
319bf53 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
 7fc392d 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 9cdf8e8 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql d616e77 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0530651 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 1d16eb6 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql da23e95 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e410555 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
 44b5d91 


Diff: https://reviews.apache.org/r/61856/diff/3/

Changes: https://reviews.apache.org/r/61856/diff/2-3/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 61707: Pre-configure services when Kerberos is enabled to reduce number of core service restarts when services are added

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61707/#review183664
---


Ship it!




Ship It!

- Jonathan Hurley


On Aug. 23, 2017, 3:37 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61707/
> ---
> 
> (Updated Aug. 23, 2017, 3:37 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21602
> https://issues.apache.org/jira/browse/AMBARI-21602
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pre-configure (certain) services when Kerberos is enabled to reduce number of 
> core service restarts when services are added.  
> 
> While processing the Kerberos descriptor, include services marked to be 
> _pre-configured_.  When a tagged service is encountered, process it weather 
> it is installed or not. However if it is not installed, only apply 
> configuration changes for existing configuration types.  This will set at 
> least the core-site changes related to proxyuser and auth-to-local rules 
> properties. By doing this, if a tagged service is later installed, the 
> settings will already be in place in the existing service configs and thus 
> the existing services will not need to be restarted. 
> 
> Caveats:
> - Default values for the uninstalled, tagged, services will be assumed 
> - The stack advisor will be used to suggest locations of components - used to 
> build the clusterHostInfo structure that may be used to derive property 
> values. 
> 
> Note: This processing is to occur when Kerberos is enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  7eb82e4137 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java
>  7ba1b1887d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
>  3329e76226 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  a334542b8b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  8d0fd0fd93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  91a84ea643 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
>  59bd96a8f5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackVersionResourceProvider.java
>  64ead405eb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
>  d4bb501231 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
>  46f5642eb1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/customactions/ActionDefinitionManager.java
>  1f2d26f079 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java
>  dd2b2237e5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerAction.java
>  2e331bb77f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PreconfigureServiceType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareDisableKerberosServerAction.java
>  60523cdd75 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareEnableKerberosServerAction.java
>  ca15695564 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
>  f239cffd93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  78aaa77a48 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
>  4b6e9a8a63 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionDirectory.java
>  131318b5a1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  abad7edd9b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> 74120b6ab8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  23fd0a95f5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 245e6236e5 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> ff1d37808d 
>   
> 

Re: Review Request 61861: Post Ambari upgrade on a migration cluster Oozie failed to restart when LZO compression is enabled

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61861/#review183663
---


Ship it!




Ship It!

- Tim Thorpe


On Aug. 23, 2017, 8:25 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61861/
> ---
> 
> (Updated Aug. 23, 2017, 8:25 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Robert Levas, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-21794
> https://issues.apache.org/jira/browse/AMBARI-21794
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Restart Oozie fails due lzo package installation failure. The rpm package 
> name is different for IOP stack vs HDP.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/alerts/alert_check_oozie_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/oozieSmoke2.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/prepareOozieHdfsDirectories.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/wrap_ooziedb.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/check_oozie_server_status.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_client.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server_upgrade.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_windows.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/status_params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/adminusers.txt.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/zkmigrator_jaas.conf.j2
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61861/diff/1/
> 
> 
> Testing
> ---
> 
> Manual verification pending.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 61861: Post Ambari upgrade on a migration cluster Oozie failed to restart when LZO compression is enabled

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61861/#review183660
---


Ship it!




Ship It!

- Jonathan Hurley


On Aug. 23, 2017, 4:25 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61861/
> ---
> 
> (Updated Aug. 23, 2017, 4:25 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Robert Levas, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-21794
> https://issues.apache.org/jira/browse/AMBARI-21794
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Restart Oozie fails due lzo package installation failure. The rpm package 
> name is different for IOP stack vs HDP.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/alerts/alert_check_oozie_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/oozieSmoke2.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/prepareOozieHdfsDirectories.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/wrap_ooziedb.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/check_oozie_server_status.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_client.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server_upgrade.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_windows.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/status_params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/adminusers.txt.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/zkmigrator_jaas.conf.j2
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61861/diff/1/
> 
> 
> Testing
> ---
> 
> Manual verification pending.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 61707: Pre-configure services when Kerberos is enabled to reduce number of core service restarts when services are added

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61707/#review183659
---


Ship it!




Ship It!

- Nate Cole


On Aug. 23, 2017, 3:37 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61707/
> ---
> 
> (Updated Aug. 23, 2017, 3:37 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21602
> https://issues.apache.org/jira/browse/AMBARI-21602
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pre-configure (certain) services when Kerberos is enabled to reduce number of 
> core service restarts when services are added.  
> 
> While processing the Kerberos descriptor, include services marked to be 
> _pre-configured_.  When a tagged service is encountered, process it weather 
> it is installed or not. However if it is not installed, only apply 
> configuration changes for existing configuration types.  This will set at 
> least the core-site changes related to proxyuser and auth-to-local rules 
> properties. By doing this, if a tagged service is later installed, the 
> settings will already be in place in the existing service configs and thus 
> the existing services will not need to be restarted. 
> 
> Caveats:
> - Default values for the uninstalled, tagged, services will be assumed 
> - The stack advisor will be used to suggest locations of components - used to 
> build the clusterHostInfo structure that may be used to derive property 
> values. 
> 
> Note: This processing is to occur when Kerberos is enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  7eb82e4137 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java
>  7ba1b1887d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
>  3329e76226 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  a334542b8b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  8d0fd0fd93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  91a84ea643 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
>  59bd96a8f5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackVersionResourceProvider.java
>  64ead405eb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
>  d4bb501231 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
>  46f5642eb1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/customactions/ActionDefinitionManager.java
>  1f2d26f079 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java
>  dd2b2237e5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerAction.java
>  2e331bb77f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PreconfigureServiceType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareDisableKerberosServerAction.java
>  60523cdd75 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareEnableKerberosServerAction.java
>  ca15695564 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
>  f239cffd93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  78aaa77a48 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
>  4b6e9a8a63 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionDirectory.java
>  131318b5a1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  abad7edd9b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> 74120b6ab8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  23fd0a95f5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
> 245e6236e5 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> ff1d37808d 
>   
> 

Re: Review Request 61861: Post Ambari upgrade on a migration cluster Oozie failed to restart when LZO compression is enabled

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61861/#review183656
---


Ship it!




Ship It!

- Di Li


On Aug. 23, 2017, 8:25 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61861/
> ---
> 
> (Updated Aug. 23, 2017, 8:25 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Robert Levas, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-21794
> https://issues.apache.org/jira/browse/AMBARI-21794
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Restart Oozie fails due lzo package installation failure. The rpm package 
> name is different for IOP stack vs HDP.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/alerts/alert_check_oozie_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/oozieSmoke2.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/prepareOozieHdfsDirectories.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/wrap_ooziedb.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/check_oozie_server_status.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_client.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server_upgrade.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_windows.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/status_params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/adminusers.txt.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie.conf.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/zkmigrator_jaas.conf.j2
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61861/diff/1/
> 
> 
> Testing
> ---
> 
> Manual verification pending.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Review Request 61861: Post Ambari upgrade on a migration cluster Oozie failed to restart when LZO compression is enabled

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61861/
---

Review request for Ambari, Di Li, Jonathan Hurley, Robert Levas, and Sumit 
Mohanty.


Bugs: AMBARI-21794
https://issues.apache.org/jira/browse/AMBARI-21794


Repository: ambari


Description
---

Restart Oozie fails due lzo package installation failure. The rpm package name 
is different for IOP stack vs HDP.


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/alerts/alert_check_oozie_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/oozieSmoke2.sh
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/prepareOozieHdfsDirectories.sh
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/wrap_ooziedb.sh
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/check_oozie_server_status.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_client.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server_upgrade.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_windows.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/status_params.py
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/adminusers.txt.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie-log4j.properties.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie.conf.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/zkmigrator_jaas.conf.j2
 PRE-CREATION 


Diff: https://reviews.apache.org/r/61861/diff/1/


Testing
---

Manual verification pending.


Thanks,

Sid Wagle

Re: Review Request 61707: Pre-configure services when Kerberos is enabled to reduce number of core service restarts when services are added

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61707/
---

(Updated Aug. 23, 2017, 3:37 p.m.)


Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene Chekanskiy, 
Jonathan Hurley, Laszlo Puskas, Nate Cole, and Sebastian Toader.


Changes
---

Updates based on reviewer comments.


Bugs: AMBARI-21602
https://issues.apache.org/jira/browse/AMBARI-21602


Repository: ambari


Description
---

Pre-configure (certain) services when Kerberos is enabled to reduce number of 
core service restarts when services are added.  

While processing the Kerberos descriptor, include services marked to be 
_pre-configured_.  When a tagged service is encountered, process it weather it 
is installed or not. However if it is not installed, only apply configuration 
changes for existing configuration types.  This will set at least the core-site 
changes related to proxyuser and auth-to-local rules properties. By doing this, 
if a tagged service is later installed, the settings will already be in place 
in the existing service configs and thus the existing services will not need to 
be restarted. 

Caveats:
- Default values for the uninstalled, tagged, services will be assumed 
- The stack advisor will be used to suggest locations of components - used to 
build the clusterHostInfo structure that may be used to derive property values. 

Note: This processing is to occur when Kerberos is enabled.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
 7eb82e4137 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java
 7ba1b1887d 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
 3329e76226 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
 a334542b8b 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 8d0fd0fd93 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 91a84ea643 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
 59bd96a8f5 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackVersionResourceProvider.java
 64ead405eb 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
 d4bb501231 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
 46f5642eb1 
  
ambari-server/src/main/java/org/apache/ambari/server/customactions/ActionDefinitionManager.java
 1f2d26f079 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java
 dd2b2237e5 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerAction.java
 2e331bb77f 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PreconfigureServiceType.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareDisableKerberosServerAction.java
 60523cdd75 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareEnableKerberosServerAction.java
 ca15695564 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
 f239cffd93 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
 78aaa77a48 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
 4b6e9a8a63 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionDirectory.java
 131318b5a1 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
 abad7edd9b 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
74120b6ab8 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
23fd0a95f5 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 
245e6236e5 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
ff1d37808d 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
cb98dd2c4e 
  ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
573a197a2b 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
350611dac8 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptor.java
 7f53daa357 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosDescriptor.java
 eba1b3aecc 
  

Re: Review Request 61707: Pre-configure services when Kerberos is enabled to reduce number of core service restarts when services are added

[Robert Levas]

> On Aug. 21, 2017, 7:11 a.m., Sebastian Toader wrote:
> > ```HDP/2.6/kerberos_preconfigure.json``` has to be added to ```HDP/3.0/``` 
> > as well.

I beleive that this is not needed at this time. The requirement is to make this 
work for HDP 2.6. It is not clear whether HDP 3.0.0 need this since features of 
Ambari 3.0.0 may have a better way to handle service restarts.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61707/#review183323
---


On Aug. 23, 2017, 3:37 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61707/
> ---
> 
> (Updated Aug. 23, 2017, 3:37 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21602
> https://issues.apache.org/jira/browse/AMBARI-21602
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pre-configure (certain) services when Kerberos is enabled to reduce number of 
> core service restarts when services are added.  
> 
> While processing the Kerberos descriptor, include services marked to be 
> _pre-configured_.  When a tagged service is encountered, process it weather 
> it is installed or not. However if it is not installed, only apply 
> configuration changes for existing configuration types.  This will set at 
> least the core-site changes related to proxyuser and auth-to-local rules 
> properties. By doing this, if a tagged service is later installed, the 
> settings will already be in place in the existing service configs and thus 
> the existing services will not need to be restarted. 
> 
> Caveats:
> - Default values for the uninstalled, tagged, services will be assumed 
> - The stack advisor will be used to suggest locations of components - used to 
> build the clusterHostInfo structure that may be used to derive property 
> values. 
> 
> Note: This processing is to occur when Kerberos is enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  7eb82e4137 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java
>  7ba1b1887d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/DeleteIdentityHandler.java
>  3329e76226 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  a334542b8b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  8d0fd0fd93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  91a84ea643 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterKerberosDescriptorResourceProvider.java
>  59bd96a8f5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackVersionResourceProvider.java
>  64ead405eb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/RemovableIdentities.java
>  d4bb501231 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/UsedIdentities.java
>  46f5642eb1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/customactions/ActionDefinitionManager.java
>  1f2d26f079 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/AbstractPrepareKerberosServerAction.java
>  dd2b2237e5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/KerberosServerAction.java
>  2e331bb77f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PreconfigureServiceType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareDisableKerberosServerAction.java
>  60523cdd75 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareEnableKerberosServerAction.java
>  ca15695564 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
>  f239cffd93 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/UpgradeUserKerberosDescriptor.java
>  78aaa77a48 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
>  4b6e9a8a63 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionDirectory.java
>  131318b5a1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
>  abad7edd9b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> 74120b6ab8 
>   
> 

Re: Review Request 61856: Auto fix service config mapping issue discovered during ambari-upgrade

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61856/#review183641
---




ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
Lines 160 (patched)


Why doesn't this check is service exists or not?



ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
Lines 168 (patched)


Is this within a transaction boundary?



ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
Lines 48 (patched)


Make sure to run checkstyle on this.


- Sid Wagle


On Aug. 23, 2017, 6:49 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61856/
> ---
> 
> (Updated Aug. 23, 2017, 6:49 p.m.)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.
> 
> 
> Bugs: AMBARI-21795
> https://issues.apache.org/jira/browse/AMBARI-21795
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If user has deleted services, after upgrade db consistency check will throw 
> warnings about unmapped configs.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  40049c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigEntity.java
>  34f3034 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
> 319bf53 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
>  7fc392d 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 9cdf8e8 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql d616e77 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0530651 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 1d16eb6 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql da23e95 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e410555 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
>  44b5d91 
> 
> 
> Diff: https://reviews.apache.org/r/61856/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 61853: Incorrect 'End Time' and 'Duration' shown in Upgrade History

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61853/#review183639
---


Ship it!




Ship It!

- Nate Cole


On Aug. 23, 2017, 1:17 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61853/
> ---
> 
> (Updated Aug. 23, 2017, 1:17 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
> 
> 
> Bugs: AMBARI-21791
> https://issues.apache.org/jira/browse/AMBARI-21791
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *STR*
> # Deploy BI-4.2.0 cluster with Ambari 2.2.0
> # Upgrade Ambari to 2.5.2.0-274 and perform post upgrade tasks like delete 
> unsupported services (Solr, Titan)
> # Perform EU to HDP-2.6.2.0-194 - while EU was in progress, I paused the 
> upgrade few times; Later the EU was finalized
> # Observe Upgrade History page
> 
> *Result*
> The End time and duration of upgrade are incorrect.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java
>  b0550c0138 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  ee1800765d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
>  621b6b4ed8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  1ac28641d1 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
>  f0469ebbcf 
> 
> 
> Diff: https://reviews.apache.org/r/61853/diff/1/
> 
> 
> Testing
> ---
> 
> Tests run: 4834, Failures: 0, Errors: 0, Skipped: 35
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 61856: Auto fix service config mapping issue discovered during ambari-upgrade

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61856/
---

Review request for Ambari, Myroslav Papirkovskyy and Sid Wagle.


Bugs: AMBARI-21795
https://issues.apache.org/jira/browse/AMBARI-21795


Repository: ambari


Description
---

If user has deleted services, after upgrade db consistency check will throw 
warnings about unmapped configs.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 40049c0 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigEntity.java
 34f3034 
  ambari-server/src/main/java/org/apache/ambari/server/state/ServiceImpl.java 
319bf53 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog260.java
 7fc392d 
  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 9cdf8e8 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql d616e77 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0530651 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 1d16eb6 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql da23e95 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql e410555 
  ambari-server/src/main/resources/stacks/HDP/2.3.ECS/repos/repoinfo.xml 
329539e 
  ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/repos/repoinfo.xml 
ad79215 
  ambari-server/src/main/resources/stacks/HDP/2.6/repos/repoinfo.xml ff132aa 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog260Test.java
 44b5d91 


Diff: https://reviews.apache.org/r/61856/diff/1/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---

(Updated Aug. 23, 2017, 5:50 p.m.)


Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sebastian 
Toader.


Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784


Repository: ambari


Description
---

Ambari server log is constantly populated with 'Config inconsistency exists: 
unknown configType=solr-site' even after Solr deletion due to existence of 
config group.

{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config 
inconsistency exists: unknown configType=solr-site
{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
 200cf27 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 3fb45ab 
  
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
 906d948 
  
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
 9058a63 
  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
7282485 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 5214f04 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
 eec379e 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java 
c70dcc6 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
 21ce2c2 
  ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java 
1ad8cef 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
 e8e7206 
  
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
 fc1f334 


Diff: https://reviews.apache.org/r/61833/diff/3/


Testing (updated)
---

Manually verfied with a 2.5.2 cluster.
Unit test added.
All modified unit test pass.


Thanks,

Sid Wagle

Review Request 61853: Incorrect 'End Time' and 'Duration' shown in Upgrade History

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61853/
---

Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.


Bugs: AMBARI-21791
https://issues.apache.org/jira/browse/AMBARI-21791


Repository: ambari


Description
---

*STR*
# Deploy BI-4.2.0 cluster with Ambari 2.2.0
# Upgrade Ambari to 2.5.2.0-274 and perform post upgrade tasks like delete 
unsupported services (Solr, Titan)
# Perform EU to HDP-2.6.2.0-194 - while EU was in progress, I paused the 
upgrade few times; Later the EU was finalized
# Observe Upgrade History page

*Result*
The End time and duration of upgrade are incorrect.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessor.java
 b0550c0138 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
 ee1800765d 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
 621b6b4ed8 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 1ac28641d1 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
 f0469ebbcf 


Diff: https://reviews.apache.org/r/61853/diff/1/


Testing
---

Tests run: 4834, Failures: 0, Errors: 0, Skipped: 35


Thanks,

Jonathan Hurley

Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---

(Updated Aug. 23, 2017, 5:10 p.m.)


Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sebastian 
Toader.


Changes
---

Minor nit corrected.


Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784


Repository: ambari


Description
---

Ambari server log is constantly populated with 'Config inconsistency exists: 
unknown configType=solr-site' even after Solr deletion due to existence of 
config group.

{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config 
inconsistency exists: unknown configType=solr-site
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
 200cf27 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 3fb45ab 
  
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
 906d948 
  
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
 9058a63 
  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
7282485 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 5214f04 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
 eec379e 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java 
c70dcc6 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
 21ce2c2 
  ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java 
1ad8cef 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
 e8e7206 
  
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
 fc1f334 


Diff: https://reviews.apache.org/r/61833/diff/3/

Changes: https://reviews.apache.org/r/61833/diff/2-3/


Testing
---

Manually verfied with a 2.5.2 cluster.
Unit test added.


Thanks,

Sid Wagle

Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/
---

(Updated Aug. 23, 2017, 5:08 p.m.)


Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sebastian 
Toader.


Changes
---

- Addressed cahnge suggested by Sebastian Toader.
- Removed @Transactional annotation on deleteService(Service service), because 
it is calling transactional methods which would just bind to the enclosing 
transaction from the callers.


Bugs: AMBARI-21784
https://issues.apache.org/jira/browse/AMBARI-21784


Repository: ambari


Description
---

Ambari server log is constantly populated with 'Config inconsistency exists: 
unknown configType=solr-site' even after Solr deletion due to existence of 
config group.

{code}
22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - Config 
inconsistency exists: unknown configType=solr-site
{code}


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProvider.java
 200cf27 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 3fb45ab 
  
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupFactory.java
 906d948 
  
ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
 9058a63 
  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
7282485 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 5214f04 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ConfigGroupResourceProviderTest.java
 eec379e 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigGroupTest.java 
c70dcc6 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
 21ce2c2 
  ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java 
1ad8cef 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
 e8e7206 
  
ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
 fc1f334 


Diff: https://reviews.apache.org/r/61833/diff/2/

Changes: https://reviews.apache.org/r/61833/diff/1-2/


Testing
---

Manually verfied with a 2.5.2 cluster.
Unit test added.


Thanks,

Sid Wagle

Re: Review Request 61813: Storm Service should not have ranger 'ranger-admin-site' and 'ranger-ugsync-site' dependencies

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61813/#review183622
---



This dependency was added in 
https://issues.apache.org/jira/browse/AMBARI-14202. Removing this dependency 
will cause issues when WE is enabled. We need to improve our overall restart 
required logic but that is a larger improvement to be done. Unfortunatel, for 
now though we will need to keep these dependencies.

- Jayush Luniya


On Aug. 22, 2017, 1:42 p.m., Jay SenSharma wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61813/
> ---
> 
> (Updated Aug. 22, 2017, 1:42 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Juanjo  Marron, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-21779
> https://issues.apache.org/jira/browse/AMBARI-21779
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Storm Service should not include the following ""
> 
> ranger-ugsync-site
> ranger-admin-site
> 
> Else making any configuration change in Ranger service (for example: "Ranger 
> User Info") will ask Storm service to be restarted.
> 
> Looks like as part of JIRA "AMBARI-16250"  this dependency was added in bulk 
> by mistake.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/STORM/0.9.1/metainfo.xml 
> 0c62c1a 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/metainfo.xml 
> 1bc23e4 
>   ambari-server/src/main/resources/common-services/STORM/1.0.1/metainfo.xml 
> 084eac2 
>   ambari-server/src/main/resources/common-services/STORM/1.1.0/metainfo.xml 
> 94f5ca3 
> 
> 
> Diff: https://reviews.apache.org/r/61813/diff/1/
> 
> 
> Testing
> ---
> 
> .
> 
> 
> Thanks,
> 
> Jay SenSharma
> 
>

Review Request 61850: Log Search: hdfs audit log fields are not mapped

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61850/
---

Review request for Ambari, Miklos Gergely and Sebastian Toader.


Bugs: AMBARI-21790
https://issues.apache.org/jira/browse/AMBARI-21790


Repository: ambari


Description
---

Serialized json field name was wrong, that caused map existing key fields to 
new fields does not work.


Diffs
-

  
ambari-logsearch/ambari-logsearch-config-zookeeper/src/main/java/org/apache/ambari/logsearch/config/zookeeper/model/inputconfig/impl/MapFieldNameDescriptorImpl.java
 e1b71e6 


Diff: https://reviews.apache.org/r/61850/diff/1/


Testing
---

done, manually


Thanks,

Oliver Szabo

Re: Review Request 61707: Pre-configure services when Kerberos is enabled to reduce number of core service restarts when services are added

[Robert Levas]

> On Aug. 18, 2017, 9:44 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> > Lines 1312 (patched)
> > 
> >
> > Is it OK if the above if-statement executes and sets the 
> > `kerberosDescription` from 
> > `getKerberosDescriptorPreConfigurationFileLocation` so that this if-check 
> > is always skipped?
> > 
> > If it's OK to skip getting the descriptor if the pre-config descriptor 
> > is fetched, then drop this.
> 
> Robert Levas wrote:
> The logic here is to build the Kerberos descriptor by setting lower 
> priority values before the higher ones.  Where as the lower priority items 
> may be overwritten by highter priority ones.  In this case, the preconfigure 
> items may be overwritten by the stack defaults, which may be overwritten by 
> user-supplied values.  
> 
> Each of the levels of data are optional, so properly handling is needed 
> to avoid NPEs.  The flow first builds a base KerberosDescriptor from the 
> preconfigure data.  If it does not exist, the base KerberosDescriptor will be 
> the stack-level KerberosDescriptor.  If that does not exist, the base 
> KerberosDescriptor will be the the user-supplied KerberosDescriptor.  If a 
> user-suppled KerberosDescriptor is not provided, than and empty 
> KerberosDescriptor should be returned.  Meanwhile at any step; if a 
> KerberosDescriptor exists, it will be updated with data from the relevant 
> level-specific KerberosDescriptor.  
> 
> Workflow:
> 
> - Get the preconfigure data and create the composite KerberosDescriptor; 
> else if there is no preconfigure data, the composite KerberosDescriptor is 
> null
> - Get the stack-level data and create the stack-level KerberosDescriptor; 
> else if there is no stack-level data, the stack-level KerberosDescriptor is 
> null
> - If the composite KerberosDescriptor is null, use the stack-level data 
> as the composite; else if the composite KerberosDescriptor exists and the 
> stack-level KerberosDescritor exists, then update the composite 
> KerberosDescriptor with the stack-level KerberosDescriptor.
> - Get the user-supplied data and create the user-level 
> KerberosDescriptor; else if there is no user-supplied data, the user-level 
> KerberosDescriptor is null
> - If the composite KerberosDescriptor is null, use the user-level data as 
> the composite; else if the composite KerberosDescriptor exists and the 
> user-level KerberosDescritor exists, then update the composite 
> KerberosDescriptor with the user-level KerberosDescriptor.
> - If the composite KerberosDescriptor is null, return an empty 
> KerberosDescriptor; else return the composite KerberosDescriptor.
> 
> 
> Unless I am missing something, the code basically implements this.

oops.. I jumped the gun on the user-suppled KerberorDescriptor.  I got ahead of 
myself.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61707/#review183290
---


On Aug. 17, 2017, 8:51 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61707/
> ---
> 
> (Updated Aug. 17, 2017, 8:51 a.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21602
> https://issues.apache.org/jira/browse/AMBARI-21602
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pre-configure (certain) services when Kerberos is enabled to reduce number of 
> core service restarts when services are added.  
> 
> While processing the Kerberos descriptor, include services marked to be 
> _pre-configured_.  When a tagged service is encountered, process it weather 
> it is installed or not. However if it is not installed, only apply 
> configuration changes for existing configuration types.  This will set at 
> least the core-site changes related to proxyuser and auth-to-local rules 
> properties. By doing this, if a tagged service is later installed, the 
> settings will already be in place in the existing service configs and thus 
> the existing services will not need to be restarted. 
> 
> Caveats:
> - Default values for the uninstalled, tagged, services will be assumed 
> - The stack advisor will be used to suggest locations of components - used to 
> build the clusterHostInfo structure that may be used to derive property 
> values. 
> 
> Note: This processing is to occur when Kerberos is enabled.
> 
> 
> Diffs
> -
> 
>   
> 

Re: Review Request 61707: Pre-configure services when Kerberos is enabled to reduce number of core service restarts when services are added

[Robert Levas]

> On Aug. 18, 2017, 9:44 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> > Lines 1312 (patched)
> > 
> >
> > Is it OK if the above if-statement executes and sets the 
> > `kerberosDescription` from 
> > `getKerberosDescriptorPreConfigurationFileLocation` so that this if-check 
> > is always skipped?
> > 
> > If it's OK to skip getting the descriptor if the pre-config descriptor 
> > is fetched, then drop this.

The logic here is to build the Kerberos descriptor by setting lower priority 
values before the higher ones.  Where as the lower priority items may be 
overwritten by highter priority ones.  In this case, the preconfigure items may 
be overwritten by the stack defaults, which may be overwritten by user-supplied 
values.  

Each of the levels of data are optional, so properly handling is needed to 
avoid NPEs.  The flow first builds a base KerberosDescriptor from the 
preconfigure data.  If it does not exist, the base KerberosDescriptor will be 
the stack-level KerberosDescriptor.  If that does not exist, the base 
KerberosDescriptor will be the the user-supplied KerberosDescriptor.  If a 
user-suppled KerberosDescriptor is not provided, than and empty 
KerberosDescriptor should be returned.  Meanwhile at any step; if a 
KerberosDescriptor exists, it will be updated with data from the relevant 
level-specific KerberosDescriptor.  

Workflow:

- Get the preconfigure data and create the composite KerberosDescriptor; else 
if there is no preconfigure data, the composite KerberosDescriptor is null
- Get the stack-level data and create the stack-level KerberosDescriptor; else 
if there is no stack-level data, the stack-level KerberosDescriptor is null
- If the composite KerberosDescriptor is null, use the stack-level data as the 
composite; else if the composite KerberosDescriptor exists and the stack-level 
KerberosDescritor exists, then update the composite KerberosDescriptor with the 
stack-level KerberosDescriptor.
- Get the user-supplied data and create the user-level KerberosDescriptor; else 
if there is no user-supplied data, the user-level KerberosDescriptor is null
- If the composite KerberosDescriptor is null, use the user-level data as the 
composite; else if the composite KerberosDescriptor exists and the user-level 
KerberosDescritor exists, then update the composite KerberosDescriptor with the 
user-level KerberosDescriptor.
- If the composite KerberosDescriptor is null, return an empty 
KerberosDescriptor; else return the composite KerberosDescriptor.


Unless I am missing something, the code basically implements this.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61707/#review183290
---


On Aug. 17, 2017, 8:51 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61707/
> ---
> 
> (Updated Aug. 17, 2017, 8:51 a.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Jonathan Hurley, Laszlo Puskas, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-21602
> https://issues.apache.org/jira/browse/AMBARI-21602
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Pre-configure (certain) services when Kerberos is enabled to reduce number of 
> core service restarts when services are added.  
> 
> While processing the Kerberos descriptor, include services marked to be 
> _pre-configured_.  When a tagged service is encountered, process it weather 
> it is installed or not. However if it is not installed, only apply 
> configuration changes for existing configuration types.  This will set at 
> least the core-site changes related to proxyuser and auth-to-local rules 
> properties. By doing this, if a tagged service is later installed, the 
> settings will already be in place in the existing service configs and thus 
> the existing services will not need to be restarted. 
> 
> Caveats:
> - Default values for the uninstalled, tagged, services will be assumed 
> - The stack advisor will be used to suggest locations of components - used to 
> build the clusterHostInfo structure that may be used to derive property 
> values. 
> 
> Note: This processing is to occur when Kerberos is enabled.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
>  eb97ee376d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorRequest.java
>  7ba1b1887d 
>   
> 

Review Request 61847: Component version is not reported

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61847/
---

Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, and Sumit 
Mohanty.


Bugs: AMBARI-21789
https://issues.apache.org/jira/browse/AMBARI-21789


Repository: ambari


Description
---


Diffs
-

  ambari-common/src/main/python/resource_management/libraries/script/script.py 
269fc72 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 80c3623 


Diff: https://reviews.apache.org/r/61847/diff/1/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 61205: Users randomly getting "HDFS020 Could not write file" exceptions while running query from Hive View

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61205/#review183587
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On Aug. 21, 2017, 8:28 a.m., venkat sairam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61205/
> ---
> 
> (Updated Aug. 21, 2017, 8:28 a.m.)
> 
> 
> Review request for Ambari, Gaurav Nagar, Nitiraj Rathore, Pallav 
> Kulshreshtha, and Rohit Choudhary.
> 
> 
> Bugs: AMBARI-21569
> https://issues.apache.org/jira/browse/AMBARI-21569
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ServiceFormattedException:100 - 
> org.apache.ambari.view.utils.hdfs.HdfsApiException: HDFS020 Could not write 
> file /user/user/hive/jobs/hive-job-5290-2017-03-22_02-35/logs
> 
> org.apache.ambari.view.utils.hdfs.HdfsApiException: HDFS020 Could not write 
> file /user/user/hive/jobs/hive-job-5290-2017-03-22_02-35/logs
> at 
> org.apache.ambari.view.utils.hdfs.HdfsUtil.putStringToFile(HdfsUtil.java:51)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobControllerImpl.updateOperationLogs(JobControllerImpl.java:202)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobControllerImpl.update(JobControllerImpl.java:160)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobResourceManager.read(JobResourceManager.java:79)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobResourceManager.readController(JobResourceManager.java:95)
> at 
> org.apache.ambari.view.hive.resources.jobs.JobService.getOne(JobService.java:133)
> at sun.reflect.GeneratedMethodAccessor191.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at 
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at 
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at 
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1507)
> at 
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at 
> 

Re: Review Request 61842: Ensure topology_mappings.data has 0644 permissions

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61842/#review183583
---


Ship it!




Ship It!

- Dmytro Sen


On Aug. 23, 2017, 8:39 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61842/
> ---
> 
> (Updated Aug. 23, 2017, 8:39 a.m.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-21788
> https://issues.apache.org/jira/browse/AMBARI-21788
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In restricted umask environments, the topology_metadata.data file is created
> but the permission bits are stripped off and that results in jobs running with
> the default rack. A workaround is available to manually change permissions for
> this file on each host, but it is not optimal.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py
>  548f051 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py
>  4f62ace 
> 
> 
> Diff: https://reviews.apache.org/r/61842/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 61842: Ensure topology_mappings.data has 0644 permissions

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61842/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-21788
https://issues.apache.org/jira/browse/AMBARI-21788


Repository: ambari


Description
---

In restricted umask environments, the topology_metadata.data file is created
but the permission bits are stripped off and that results in jobs running with
the default rack. A workaround is available to manually change permissions for
this file on each host, but it is not optimal.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py
 548f051 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py
 4f62ace 


Diff: https://reviews.apache.org/r/61842/diff/1/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 61833: Ambari server logs consistency check warning after service delete with ConfigGroup present

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61833/#review183581
---




ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
Lines 118 (patched)


Can we pass in service name to the constructor and used that instead of 
tag? Similar to how this has  been done in AMBARI-21078 ?


- Sebastian Toader


On Aug. 23, 2017, 3:51 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61833/
> ---
> 
> (Updated Aug. 23, 2017, 3:51 a.m.)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-21784
> https://issues.apache.org/jira/browse/AMBARI-21784
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari server log is constantly populated with 'Config inconsistency exists: 
> unknown configType=solr-site' even after Solr deletion due to existence of 
> config group.
> 
> {code}
> 22 Aug 2017 19:00:20,094 ERROR [ambari-hearbeat-monitor] HostImpl:1085 - 
> Config inconsistency exists: unknown configType=solr-site
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  3fb45ab 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/configgroup/ConfigGroupImpl.java
>  9058a63 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> 7282485 
>   ambari-server/src/test/java/org/apache/ambari/server/state/ServiceTest.java 
> 1ad8cef 
> 
> 
> Diff: https://reviews.apache.org/r/61833/diff/1/
> 
> 
> Testing
> ---
> 
> Manually verfied with a 2.5.2 cluster.
> Unit test added.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>