Review Request 61239: AMBARI-21611 Failed to install packages before EU on powerpc

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61239/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sid Wagle, and 
Tim Thorpe.


Bugs: AMBARI-21611
https://issues.apache.org/jira/browse/AMBARI-21611


Repository: ambari


Description
---

Error message: Repositories for os type redhat-ppc7 are not defined.
This is because:
AmbariAgent reports os type (to be stored in the Hosts table) as redhat-ppc7. 
"-ppc" is a hardcoded post fix attached to the os type in os_check.py 
"""
os_check.py (AMBARI-21054)
if _is_powerpc():
operatingSystem += '-ppc'
"""
When Ambari server backend generates the
{os, repo}
map, it uses os information from repo_versoin as the key . But later on uses os 
info from the Hosts table to query the map, thus the mismatch (redhat-ppc7 vs 
redhat7).
The repo_version table repository column however only uses redhat6/7 and does 
not distinguish ppc vs x86 ( nor it should )


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 633fe8c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
 7018245 


Diff: https://reviews.apache.org/r/61239/diff/1/


Testing
---

one IOP ppc one IOP x86, upgrade Ambari, patch with server jar, run install 
packages after registering HDP 2.6.2 stack. verify the error did not appear any 
 more for ppc, both ppc and x86 cluster were able to finish the install package 
step successfully


Thanks,

Di Li

Re: Review Request 61235: EU: Modify configs to enable custom extensions functionality

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61235/#review181750
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 28, 2017, 11:42 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61235/
> ---
> 
> (Updated July 28, 2017, 11:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-21610
> https://issues.apache.org/jira/browse/AMBARI-21610
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need EU to modify the configs that we have changed to add the path to the 
> class path.
> 
> Pre Upgrade (say for hadoop in a BI-4.2.5 cluster)
> - There is an HDFS folder /iop/ext/4.2.5/hadoop where users have put their 
> custom extension jars
> - On start of hadoop services we copy jars from HDFS to on disk location 
> /usr/iop/current/ext/hadoop
> - All class path variables include the on disk path
> 
> Post Upgrade (for HDP-2.6.2 cluster)
> - HDP clusters now have an HDFS folder /hdp/ext/2.6/hadoop where the users 
> must copy the extension jars again (some jars may be new as its an upgrade)
> - On start of hadoop services we copy jars from the above location to disk at 
> /usr/hdp/current/ext/hadoop
> - Class paths now should now have the above path
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f71e91a 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  a7fdeec 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  a3b190f 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  5532248 
> 
> 
> Diff: https://reviews.apache.org/r/61235/diff/3/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 61235: EU: Modify configs to enable custom extensions functionality

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61235/
---

(Updated July 28, 2017, 11:42 p.m.)


Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Vitalyi 
Brodetskyi.


Changes
---

Minor changes to log messages.


Bugs: AMBARI-21610
https://issues.apache.org/jira/browse/AMBARI-21610


Repository: ambari


Description
---

We need EU to modify the configs that we have changed to add the path to the 
class path.

Pre Upgrade (say for hadoop in a BI-4.2.5 cluster)
- There is an HDFS folder /iop/ext/4.2.5/hadoop where users have put their 
custom extension jars
- On start of hadoop services we copy jars from HDFS to on disk location 
/usr/iop/current/ext/hadoop
- All class path variables include the on disk path

Post Upgrade (for HDP-2.6.2 cluster)
- HDP clusters now have an HDFS folder /hdp/ext/2.6/hadoop where the users must 
copy the extension jars again (some jars may be new as its an upgrade)
- On start of hadoop services we copy jars from the above location to disk at 
/usr/hdp/current/ext/hadoop
- Class paths now should now have the above path


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f71e91a 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 a7fdeec 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 a3b190f 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5532248 


Diff: https://reviews.apache.org/r/61235/diff/3/

Changes: https://reviews.apache.org/r/61235/diff/2-3/


Testing
---

Manually verified.


Thanks,

Sid Wagle

Re: Review Request 61235: EU: Modify configs to enable custom extensions functionality

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61235/
---

(Updated July 28, 2017, 11:10 p.m.)


Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Vitalyi 
Brodetskyi.


Changes
---

Fixed newline errors.


Bugs: AMBARI-21610
https://issues.apache.org/jira/browse/AMBARI-21610


Repository: ambari


Description
---

We need EU to modify the configs that we have changed to add the path to the 
class path.

Pre Upgrade (say for hadoop in a BI-4.2.5 cluster)
- There is an HDFS folder /iop/ext/4.2.5/hadoop where users have put their 
custom extension jars
- On start of hadoop services we copy jars from HDFS to on disk location 
/usr/iop/current/ext/hadoop
- All class path variables include the on disk path

Post Upgrade (for HDP-2.6.2 cluster)
- HDP clusters now have an HDFS folder /hdp/ext/2.6/hadoop where the users must 
copy the extension jars again (some jars may be new as its an upgrade)
- On start of hadoop services we copy jars from the above location to disk at 
/usr/hdp/current/ext/hadoop
- Class paths now should now have the above path


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f71e91a 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 a7fdeec 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 a3b190f 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5532248 


Diff: https://reviews.apache.org/r/61235/diff/2/

Changes: https://reviews.apache.org/r/61235/diff/1-2/


Testing
---

Manually verified.


Thanks,

Sid Wagle

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Madhuvanthi Radhakrishnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181741
---



We also need to add ServiceGroupResourceProvider initialization in 
DefaultProviderModule

- Madhuvanthi Radhakrishnan


On July 28, 2017, 10 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 10 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
>  44d50731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  d792717 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  12e4a08 
>   
> ambari-server/src/main/java/or

Review Request 61235: EU: Modify configs to enable custom extensions functionality

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61235/
---

Review request for Ambari, Alejandro Fernandez, Sumit Mohanty, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-21610
https://issues.apache.org/jira/browse/AMBARI-21610


Repository: ambari


Description
---

We need EU to modify the configs that we have changed to add the path to the 
class path.

Pre Upgrade (say for hadoop in a BI-4.2.5 cluster)
- There is an HDFS folder /iop/ext/4.2.5/hadoop where users have put their 
custom extension jars
- On start of hadoop services we copy jars from HDFS to on disk location 
/usr/iop/current/ext/hadoop
- All class path variables include the on disk path

Post Upgrade (for HDP-2.6.2 cluster)
- HDP clusters now have an HDFS folder /hdp/ext/2.6/hadoop where the users must 
copy the extension jars again (some jars may be new as its an upgrade)
- On start of hadoop services we copy jars from the above location to disk at 
/usr/hdp/current/ext/hadoop
- Class paths now should now have the above path


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f71e91a 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 a7fdeec 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 a3b190f 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 5532248 


Diff: https://reviews.apache.org/r/61235/diff/1/


Testing
---

Manually verified.


Thanks,

Sid Wagle

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Madhuvanthi Radhakrishnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181740
---


Ship it!




- Madhuvanthi Radhakrishnan


On July 28, 2017, 10 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 10 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
>  44d50731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  d792717 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  12e4a08 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  28c0d10 
>   
> amba

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Madhuvanthi Radhakrishnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181739
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceGroupResourceProvider.java
Lines 230 (patched)


You can also choose to send a response back informing which key was deleted 
or any other information using
DeleteStatusMetaData, the invoke method and return a DeleteStatusMetaData 
object instead of RequestStatusResponse and then you can do an 
deleteStatusMetaData.addDeletedKey());

getRequestStatus(null, null, deleteStatusMetaData);



ambari-server/src/main/resources/key_properties.json
Lines 9 (patched)


Do we want to avoid using key_properties.json? You can directly define it 
in the ResourceProvider



ambari-server/src/main/resources/properties.json
Lines 15 (patched)


Do we want to avoid using properties.json? You can directly define it in 
the ResourceProvider


- Madhuvanthi Radhakrishnan


On July 28, 2017, 10 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 10 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/Ser

Re: Review Request 61178: AMBARI-21572. Spark configuration mismatch post express upgrade

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61178/#review181738
---


Ship it!




Ship It!

- Di Li


On July 28, 2017, 9:44 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61178/
> ---
> 
> (Updated July 28, 2017, 9:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Eric Yang, Jonathan Hurley, 
> Sumit Mohanty, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21572
> https://issues.apache.org/jira/browse/AMBARI-21572
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove unnecessary `spark.yarn.archive` and `spark.thriftserver.ui.port` 
> properties
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f71e91adcb1091074f83f1f09759968de50c82a9 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  a7fdeecd86533c70975bdc072815b3b7d5abed2b 
> 
> 
> Diff: https://reviews.apache.org/r/61178/diff/3/
> 
> 
> Testing
> ---
> 
> Tested upgrade from BI 4.2.5 to HDP 2.6 manually.  Properties are removed 
> during upgrade and no longer present in Spark configs.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 61203: AMBARI-21593 : AMS stopped after RU [AMS distributed mode with 2 collectors]

[Sid Wagle]

> On July 28, 2017, 5:17 a.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
> > Lines 137 (patched)
> > 
> >
> > So what happens if one collector only created partial structure? This 
> > situation would require restart to get otu of.
> 
> Aravindan Vijayan wrote:
> Yes. Do you suggest we go ahead and create the znode again when a 
> collector has failed after sleeping and trying for 5 times? The reason being 
> either the other collector did not create it fully or there is no other 
> collector and we have a partial znode created somehow.

Yea that makes it full proof.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61203/#review181647
---


On July 28, 2017, 4:50 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61203/
> ---
> 
> (Updated July 28, 2017, 4:50 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21593
> https://issues.apache.org/jira/browse/AMBARI-21593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> When 2 metric collectors are started up simultaneously, both of them fail to 
> start.
> 
> BUG
> There exists a race condition in the Metric Collector HA controller 
> initialization which was introduced through AMBARI-20179Link. When a helix 
> controller instance finds that the /ambari-metrics-collector znode exists but 
> a child node does not exists, it deletes the entire znode and recreates. If 
> another controller instance also initializes simultaneously, a race condition 
> can occur wherein each instance will end up cancelling the effort of the 
> other.
> 
> FIX
> Do not delete and recreate the znode. Wait and retry for a few seconds to 
> check if /ambari-metrics-collector was fully initailized.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  53e6304 
> 
> 
> Diff: https://reviews.apache.org/r/61203/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/
---

(Updated July 28, 2017, 10 p.m.)


Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.


Changes
---

Updated code based on review comments by Jayush.


Bugs: AMBARI-21594
https://issues.apache.org/jira/browse/AMBARI-21594


Repository: ambari


Description
---

**ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
be used to logically group a set of services (coming out of Management Pack 
selected).
 
**Example of a Service groups can be a:** 
- “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
default.
- “Streaming” Service Group containing Kafka, Storm, Nifi.
- “Data Science” Service Group containing Hive, Spark, Zeppelin.
- “EDW” Service Group containing Hive, Spark.

As part of this review, following has been implemented:
- SG added as sub-resource of cluster.
- GET, POST and DELETE API for SG.



*API calls and response:*

*1.* 

POST:


API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
*Body :*


[
{
  "ServiceGroupInfo" : {
"cluster_id": "2",
"service_group_name": "CORE"
  }
},
{
  "ServiceGroupInfo" : {
"cluster_id": "2",
"service_group_name": "EDW"
  }
}
]


*Response:* 201 CREATED


{
  "resources" : [
{
  "href" : 
"http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
  "ServiceGroupInfo" : {
"cluster_id" : 2,
"cluster_name" : "c1",
"service_group_id" : 804,
"service_group_name" : "CORE"
  }
},
{
  "href" : 
"http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
  "ServiceGroupInfo" : {
"cluster_id" : 2,
"cluster_name" : "c1",
"service_group_id" : 803,
"service_group_name" : "EDW"
  }
}
  ]
}


*2.*
===
GET :
===

API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/

*Body:*

[]


*Response:* 200 OK


{
  "href" : 
"http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
  "items" : [
{
  "href" : 
"http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
  "ServiceGroupInfo" : {
"cluster_name" : "c1",
"service_group_name" : "CORE"
  }
},
{
  "href" : 
"http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
  "ServiceGroupInfo" : {
"cluster_name" : "c1",
"service_group_name" : "EDW"
  }
}
  ]
}


API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE

*Body:*

[]


*Response:* 200 OK

{
  "href" : 
"http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
  "ServiceGroupInfo" : {
"cluster_id" : 2,
"cluster_name" : "c1",
"service_group_id" : 804,
"service_group_name" : "CORE"
  }
}


*3.*
===
DELETE:
===

API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE

*Body:*

[]


*Response:* 200 OK


[]


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
 f689841 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
 73963df 
  
ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
 44d50731 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 d792717 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 12e4a08 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 28c0d10 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ResourceProviderFactory.java
 3912138 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceGroupRequest.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ServiceGroupResponse.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java
 3228a7f 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ServiceGroupResourceProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/spi/Resource.java
 199450

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Swapan Shridhar]

> On July 28, 2017, 6:51 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceGroupEntityPK.java
> > Lines 43 (patched)
> > 
> >
> > service group id instead

Fixed.


> On July 28, 2017, 6:51 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
> > Lines 159 (patched)
> > 
> >
> > primary key should be service group id.

Fixed.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181709
---


On July 28, 2017, 3:48 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 3:48 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Swapan Shridhar]

> On July 28, 2017, 6:49 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceGroupEntity.java
> > Lines 57 (patched)
> > 
> >
> > Primary key in DB should be service group id and not service group name

Fixed.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181708
---


On July 28, 2017, 3:48 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 3:48 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
>  44d50731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Swapan Shridhar]

> On July 28, 2017, 6:46 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
> > Lines 201 (patched)
> > 
> >
> > Bulk update for multiple service groups shouldnt take 
> > "serviceGroupName" path param.

Removed the parameter.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181707
---


On July 28, 2017, 3:48 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 3:48 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
>  44d50731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
>  PRE-CREATIO

Re: Review Request 61178: AMBARI-21572. Spark configuration mismatch post express upgrade

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61178/
---

(Updated July 28, 2017, 11:44 p.m.)


Review request for Ambari, Alejandro Fernandez, Eric Yang, Jonathan Hurley, 
Sumit Mohanty, Sebastian Toader, and Sid Wagle.


Changes
---

Also remove spark.thriftserver.ui.port, no longer used


Bugs: AMBARI-21572
https://issues.apache.org/jira/browse/AMBARI-21572


Repository: ambari


Description (updated)
---

Remove unnecessary `spark.yarn.archive` and `spark.thriftserver.ui.port` 
properties


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f71e91adcb1091074f83f1f09759968de50c82a9 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 a7fdeecd86533c70975bdc072815b3b7d5abed2b 


Diff: https://reviews.apache.org/r/61178/diff/3/

Changes: https://reviews.apache.org/r/61178/diff/2-3/


Testing (updated)
---

Tested upgrade from BI 4.2.5 to HDP 2.6 manually.  Properties are removed 
during upgrade and no longer present in Spark configs.


Thanks,

Attila Doroszlai

Re: Review Request 61228: AMBARI-21590. Connection refused to Spark Thrift Server alert after upgrade

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61228/#review181725
---


Ship it!




Ship It!

- Sumit Mohanty


On July 28, 2017, 9:12 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61228/
> ---
> 
> (Updated July 28, 2017, 9:12 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21590
> https://issues.apache.org/jira/browse/AMBARI-21590
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The least impact option is to delete the alert that uses the custom port left 
> over from BI
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/RemoveObsoleteAlerts.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  a7fdeecd86533c70975bdc072815b3b7d5abed2b 
> 
> 
> Diff: https://reviews.apache.org/r/61228/diff/1/
> 
> 
> Testing
> ---
> 
> Manual test: upgrade from BI 4.2.5.
> Verified that only one "Spark2 Thrift Server" alert is present, and is in OK 
> state if Spark2 Thrift Server is running.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 61232: AMBARI-21609 - Unable to view HDP or HDF stack version.

[Richard Zang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61232/
---

Review request for Ambari, Jaimin Jetly and Yusaku Sako.


Bugs: AMBARI-21609
https://issues.apache.org/jira/browse/AMBARI-21609


Repository: ambari


Description
---

Fix NPE at UI installation flow with RedHat Satellite selected


Diffs
-

  ambari-web/app/mappers/repository_version_mapper.js 8ef50db 
  ambari-web/app/mappers/stack_mapper.js 4b6a6f8 


Diff: https://reviews.apache.org/r/61232/diff/1/


Testing
---

Manually tested on live cluster. All unit tests passed.
  30397 passing (22s)
  157 pending


Thanks,

Richard Zang

Review Request 61228: AMBARI-21590. Connection refused to Spark Thrift Server alert after upgrade

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61228/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sumit Mohanty, 
and Sid Wagle.


Bugs: AMBARI-21590
https://issues.apache.org/jira/browse/AMBARI-21590


Repository: ambari


Description
---

The least impact option is to delete the alert that uses the custom port left 
over from BI


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/RemoveObsoleteAlerts.java
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 a7fdeecd86533c70975bdc072815b3b7d5abed2b 


Diff: https://reviews.apache.org/r/61228/diff/1/


Testing
---

Manual test: upgrade from BI 4.2.5.
Verified that only one "Spark2 Thrift Server" alert is present, and is in OK 
state if Spark2 Thrift Server is running.


Thanks,

Attila Doroszlai

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181709
---




ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceGroupEntityPK.java
Lines 43 (patched)


service group id instead



ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
Lines 159 (patched)


primary key should be service group id.


- Jayush Luniya


On July 28, 2017, 3:48 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 3:48 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
>  44d50731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
>  PRE-

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181708
---




ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ServiceGroupEntity.java
Lines 57 (patched)


Primary key in DB should be service group id and not service group name


- Jayush Luniya


On July 28, 2017, 3:48 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 3:48 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
>  44d50731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  d792717 
>   
> ambari-

Re: Review Request 61201: AMBARI-21594. MultiEverything : Add Servicegroup as a subresource of Cluster.

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61201/#review181707
---




ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
Lines 201 (patched)


Bulk update for multiple service groups shouldnt take "serviceGroupName" 
path param.


- Jayush Luniya


On July 28, 2017, 3:48 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61201/
> ---
> 
> (Updated July 28, 2017, 3:48 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Madhuvanthi Radhakrishnan.
> 
> 
> Bugs: AMBARI-21594
> https://issues.apache.org/jira/browse/AMBARI-21594
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **ServiceGroup (SG)** : is defined as sub-resource of cluster. It would later 
> be used to logically group a set of services (coming out of Management Pack 
> selected).
>  
> **Example of a Service groups can be a:** 
> - “Core” Service Group  containing HDFS and Zookeeper. This will be formed by 
> default.
> - “Streaming” Service Group containing Kafka, Storm, Nifi.
> - “Data Science” Service Group containing Hive, Spark, Zeppelin.
> - “EDW” Service Group containing Hive, Spark.
> 
> As part of this review, following has been implemented:
> - SG added as sub-resource of cluster.
> - GET, POST and DELETE API for SG.
> 
> 
> 
> *API calls and response:*
> 
> *1.* 
> 
> POST:
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups
> *Body :*
> 
> 
> [
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "CORE"
>   }
> },
> {
>   "ServiceGroupInfo" : {
> "cluster_id": "2",
> "service_group_name": "EDW"
>   }
> }
> ]
> 
> 
> *Response:* 201 CREATED
> 
> 
> {
>   "resources" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 803,
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> *2.*
> ===
> GET :
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/";,
>   "items" : [
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "CORE"
>   }
> },
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/EDW";,
>   "ServiceGroupInfo" : {
> "cluster_name" : "c1",
> "service_group_name" : "EDW"
>   }
> }
>   ]
> }
> 
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> {
>   "href" : 
> "http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE";,
>   "ServiceGroupInfo" : {
> "cluster_id" : 2,
> "cluster_name" : "c1",
> "service_group_id" : 804,
> "service_group_name" : "CORE"
>   }
> }
> 
> 
> *3.*
> ===
> DELETE:
> ===
> 
> API: http://c6404.ambari.apache.org:8080/api/v1/clusters/c1/servicegroups/CORE
> 
> *Body:*
> 
> []
> 
> 
> *Response:* 200 OK
> 
> 
> []
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/ServiceGroupNotFoundException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ClusterResourceDefinition.java
>  f689841 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ResourceInstanceFactoryImpl.java
>  73963df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/ServiceGroupResourceDefinition.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ClusterService.java
>  44d50731 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ServiceGroupService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  d792717

Re: Review Request 61203: AMBARI-21593 : AMS stopped after RU [AMS distributed mode with 2 collectors]

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61203/#review181701
---


Ship it!




Ship It!

- Sumit Mohanty


On July 28, 2017, 4:50 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61203/
> ---
> 
> (Updated July 28, 2017, 4:50 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21593
> https://issues.apache.org/jira/browse/AMBARI-21593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> When 2 metric collectors are started up simultaneously, both of them fail to 
> start.
> 
> BUG
> There exists a race condition in the Metric Collector HA controller 
> initialization which was introduced through AMBARI-20179Link. When a helix 
> controller instance finds that the /ambari-metrics-collector znode exists but 
> a child node does not exists, it deletes the entire znode and recreates. If 
> another controller instance also initializes simultaneously, a race condition 
> can occur wherein each instance will end up cancelling the effort of the 
> other.
> 
> FIX
> Do not delete and recreate the znode. Wait and retry for a few seconds to 
> check if /ambari-metrics-collector was fully initailized.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  53e6304 
> 
> 
> Diff: https://reviews.apache.org/r/61203/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 61203: AMBARI-21593 : AMS stopped after RU [AMS distributed mode with 2 collectors]

[Aravindan Vijayan]

> On July 28, 2017, 5:02 a.m., Sumit Mohanty wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
> > Line 126 (original), 127 (patched)
> > 
> >
> > What was the reason, both instances failed?
> 
> Aravindan Vijayan wrote:
> It is because of the code which deleted and recreated the znode whenever 
> a sub path is not found (i.e ZkNoNodeException is thrown from ZkHelixAdmin)
> 
> Let's say collectors A & B start this at the same time. 
> 
> A : Check parent /ambari-metrics-cluster. Not found. Create parent 
> /ambari-metrics-cluster
> B : Check parent /ambari-metrics-cluster. Found. So return true.
> B : Try to check child C2. Not yet created by A. ZkNoNodeException thrown.
> B : Catch exception. Delete the entire znode.
> A : Try to create a child node. Someone deleted the top level znode 
> itself. ZkNoNodeException thrown.
> A : Catch exception. Try to Delete the entire znode.
> A : Deleted children C1, C3
> B : Created /ambari-metrics-cluster and children nodes C1, C2, C3.
> A : Deleted child C2. 
> B : Trying to delete root node. Failed since directory not empty 
> > FAILED START
> B : Finished creating /ambari-metrics-cluster.
> A : Access C2. Not found. --> FAILED START.

Small mistake. Correct flow.


A : Check parent /ambari-metrics-cluster. Not found. Create parent 
/ambari-metrics-cluster
B : Check parent /ambari-metrics-cluster. Found. So return true.
B : Try to check child C2. Not yet created by A. ZkNoNodeException thrown.
B : Catch exception. Delete the entire znode.
A : Try to create a child node. Someone deleted the top level znode itself. 
ZkNoNodeException thrown.
A : Catch exception. Try to Delete the entire znode.
A : Deleted children C1, C3
B : Created /ambari-metrics-cluster and children nodes C1, C2, C3.
A : Deleted child C2. 
A : Trying to delete root node. Failed since directory not empty (C1 and C3 are 
there) > FAILED START
B : Finished creating /ambari-metrics-cluster.
B : Access C2. Not found. --> FAILED START.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61203/#review181646
---


On July 28, 2017, 4:50 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61203/
> ---
> 
> (Updated July 28, 2017, 4:50 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21593
> https://issues.apache.org/jira/browse/AMBARI-21593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> When 2 metric collectors are started up simultaneously, both of them fail to 
> start.
> 
> BUG
> There exists a race condition in the Metric Collector HA controller 
> initialization which was introduced through AMBARI-20179Link. When a helix 
> controller instance finds that the /ambari-metrics-collector znode exists but 
> a child node does not exists, it deletes the entire znode and recreates. If 
> another controller instance also initializes simultaneously, a race condition 
> can occur wherein each instance will end up cancelling the effort of the 
> other.
> 
> FIX
> Do not delete and recreate the znode. Wait and retry for a few seconds to 
> check if /ambari-metrics-collector was fully initailized.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  53e6304 
> 
> 
> Diff: https://reviews.apache.org/r/61203/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 61178: AMBARI-21572. Spark configuration mismatch post express upgrade

[Attila Doroszlai]

> On July 28, 2017, 8:17 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
> > Lines 266-269 (patched)
> > 
> >
> > This just deletes the property outright. The description of the bug 
> > says that if it's been converted to leave it. Is that actually the desired 
> > behavior?

"If this property isn't used by HDP it should be deleted during the stack 
upgrade." -- this is what's implemented.  Also see comment above by Alejandro: 
"Since not used, then delete the config without checking the value".


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61178/#review181696
---


On July 28, 2017, 9:22 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61178/
> ---
> 
> (Updated July 28, 2017, 9:22 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Eric Yang, Jonathan Hurley, 
> Sumit Mohanty, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21572
> https://issues.apache.org/jira/browse/AMBARI-21572
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove unnecessary `spark.yarn.archive` property (if value matches IOP 
> default one, except it's already converted to `hdfs:///hdp/...`).
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f71e91adcb1091074f83f1f09759968de50c82a9 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  9d3cc1e48e090f8a1f43cfaeb024dbeabdf4384b 
> 
> 
> Diff: https://reviews.apache.org/r/61178/diff/2/
> 
> 
> Testing
> ---
> 
> Tested upgrade from BI 4.2.5 to HDP 2.6 manually.  Property is removed during 
> upgrade and no longer present in Spark configs.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 61203: AMBARI-21593 : AMS stopped after RU [AMS distributed mode with 2 collectors]

[Aravindan Vijayan]

> On July 28, 2017, 5:02 a.m., Sumit Mohanty wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
> > Line 126 (original), 127 (patched)
> > 
> >
> > What was the reason, both instances failed?

It is because of the code which deleted and recreated the znode whenever a sub 
path is not found (i.e ZkNoNodeException is thrown from ZkHelixAdmin)

Let's say collectors A & B start this at the same time. 

A : Check parent /ambari-metrics-cluster. Not found. Create parent 
/ambari-metrics-cluster
B : Check parent /ambari-metrics-cluster. Found. So return true.
B : Try to check child C2. Not yet created by A. ZkNoNodeException thrown.
B : Catch exception. Delete the entire znode.
A : Try to create a child node. Someone deleted the top level znode itself. 
ZkNoNodeException thrown.
A : Catch exception. Try to Delete the entire znode.
A : Deleted children C1, C3
B : Created /ambari-metrics-cluster and children nodes C1, C2, C3.
A : Deleted child C2. 
B : Trying to delete root node. Failed since directory not empty > 
FAILED START
B : Finished creating /ambari-metrics-cluster.
A : Access C2. Not found. --> FAILED START.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61203/#review181646
---


On July 28, 2017, 4:50 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61203/
> ---
> 
> (Updated July 28, 2017, 4:50 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21593
> https://issues.apache.org/jira/browse/AMBARI-21593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> When 2 metric collectors are started up simultaneously, both of them fail to 
> start.
> 
> BUG
> There exists a race condition in the Metric Collector HA controller 
> initialization which was introduced through AMBARI-20179Link. When a helix 
> controller instance finds that the /ambari-metrics-collector znode exists but 
> a child node does not exists, it deletes the entire znode and recreates. If 
> another controller instance also initializes simultaneously, a race condition 
> can occur wherein each instance will end up cancelling the effort of the 
> other.
> 
> FIX
> Do not delete and recreate the znode. Wait and retry for a few seconds to 
> check if /ambari-metrics-collector was fully initailized.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  53e6304 
> 
> 
> Diff: https://reviews.apache.org/r/61203/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 61178: AMBARI-21572. Spark configuration mismatch post express upgrade

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61178/#review181696
---




ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
Lines 266-269 (patched)


This just deletes the property outright. The description of the bug says 
that if it's been converted to leave it. Is that actually the desired behavior?


- Jonathan Hurley


On July 28, 2017, 3:22 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61178/
> ---
> 
> (Updated July 28, 2017, 3:22 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Eric Yang, Jonathan Hurley, 
> Sumit Mohanty, Sebastian Toader, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21572
> https://issues.apache.org/jira/browse/AMBARI-21572
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove unnecessary `spark.yarn.archive` property (if value matches IOP 
> default one, except it's already converted to `hdfs:///hdp/...`).
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
>  f71e91adcb1091074f83f1f09759968de50c82a9 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  9d3cc1e48e090f8a1f43cfaeb024dbeabdf4384b 
> 
> 
> Diff: https://reviews.apache.org/r/61178/diff/2/
> 
> 
> Testing
> ---
> 
> Tested upgrade from BI 4.2.5 to HDP 2.6 manually.  Property is removed during 
> upgrade and no longer present in Spark configs.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 61203: AMBARI-21593 : AMS stopped after RU [AMS distributed mode with 2 collectors]

[Aravindan Vijayan]

> On July 28, 2017, 5:17 a.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
> > Lines 137 (patched)
> > 
> >
> > So what happens if one collector only created partial structure? This 
> > situation would require restart to get otu of.

Yes. Do you suggest we go ahead and create the znode again when a collector has 
failed after sleeping and trying for 5 times? The reason being either the other 
collector did not create it fully or there is no other collector and we have a 
partial znode created somehow.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61203/#review181647
---


On July 28, 2017, 4:50 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61203/
> ---
> 
> (Updated July 28, 2017, 4:50 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21593
> https://issues.apache.org/jira/browse/AMBARI-21593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> When 2 metric collectors are started up simultaneously, both of them fail to 
> start.
> 
> BUG
> There exists a race condition in the Metric Collector HA controller 
> initialization which was introduced through AMBARI-20179Link. When a helix 
> controller instance finds that the /ambari-metrics-collector znode exists but 
> a child node does not exists, it deletes the entire znode and recreates. If 
> another controller instance also initializes simultaneously, a race condition 
> can occur wherein each instance will end up cancelling the effort of the 
> other.
> 
> FIX
> Do not delete and recreate the znode. Wait and retry for a few seconds to 
> check if /ambari-metrics-collector was fully initailized.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  53e6304 
> 
> 
> Diff: https://reviews.apache.org/r/61203/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 61203: AMBARI-21593 : AMS stopped after RU [AMS distributed mode with 2 collectors]

[Aravindan Vijayan]

> On July 28, 2017, 5:02 a.m., Sumit Mohanty wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
> > Lines 147 (patched)
> > 
> >
> > Should we add a randomness to sleep (say 5 + random value between 0-5) 
> > so that both instances do not retry at the same time?

Both the collector instances should not error out and come to this step. Since 
one of them will be creating the znode undisturbed and the other one will sleep 
and retry until the first one completes the work.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61203/#review181646
---


On July 28, 2017, 4:50 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61203/
> ---
> 
> (Updated July 28, 2017, 4:50 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21593
> https://issues.apache.org/jira/browse/AMBARI-21593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM
> When 2 metric collectors are started up simultaneously, both of them fail to 
> start.
> 
> BUG
> There exists a race condition in the Metric Collector HA controller 
> initialization which was introduced through AMBARI-20179Link. When a helix 
> controller instance finds that the /ambari-metrics-collector znode exists but 
> a child node does not exists, it deletes the entire znode and recreates. If 
> another controller instance also initializes simultaneously, a race condition 
> can occur wherein each instance will end up cancelling the effort of the 
> other.
> 
> FIX
> Do not delete and recreate the znode. Wait and retry for a few seconds to 
> check if /ambari-metrics-collector was fully initailized.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  53e6304 
> 
> 
> Diff: https://reviews.apache.org/r/61203/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 61168: Hive-Service check failing in post EU validation (BI-HDP)

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61168/#review181676
---


Ship it!




Ship It!

- Robert Levas


On July 28, 2017, 11:28 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61168/
> ---
> 
> (Updated July 28, 2017, 11:28 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Jonathan 
> Hurley, Robert Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21577
> https://issues.apache.org/jira/browse/AMBARI-21577
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hive Service check is failing :- 
> 
> {code}
> HTTP/vs-iop420tofnsec-re-2.openstacklo...@example.com is not allowed to 
> impersonate ambari-qa
> {code}
> 
> Patch executes custom action to remove offending rules.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingAction.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  9d3cc1e 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  3637927 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingActionTest.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61168/diff/7/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 61168: Hive-Service check failing in post EU validation (BI-HDP)

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61168/
---

(Updated July 28, 2017, 3:28 p.m.)


Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Jonathan 
Hurley, Robert Levas, and Sumit Mohanty.


Changes
---

Thanks Rob for the review. Incorporated the extra safety checks.


Bugs: AMBARI-21577
https://issues.apache.org/jira/browse/AMBARI-21577


Repository: ambari


Description
---

Hive Service check is failing :- 

{code}
HTTP/vs-iop420tofnsec-re-2.openstacklo...@example.com is not allowed to 
impersonate ambari-qa
{code}

Patch executes custom action to remove offending rules.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingAction.java
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 9d3cc1e 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 3637927 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingActionTest.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/61168/diff/7/

Changes: https://reviews.apache.org/r/61168/diff/6-7/


Testing
---

Manually verified.


Thanks,

Sid Wagle

Re: Review Request 61214: AMBARI-21598. Spark Thrift Server stopped after upgrade due to undefined port

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61214/#review181673
---


Ship it!




Ship It!

- Di Li


On July 28, 2017, 2:50 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61214/
> ---
> 
> (Updated July 28, 2017, 2:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Eric Yang, Jonathan 
> Hurley, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21598
> https://issues.apache.org/jira/browse/AMBARI-21598
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> BI uses `spark-env/spark_thriftserver_port` for configuring Thrift Server 
> port, while HDP has `spark-hive-site-override/hive.server2.thrift.port`
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  a3b190f4a13b910050e3f58a895a4297f8d25d38 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  3637927dbcc88fd36f4bc1d0379ad1ef165e117d 
> 
> 
> Diff: https://reviews.apache.org/r/61214/diff/1/
> 
> 
> Testing
> ---
> 
> Manual test:
> 
> * Installed BI 4.2
> * Configured custom port for Spark Thrift Server
> * Upgraded to HDP 2.6
> * Verified that Thrift Server is running, and listening on the custom port
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 61214: AMBARI-21598. Spark Thrift Server stopped after upgrade due to undefined port

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61214/
---

(Updated July 28, 2017, 4:50 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, Eric Yang, Jonathan 
Hurley, Sumit Mohanty, and Sid Wagle.


Summary (updated)
-

AMBARI-21598. Spark Thrift Server stopped after upgrade due to undefined port


Bugs: AMBARI-21598
https://issues.apache.org/jira/browse/AMBARI-21598


Repository: ambari


Description
---

BI uses `spark-env/spark_thriftserver_port` for configuring Thrift Server port, 
while HDP has `spark-hive-site-override/hive.server2.thrift.port`


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 a3b190f4a13b910050e3f58a895a4297f8d25d38 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 3637927dbcc88fd36f4bc1d0379ad1ef165e117d 


Diff: https://reviews.apache.org/r/61214/diff/1/


Testing
---

Manual test:

* Installed BI 4.2
* Configured custom port for Spark Thrift Server
* Upgraded to HDP 2.6
* Verified that Thrift Server is running, and listening on the custom port


Thanks,

Attila Doroszlai

Re: Review Request 61217: AMBARI-21588 : MetadataServer start fails if Atlas service is started with custom service user.

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61217/#review181671
---


Ship it!




Ship It!

- Mugdha Varadkar


On July 28, 2017, 2:13 p.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61217/
> ---
> 
> (Updated July 28, 2017, 2:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
> Madhan Neethiraj, Mugdha Varadkar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21588
> https://issues.apache.org/jira/browse/AMBARI-21588
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> File permissons for stack_root/current/atlas-server/server/webapp/atlas is 
> root:root, when atlas user is custom, Atlas Metadata Server can not start.
> After changing permissions to the custom atlas user then Atlas was 
> successfully started.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  74b1688 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> d1d8b7f 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 539bef5 
> 
> 
> Diff: https://reviews.apache.org/r/61217/diff/1/
> 
> 
> Testing
> ---
> 
> Verified with install on CentOS 6.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

Re: Review Request 61043: AMBARI-21325: Quicklink support through Knox

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61043/#review181664
---




ambari-server/src/main/java/org/apache/ambari/server/state/quicklinks/Link.java
Lines 67 (patched)


Why not boolean?


- Balázs Bence Sári


On July 25, 2017, 12:53 a.m., Chandana Mirashi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61043/
> ---
> 
> (Updated July 25, 2017, 12:53 a.m.)
> 
> 
> Review request for Ambari, Balázs Bence Sári, Juanjo  Marron, and Sid Wagle.
> 
> 
> Bugs: AMBARI-21325
> https://issues.apache.org/jira/browse/AMBARI-21325
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add quicklink support through Knox.
> Changes:
> 1. Add new json properties knox_url, knox_path, supports_knox
>a. knox_url: template to be used for urls that are proxied through Knox 
>b. knox_path: Knox gateway path that will be added to the proxy url.
>c. supports_knox: whether link will be redirected through Knox
> 2. Add above json properties to quicklinks.json
> 3. Add HDFSUI & DATANODE,YARNUI & NODEUI, JOBHISTORYUI, HBASEUI, OOZIEUI, 
> SPARKUI services to Knox topology template.
> 4. Automate protocol and port added to Knox topology file. Based on whether 
> SSL is enabled for the services listed above, the port and protocol in 
> params_linux.py will be updated.
> 5. Update quick_view_link_view.js so that when Knox is installed and 
> support_knox is true, quicklink url follows knox url template specified in 
> the quicklinks.json for the service/component.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/quicklinks/Link.java
>  1d2e712 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  4558069 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/quicklinks/quicklinks.json
>  5568122 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/quicklinks/quicklinks.json
>  a4216e3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/quicklinks/quicklinks.json
>  81e7cbe 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/quicklinks-mapred/quicklinks.json
>  36f71b5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/quicklinks/quicklinks.json
>  0aca8e3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/KNOX/configuration/topology.xml
>  df4c1b4 
>   ambari-server/src/test/python/stacks/2.2/configs/knox_upgrade.json 1805c3b 
>   ambari-web/app/views/common/quick_view_link_view.js e39f230 
> 
> 
> Diff: https://reviews.apache.org/r/61043/diff/2/
> 
> 
> Testing
> ---
> 
> 1. ambari-server: mvn test
> 2. mvn -DskipSurefireTests -Dpython.test.mask=test_knox_gateway.py test
> 3. ambari-web: mvn test
>  21213 passing (54s)
>  128 pending
> 
> 
> Thanks,
> 
> Chandana Mirashi
> 
>

Re: Review Request 61217: AMBARI-21588 : MetadataServer start fails if Atlas service is started with custom service user.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61217/#review181668
---


Ship it!




Ship It!

- Sumit Mohanty


On July 28, 2017, 2:13 p.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61217/
> ---
> 
> (Updated July 28, 2017, 2:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
> Madhan Neethiraj, Mugdha Varadkar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21588
> https://issues.apache.org/jira/browse/AMBARI-21588
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> File permissons for stack_root/current/atlas-server/server/webapp/atlas is 
> root:root, when atlas user is custom, Atlas Metadata Server can not start.
> After changing permissions to the custom atlas user then Atlas was 
> successfully started.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  74b1688 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> d1d8b7f 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 539bef5 
> 
> 
> Diff: https://reviews.apache.org/r/61217/diff/1/
> 
> 
> Testing
> ---
> 
> Verified with install on CentOS 6.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

Review Request 61217: AMBARI-21588 : MetadataServer start fails if Atlas service is started with custom service user.

[Vishal Suvagia via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61217/
---

Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jayush Luniya, 
Madhan Neethiraj, Mugdha Varadkar, and Sumit Mohanty.


Bugs: AMBARI-21588
https://issues.apache.org/jira/browse/AMBARI-21588


Repository: ambari


Description
---

File permissons for stack_root/current/atlas-server/server/webapp/atlas is 
root:root, when atlas user is custom, Atlas Metadata Server can not start.
After changing permissions to the custom atlas user then Atlas was successfully 
started.


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 74b1688 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
d1d8b7f 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 539bef5 


Diff: https://reviews.apache.org/r/61217/diff/1/


Testing
---

Verified with install on CentOS 6.


Thanks,

Vishal Suvagia

Re: Review Request 61214: Spark Thrift Server stopped after upgrade due to undefined port

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61214/#review181663
---


Ship it!




Ship It!

- Jonathan Hurley


On July 28, 2017, 9:09 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61214/
> ---
> 
> (Updated July 28, 2017, 9:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Eric Yang, Jonathan 
> Hurley, Sumit Mohanty, and Sid Wagle.
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> BI uses `spark-env/spark_thriftserver_port` for configuring Thrift Server 
> port, while HDP has `spark-hive-site-override/hive.server2.thrift.port`
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
>  a3b190f4a13b910050e3f58a895a4297f8d25d38 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  3637927dbcc88fd36f4bc1d0379ad1ef165e117d 
> 
> 
> Diff: https://reviews.apache.org/r/61214/diff/1/
> 
> 
> Testing
> ---
> 
> Manual test:
> 
> * Installed BI 4.2
> * Configured custom port for Spark Thrift Server
> * Upgraded to HDP 2.6
> * Verified that Thrift Server is running, and listening on the custom port
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 61214: Spark Thrift Server stopped after upgrade due to undefined port

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61214/
---

Review request for Ambari, Alejandro Fernandez, Di Li, Eric Yang, Jonathan 
Hurley, Sumit Mohanty, and Sid Wagle.


Repository: ambari


Description
---

BI uses `spark-env/spark_thriftserver_port` for configuring Thrift Server port, 
while HDP has `spark-hive-site-override/hive.server2.thrift.port`


Diffs
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
 a3b190f4a13b910050e3f58a895a4297f8d25d38 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 3637927dbcc88fd36f4bc1d0379ad1ef165e117d 


Diff: https://reviews.apache.org/r/61214/diff/1/


Testing
---

Manual test:

* Installed BI 4.2
* Configured custom port for Spark Thrift Server
* Upgraded to HDP 2.6
* Verified that Thrift Server is running, and listening on the custom port


Thanks,

Attila Doroszlai

Re: Review Request 61206: Ambari Server Setup Code Fails Because of Missing Stack Upgrade

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61206/#review181658
---


Ship it!




Ship It!

- Nate Cole


On July 28, 2017, 5:01 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61206/
> ---
> 
> (Updated July 28, 2017, 5:01 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-21595
> https://issues.apache.org/jira/browse/AMBARI-21595
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On {{branch-feature-AMBARI-21450}}, there is a Python failure on 
> ambari-server setup:
> 
> {code}
> Using python  /usr/bin/python
> Setup ambari-server
> Traceback (most recent call last):
> File "/usr/sbin/ambari-server.py", line 40, in 
> from ambari_server.serverUpgrade import upgrade, upgrade_stack, set_current
> ImportError: cannot import name upgrade_stack
> {code}
> 
> This is most likely a cherry-pick issue from trunk where all "upgrade stack" 
> code was removed. I think that we should also remove it from here as well 
> since this will cause problems with PATCH repositories anyway.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari-server.py b25cd16d49 
>   ambari-server/src/main/python/ambari_server/setupActions.py 926db98c0d 
>   ambari-server/src/test/python/TestAmbariServer.py fb0bb70b92 
>   ambari-server/src/test/python/TestServerUpgrade.py 9235ff9f19 
> 
> 
> Diff: https://reviews.apache.org/r/61206/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 61209: Log Search should stop using deprecated API for solr

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61209/
---

Review request for Ambari, Oliver Szabo and Robert Nettleton.


Bugs: AMBARI-21596
https://issues.apache.org/jira/browse/AMBARI-21596


Repository: ambari


Description
---

Since we've switched to solrj 6.6.0 some API calls we are using are deprecated 
now. Let's use non-deprecated API instead.

Also suppressed some warnings and fixed the formatting of the pom file.


Diffs
-

  ambari-logsearch/ambari-logsearch-server/pom.xml ebca2d5 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/SolrAuditAliasConfigurer.java
 1eca94b 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/configurer/SolrCollectionConfigurer.java
 225f5a3 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java
 71f9f29 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/AbstractSolrConfigHandler.java
 6f3d8ca 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/CreateCollectionHandler.java
 b6e9def 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/handler/ReloadCollectionHandler.java
 52f3366 
  
ambari-logsearch/ambari-logsearch-server/src/main/java/org/apache/ambari/logsearch/solr/ResponseDataGenerator.java
 74e9c11 


Diff: https://reviews.apache.org/r/61209/diff/1/


Testing
---

Tested on docker, still works :)


Thanks,

Miklos Gergely

Re: Review Request 61206: Ambari Server Setup Code Fails Because of Missing Stack Upgrade

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61206/#review181657
---


Ship it!




Ship It!

- Jonathan Hurley


On July 28, 2017, 5:01 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61206/
> ---
> 
> (Updated July 28, 2017, 5:01 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-21595
> https://issues.apache.org/jira/browse/AMBARI-21595
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On {{branch-feature-AMBARI-21450}}, there is a Python failure on 
> ambari-server setup:
> 
> {code}
> Using python  /usr/bin/python
> Setup ambari-server
> Traceback (most recent call last):
> File "/usr/sbin/ambari-server.py", line 40, in 
> from ambari_server.serverUpgrade import upgrade, upgrade_stack, set_current
> ImportError: cannot import name upgrade_stack
> {code}
> 
> This is most likely a cherry-pick issue from trunk where all "upgrade stack" 
> code was removed. I think that we should also remove it from here as well 
> since this will cause problems with PATCH repositories anyway.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari-server.py b25cd16d49 
>   ambari-server/src/main/python/ambari_server/setupActions.py 926db98c0d 
>   ambari-server/src/test/python/TestAmbariServer.py fb0bb70b92 
>   ambari-server/src/test/python/TestServerUpgrade.py 9235ff9f19 
> 
> 
> Diff: https://reviews.apache.org/r/61206/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 61168: Hive-Service check failing in post EU validation (BI-HDP)

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61168/#review181653
---


Ship it!




Thanks for making the changes. I think this will cover all of the cases where 
auth-to-local rules may be incorrect. 

My comments are just comments, so feel feel to ignore.  I tend to be paranoid 
about nulls. :)


ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingAction.java
Lines 67 (patched)


I would be really surprised if a configuration type has multiple properties 
that were auth-to-local rule sets as would you. :)  But I would have been a 
been a bit more cautions and made this a `Map>`.  

You are probably good to go though.



ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingAction.java
Lines 105 (patched)


It is most-likely that the property value will not be `null`, but I tend to 
be a bit more paranoid...

Maybe the following would be better, but feel free to ignore since the 
likelyhood that the value is null is really small...

```
String authToLocalRules = properties.get(property);
if(!StringUtils.isEmpty(authToLocalRules)) {
  authToLocalRules = authToLocalRules.replaceAll(SPNEGO_PRINC_PATTERN, "");
  ...
}


- Robert Levas


On July 28, 2017, 12:16 a.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61168/
> ---
> 
> (Updated July 28, 2017, 12:16 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Jonathan 
> Hurley, Robert Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-21577
> https://issues.apache.org/jira/browse/AMBARI-21577
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hive Service check is failing :- 
> 
> {code}
> HTTP/vs-iop420tofnsec-re-2.openstacklo...@example.com is not allowed to 
> impersonate ambari-qa
> {code}
> 
> Patch executes custom action to remove offending rules.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingAction.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  684acfa 
>   
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
>  484e459 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixAuthToLocalMappingActionTest.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61168/diff/6/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Review Request 61206: Ambari Server Setup Code Fails Because of Missing Stack Upgrade

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61206/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-21595
https://issues.apache.org/jira/browse/AMBARI-21595


Repository: ambari


Description
---

On {{branch-feature-AMBARI-21450}}, there is a Python failure on ambari-server 
setup:

{code}
Using python  /usr/bin/python
Setup ambari-server
Traceback (most recent call last):
File "/usr/sbin/ambari-server.py", line 40, in 
from ambari_server.serverUpgrade import upgrade, upgrade_stack, set_current
ImportError: cannot import name upgrade_stack
{code}

This is most likely a cherry-pick issue from trunk where all "upgrade stack" 
code was removed. I think that we should also remove it from here as well since 
this will cause problems with PATCH repositories anyway.


Diffs
-

  ambari-server/src/main/python/ambari-server.py b25cd16d49 
  ambari-server/src/main/python/ambari_server/setupActions.py 926db98c0d 
  ambari-server/src/test/python/TestAmbariServer.py fb0bb70b92 
  ambari-server/src/test/python/TestServerUpgrade.py 9235ff9f19 


Diff: https://reviews.apache.org/r/61206/diff/1/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 61205: Users randomly getting "HDFS020 Could not write file" exceptions while running query from Hive View

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61205/#review181651
---




contrib/views/hive-next/src/main/resources/view.log4j.properties
Line 28 (original), 28 (patched)


This line will create problem in log4j. Please correct.



contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java
Lines 50 (patched)


Please use close stream in finally.


- Nitiraj Rathore


On July 28, 2017, 6:39 a.m., venkat sairam wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61205/
> ---
> 
> (Updated July 28, 2017, 6:39 a.m.)
> 
> 
> Review request for Ambari, Gaurav Nagar, Nitiraj Rathore, Pallav 
> Kulshreshtha, and Rohit Choudhary.
> 
> 
> Bugs: AMBARI-21569
> https://issues.apache.org/jira/browse/AMBARI-21569
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ServiceFormattedException:100 - 
> org.apache.ambari.view.utils.hdfs.HdfsApiException: HDFS020 Could not write 
> file /user/user/hive/jobs/hive-job-5290-2017-03-22_02-35/logs
> 
> org.apache.ambari.view.utils.hdfs.HdfsApiException: HDFS020 Could not write 
> file /user/user/hive/jobs/hive-job-5290-2017-03-22_02-35/logs
> at 
> org.apache.ambari.view.utils.hdfs.HdfsUtil.putStringToFile(HdfsUtil.java:51)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobControllerImpl.updateOperationLogs(JobControllerImpl.java:202)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobControllerImpl.update(JobControllerImpl.java:160)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobResourceManager.read(JobResourceManager.java:79)
> at 
> org.apache.ambari.view.hive.resources.jobs.viewJobs.JobResourceManager.readController(JobResourceManager.java:95)
> at 
> org.apache.ambari.view.hive.resources.jobs.JobService.getOne(JobService.java:133)
> at sun.reflect.GeneratedMethodAccessor191.invoke(Unknown Source)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> at 
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at 
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at 
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at 
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at 
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at 
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.ht

Re: Review Request 61178: AMBARI-21572. Spark configuration mismatch post express upgrade

[Attila Doroszlai]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61178/
---

(Updated July 28, 2017, 9:22 a.m.)


Review request for Ambari, Alejandro Fernandez, Eric Yang, Jonathan Hurley, 
Sumit Mohanty, Sebastian Toader, and Sid Wagle.


Changes
---

unconditionally delete spark.yarn.archive


Bugs: AMBARI-21572
https://issues.apache.org/jira/browse/AMBARI-21572


Repository: ambari


Description
---

Remove unnecessary `spark.yarn.archive` property (if value matches IOP default 
one, except it's already converted to `hdfs:///hdp/...`).


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml
 f71e91adcb1091074f83f1f09759968de50c82a9 
  
ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
 9d3cc1e48e090f8a1f43cfaeb024dbeabdf4384b 


Diff: https://reviews.apache.org/r/61178/diff/2/

Changes: https://reviews.apache.org/r/61178/diff/1-2/


Testing
---

Tested upgrade from BI 4.2.5 to HDP 2.6 manually.  Property is removed during 
upgrade and no longer present in Spark configs.


Thanks,

Attila Doroszlai