Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

> On June 2, 2016, 5:01 p.m., Jayush Luniya wrote:
> > Ship It!

@Jeff can you close this CR?


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review135966
---


On June 2, 2016, 10:27 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated June 2, 2016, 10:27 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> 46c7b9d 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

> On June 2, 2016, 4:52 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json,
> >  line 1
> > 
> >
> > Patch does not apply. I dont see a kerberos.json file in HDP-2.5
> > 
> > ls ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK
> > metainfo.xml

Nevermind, applied revision#17 as @Sumit has reverted the patch and hence the 
full patch was required.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review135960
---


On June 2, 2016, 10:27 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated June 2, 2016, 10:27 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> 46c7b9d 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review135966
---


Ship it!




Ship It!

- Jayush Luniya


On June 2, 2016, 10:27 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated June 2, 2016, 10:27 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> 46c7b9d 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review135960
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
(line 1)


Patch does not apply. I dont see a kerberos.json file in HDP-2.5

ls ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK
metainfo.xml


- Jayush Luniya


On June 2, 2016, 10:27 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated June 2, 2016, 10:27 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> 46c7b9d 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated June 2, 2016, 10:27 a.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

upload addendum patch


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
46c7b9d 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review135905
---



@Jeff, can you only attach the addendum patch for the one line change in a new 
CR request. I will verify it and commit this delta patch instead of reverting 
the old patch.

- Jayush Luniya


On June 1, 2016, 11:06 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated June 1, 2016, 11:06 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  eba1abf 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  9a3fa31 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  3f14955 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  99a3ade 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> d2e0d11 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 4a7c1d2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated June 1, 2016, 11:06 p.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 eba1abf 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 9a3fa31 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 3f14955 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 99a3ade 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
d2e0d11 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
4a7c1d2 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review135657
---


Ship it!




Ship It!

- Jayush Luniya


On May 31, 2016, 12:53 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 31, 2016, 12:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  eba1abf 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  9a3fa31 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  3f14955 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  99a3ade 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> d2e0d11 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 4a7c1d2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

> On May 18, 2016, 11:53 p.m., Jayush Luniya wrote:
> > @Jeff 
> > Can you verify that blueprint deployment of the cluster without Livy Server 
> > works? 
> > 
> > Guide on Blueprints
> > https://cwiki.apache.org/confluence/display/AMBARI/Blueprints
> 
> Jayush Luniya wrote:
> @Jeff whats the latest on this? Did you get the blueprint scenario 
> verified? Also can you update the patch with latest in trunk?
> 
> Jeff Zhang wrote:
> @Jayush, I have some issues when verifying blueprint, also ask for help 
> to ambari mail list, but still not progress yet, could you help on that ? 
> http://mail-archives.apache.org/mod_mbox/ambari-user/201605.mbox/%3CCAADy7x7ha6amOjmNFCQN2bKpMACGbLJr3uDxKjPCn=jph_7...@mail.gmail.com%3E
> 
> Jeff Zhang wrote:
> Ping @Jayush

Sorry Jeff was on vacation last few days. Since you were able to get the 
blueprint deployment verified with help from Sumit, I will go ahead with 
committing this patch.


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review133841
---


On May 31, 2016, 12:53 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 31, 2016, 12:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  eba1abf 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  9a3fa31 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  3f14955 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  99a3ade 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> d2e0d11 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 4a7c1d2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 31, 2016, 12:53 p.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

rebase patch


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 eba1abf 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 9a3fa31 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 3f14955 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 99a3ade 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
d2e0d11 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
4a7c1d2 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 31, 2016, 12:52 p.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

rebase patch


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 eba1abf 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 9a3fa31 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 3f14955 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 99a3ade 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
d2e0d11 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
4a7c1d2 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On May 18, 2016, 11:53 p.m., Jayush Luniya wrote:
> > @Jeff 
> > Can you verify that blueprint deployment of the cluster without Livy Server 
> > works? 
> > 
> > Guide on Blueprints
> > https://cwiki.apache.org/confluence/display/AMBARI/Blueprints
> 
> Jayush Luniya wrote:
> @Jeff whats the latest on this? Did you get the blueprint scenario 
> verified? Also can you update the patch with latest in trunk?
> 
> Jeff Zhang wrote:
> @Jayush, I have some issues when verifying blueprint, also ask for help 
> to ambari mail list, but still not progress yet, could you help on that ? 
> http://mail-archives.apache.org/mod_mbox/ambari-user/201605.mbox/%3CCAADy7x7ha6amOjmNFCQN2bKpMACGbLJr3uDxKjPCn=jph_7...@mail.gmail.com%3E

Ping @Jayush


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review133841
---


On May 26, 2016, 3:22 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 26, 2016, 3:22 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  775e7eb 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  aa64528 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  3cae192 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 4013835 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 26, 2016, 3:22 a.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

Rebase patch


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 775e7eb 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 aa64528 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 c5f3eb6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 3cae192 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
6e506a0 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
4013835 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On May 18, 2016, 11:53 p.m., Jayush Luniya wrote:
> > @Jeff 
> > Can you verify that blueprint deployment of the cluster without Livy Server 
> > works? 
> > 
> > Guide on Blueprints
> > https://cwiki.apache.org/confluence/display/AMBARI/Blueprints
> 
> Jayush Luniya wrote:
> @Jeff whats the latest on this? Did you get the blueprint scenario 
> verified? Also can you update the patch with latest in trunk?

@Jayush, I have some issues when verifying blueprint, also ask for help to 
ambari mail list, but still not progress yet, could you help on that ? 
http://mail-archives.apache.org/mod_mbox/ambari-user/201605.mbox/%3CCAADy7x7ha6amOjmNFCQN2bKpMACGbLJr3uDxKjPCn=jph_7...@mail.gmail.com%3E


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review133841
---


On May 18, 2016, 11:10 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 18, 2016, 11:10 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  2e18443 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  9e9cadc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  3ec5d05 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

> On May 18, 2016, 11:53 p.m., Jayush Luniya wrote:
> > @Jeff 
> > Can you verify that blueprint deployment of the cluster without Livy Server 
> > works? 
> > 
> > Guide on Blueprints
> > https://cwiki.apache.org/confluence/display/AMBARI/Blueprints

@Jeff whats the latest on this? Did you get the blueprint scenario verified? 
Also can you update the patch with latest in trunk?


- Jayush


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review133841
---


On May 18, 2016, 11:10 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 18, 2016, 11:10 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  2e18443 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  9e9cadc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  3ec5d05 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jianfeng (Jeff) Zhang]

+ Sean Roberts who may able to help

Hi Jayush & Sean,

I also ask the question on ambari mail list, but still could not make progress, 
not sure how to verify that, do you have any suggestion on that ? Very 
appreciated.

http://mail-archives.apache.org/mod_mbox/ambari-user/201605.mbox/%3CCAADy7x7ha6amOjmNFCQN2bKpMACGbLJr3uDxKjPCn=jph_7...@mail.gmail.com%3E



Best Regard,
Jeff Zhang


From: Jianfeng Zhang >
Date: Friday, May 20, 2016 at 7:28 AM
To: Jayush Luniya >, 
Sumit Mohanty >, 
Alejandro Fernandez 
>
Cc: Jeff Zhang >, Ambari 
>
Subject: Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave 
component of Spark


Hi Jayush，

Could you help take a look at it ? Thanks.


Best Regard,
Jeff Zhang


From: Jianfeng Zhang >
Date: Thursday, May 19, 2016 at 10:22 AM
To: Jayush Luniya >, 
Sumit Mohanty >, 
Alejandro Fernandez 
>
Cc: Jeff Zhang >, Ambari 
>
Subject: Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave 
component of Spark


Hi Jayush,

I try to verify it on blueprints,  but fails with 2 problems.
One is when setup hw repo, another is when registering blueprint. Here’s the 
errors I get, could you guide me what steps I miss here ? Thanks

Setup hw repo.


curl -X PUT  -u admin:admin -d @repo.json -H 'X-Requested-By: ambari' 
http://jzhang-test3-1.openstacklocal:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/centos6/repositories/hw_repo


Error I get:

{

  "status" : 404,

  "message" : "org.apache.ambari.server.controller.spi.NoSuchResourceException: 
The specified resource doesn't exist: Stack data, stackName=HDP, 
stackVersion=2.5, osType=centos6, repoId=hw_repo”

}


Register blueprint


curl -X POST -u admin:admin -d @dev_cluster.json -H 'X-Requested-By: ambari' 
http://jzhang-test3-1.openstacklocal:8080/api/v1/blueprints/dev_cluster


Error I get


{

  "status" : 400,

  "message" : "Blueprint configuration validation failed: Missing required 
properties.  Specify a value for these properties in the blueprint 
configuration. {host_group_1={kerberos-env=[kdc_type, kdc_hosts, 
encryption_types, realm, ldap_url, container_dn]}}”

}



Attach are the request body file for these 2 cases.




Best Regard,
Jeff Zhang


From: Jayush Luniya 
> on behalf of 
Jayush Luniya >
Reply-To: Jayush Luniya 
>
Date: Thursday, May 19, 2016 at 7:53 AM
To: Jayush Luniya >, 
Sumit Mohanty >, 
Alejandro Fernandez 
>
Cc: Jeff Zhang >, Ambari 
>
Subject: Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave 
component of Spark

This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/46727/


@Jeff
Can you verify that blueprint deployment of the cluster without Livy Server 
works?

Guide on Blueprints
https://cwiki.apache.org/confluence/display/AMBARI/Blueprints


- Jayush Luniya


On May 18th, 2016, 11:10 a.m. UTC, Jeff Zhang wrote:

Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.
By Jeff Zhang.

Updated May 18, 2016, 11:10 a.m.

Bugs: AMBARI-15612
Repository: ambari
Description

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Testing

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Diffs

  *   
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 (2e18443)
  *   
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 (9e9cadc)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 (PRE-CREATION)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 (PRE-CREATION)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 (c5f3eb6)
  *   

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jianfeng (Jeff) Zhang]

Hi Jayush，

Could you help take a look at it ? Thanks.


Best Regard,
Jeff Zhang


From: Jianfeng Zhang >
Date: Thursday, May 19, 2016 at 10:22 AM
To: Jayush Luniya >, 
Sumit Mohanty >, 
Alejandro Fernandez 
>
Cc: Jeff Zhang >, Ambari 
>
Subject: Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave 
component of Spark


Hi Jayush,

I try to verify it on blueprints,  but fails with 2 problems.
One is when setup hw repo, another is when registering blueprint. Here’s the 
errors I get, could you guide me what steps I miss here ? Thanks

Setup hw repo.


curl -X PUT  -u admin:admin -d @repo.json -H 'X-Requested-By: ambari' 
http://jzhang-test3-1.openstacklocal:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/centos6/repositories/hw_repo


Error I get:

{

  "status" : 404,

  "message" : "org.apache.ambari.server.controller.spi.NoSuchResourceException: 
The specified resource doesn't exist: Stack data, stackName=HDP, 
stackVersion=2.5, osType=centos6, repoId=hw_repo”

}


Register blueprint


curl -X POST -u admin:admin -d @dev_cluster.json -H 'X-Requested-By: ambari' 
http://jzhang-test3-1.openstacklocal:8080/api/v1/blueprints/dev_cluster


Error I get


{

  "status" : 400,

  "message" : "Blueprint configuration validation failed: Missing required 
properties.  Specify a value for these properties in the blueprint 
configuration. {host_group_1={kerberos-env=[kdc_type, kdc_hosts, 
encryption_types, realm, ldap_url, container_dn]}}”

}



Attach are the request body file for these 2 cases.




Best Regard,
Jeff Zhang


From: Jayush Luniya 
> on behalf of 
Jayush Luniya >
Reply-To: Jayush Luniya 
>
Date: Thursday, May 19, 2016 at 7:53 AM
To: Jayush Luniya >, 
Sumit Mohanty >, 
Alejandro Fernandez 
>
Cc: Jeff Zhang >, Ambari 
>
Subject: Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave 
component of Spark

This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/46727/


@Jeff
Can you verify that blueprint deployment of the cluster without Livy Server 
works?

Guide on Blueprints
https://cwiki.apache.org/confluence/display/AMBARI/Blueprints


- Jayush Luniya


On May 18th, 2016, 11:10 a.m. UTC, Jeff Zhang wrote:

Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.
By Jeff Zhang.

Updated May 18, 2016, 11:10 a.m.

Bugs: AMBARI-15612
Repository: ambari
Description

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Testing

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Diffs

  *   
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 (2e18443)
  *   
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 (9e9cadc)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 (PRE-CREATION)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 (PRE-CREATION)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 (c5f3eb6)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 (694f046)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 (PRE-CREATION)
  *   
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 (86e7f7d)
  *   
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 (3ec5d05)
  *   
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
(6e506a0)
  *   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
(f7e15aa)
  *   
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-conf.xml
 (PRE-CREATION)
  *   
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 (PRE-CREATION)
  *   
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 (PRE-CREATION)
  *   

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 17, 2016, 12:17 a.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

Remove mis-uploaded file


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 2e18443 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 9e9cadc 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 c5f3eb6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 3ec5d05 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
6e506a0 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f7e15aa 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 17, 2016, 12:11 a.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

Change the cardinality of livy and add new conf file spark-blacklist.proerties.


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 2e18443 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 9e9cadc 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 c5f3eb6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 3ec5d05 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
6e506a0 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f7e15aa 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-spark-blacklist.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 
  ambari-server/src/test/python/stacks/2.5/SPARK/test_livy_server.py 
PRE-CREATION 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review133396
---


Ship it!





ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 (line 31)


May want to prepend "params." to make it clear where "livy_server_start" is 
coming from.
Same with livy_server_stop below.


- Alejandro Fernandez


On May 14, 2016, 12:56 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 14, 2016, 12:56 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  2e18443 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  9e9cadc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  3ec5d05 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 14, 2016, 12:56 a.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

rebase patch


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 2e18443 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 9e9cadc 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 c5f3eb6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 3ec5d05 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
6e506a0 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f7e15aa 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review132935
---


Ship it!




Ship It!

- Jayush Luniya


On May 12, 2016, 12:41 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 12, 2016, 12:41 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  aabc37d 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 12, 2016, 12:41 a.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

minor update on role_command_order.json


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 abd2212 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 daa97de 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 c5f3eb6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 aabc37d 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
6e506a0 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f7e15aa 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On May 7, 2016, 7:31 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py,
> >  line 280
> > 
> >
> > We should add this to HDP25StackAdvisor instead?
> > 
> > 
> > ambari/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py

The method recommendHadoopProxyUsers is for hadoop configuration. I feel it is 
better to put it here. Otherwise I have to copy part of code in 
recommendHadoopProxyUsers to HDP25StackAdvisor.py which is not good for code 
reuse and maintenance.


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review132146
---


On May 7, 2016, 3:09 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 7, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  aabc37d 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review132146
---




ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
(line 280)


We should add this to HDP25StackAdvisor instead?


ambari/ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py


- Jayush Luniya


On May 7, 2016, 3:09 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 7, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  aabc37d 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review132144
---




ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 (line 40)


Why 5?


- Jayush Luniya


On May 7, 2016, 3:09 p.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated May 7, 2016, 3:09 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  aabc37d 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 6e506a0 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
> PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> d1129cd 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated May 7, 2016, 3:09 p.m.)


Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
Mohanty.


Changes
---

Rebase the patch, but unfornatelly it still fails for the same issue of 
previous patch. check_stack_feature(StackFeature.SPARK_LIVY, 
stack_version_formatted) always return False.


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 abd2212 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 daa97de 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 c5f3eb6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 aabc37d 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
6e506a0 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f7e15aa 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
d1129cd 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On April 29, 2016, 12:25 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py,
> >  line 45
> > 
> >
> > Will Spark hist server and livy server always allow http calls? Do we 
> > need support for https?
> > 
> > I also think that we can reduce the retries to 5 times. Any reason 
> > service_check should keep on trying 10 times?

https is supported. But user have to enable it manually because ambari seems 
don't have the ability to generate keystore file. 
Will change the retries to 5 times.


> On April 29, 2016, 12:25 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml,
> >  line 44
> > 
> >
> > One hour seems too long. Will it have any impact of start/service check?

It is default setting of livy. It is session timeout, should not related with 
service check.


> On April 29, 2016, 12:25 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml,
> >  line 27
> > 
> >
> > What is the motivation to move these to common-services?

This is for easy code manintance. Considering spark evolve much faster than 
other components, so I put all of the spark code under common-service, the new 
version extend the old version. All the python script is in the oldest one 
1.2.0.2.2.  In the real HDP stack definition, it just either extend from 
common-service or extend from its parent HDP stack.


> On April 29, 2016, 12:25 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json, 
> > line 10
> > 
> >
> > Why does LIVY server need all the four to be started?
> > 
> > Does SPARK service check also need to wait for LIVY server start?

hmm. Seems livy service start doesn't need these depedency. 
And yes spark service check need to wait for livy server start, because spark 
service check include livy service check.


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131041
---


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> 

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On April 29, 2016, 10:10 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py,
> >  line 61
> > 
> >
> > get_stack_to_compoent is now been removed and replaced with 
> > get_component_name(). See AMBARI-15609

Will fix it


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131201
---


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On April 29, 2016, 9:53 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml,
> >  line 1
> > 
> >
> > We shouldnt name this as 1.6.x.2.4. Just SPARK/1.6.0. I dont know if a 
> > non-digit charater here will cause any issues. The last 2-digits "2.4" is 
> > really HDP version and should not be added. Yes we have 5-digit numbering 
> > in common-services with HDP version in it but it is incorrect and needs to 
> > be fixed :). In the HDP stack we can override the version in 
> > stacks/HDP/2.4/services/SPARK/metainfo.xml if we want to be specific as say 
> > "1.6.1.2.4"

The non-digit is added in https://issues.apache.org/jira/browse/AMBARI-15497 ,  
I think the purpose is for avoid changing ambari code each time when spark 
version changes because spark evolution very rapidly.  And I notice other 
service under commonservices are all with HDP version at the end. Does it 
really need to be removed ?


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131197
---


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On April 29, 2016, 10:07 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json,
> >  line 1
> > 
> >
> > Having 1.6.x.2.4 and 1.6.x.2.5 is definitely confusing especially for 
> > Non-HDP stack. We really need to make common-services truely HDP agnostic.

As I mentioned above, I think we should create a new ticket for removing HDP 
version for all the services under common-services.


> On April 29, 2016, 10:07 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml,
> >  line 26
> > 
> >
> > Why do we call this 1.6.x.2.4? In the UI we will display version as 
> > 1.6.x.2.4 as well

I think it is OK, this is purpose of AMBARI-15497


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131200
---


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

> On April 29, 2016, 9:57 p.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml,
> >  line 24
> > 
> >
> > We now support reading properties from files. You can put this in 
> > common-services/SPARK/1.6.1/properties/fairscheduler.xml and read from 
> > there. That way you dont have to embed an encoded xml string within the 
> > config xml. See https://issues.apache.org/jira/browse/AMBARI-15842 as an 
> > example.

For this kind of code refactoring, I'd like to do it in another ticket in case 
it would introduce new bugs.


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131199
---


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131201
---




ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
 (line 61)


get_stack_to_compoent is now been removed and replaced with 
get_component_name(). See AMBARI-15609


- Jayush Luniya


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131200
---




ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json 
(line 1)


Having 1.6.x.2.4 and 1.6.x.2.5 is definitely confusing especially for 
Non-HDP stack. We really need to make common-services truely HDP agnostic.



ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
(line 26)


Why do we call this 1.6.x.2.4? In the UI we will display version as 
1.6.x.2.4 as well


- Jayush Luniya


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131199
---




ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
 (line 24)


We now support reading properties from files. You can put this in 
common-services/SPARK/1.6.1/properties/fairscheduler.xml and read from there. 
That way you dont have to embed an encoded xml string within the config xml. 
See https://issues.apache.org/jira/browse/AMBARI-15842 as an example.


- Jayush Luniya


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review131041
---




ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
 (line 188)


This seems a little odd that whether the stack is 2.5 or not is based on if 
ROLLING_UPGRADE is supported?



ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
 (line 45)


Will Spark hist server and livy server always allow http calls? Do we need 
support for https?

I also think that we can reduce the retries to 5 times. Any reason 
service_check should keep on trying 10 times?



ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
 (line 44)


One hour seems too long. Will it have any impact of start/service check?



ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
(line 27)


What is the motivation to move these to common-services?



ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json (line 
10)


Why does LIVY server need all the four to be started?

Does SPARK service check also need to wait for LIVY server start?


- Sumit Mohanty


On April 29, 2016, 7:45 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 29, 2016, 7:45 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  abd2212 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  daa97de 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  c5f3eb6 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  86e7f7d 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

(Updated April 29, 2016, 3 a.m.)


Review request for Ambari and Sumit Mohanty.


Changes
---

add LIVY_SERVER_JAVA_OPTS to livy-env.sh


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark.

Move the spark stack definiation to commoon-services and then


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 daa97de 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
 c5f3eb6 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
 86e7f7d 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
 1a6552f 
  
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
 2dda4bb 
  
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
 4f2563b 
  ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
a5a19f6 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f7e15aa 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
107ca93 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review130983
---



Please rebase the patch to the tip of trunk.

- Alejandro Fernandez


On April 28, 2016, 11:35 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 28, 2016, 11:35 a.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  6169699 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  a64f36f 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  19c9a0e 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review130954
---


Ship it!




Ship It!

- Alejandro Fernandez


On April 28, 2016, 11:35 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 28, 2016, 11:35 a.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  6169699 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  a64f36f 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  19c9a0e 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/#review130815
---




ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
 (line 24)


Prefer to use named imports



ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
 (line 188)


Take a look at how other components do this now.
There's a "feature" check that returns the version a certain feature is 
enabled.



ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
 (line 190)


We shouldn't hardcode /usr/hdp, there's a variable to get the stack root.



ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
 (line 216)


This variable will be undefined if len is 0.



ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
 (line 22)


Should avoid import * since causes problems in customer environments.


- Alejandro Fernandez


On April 27, 2016, 7:43 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46727/
> ---
> 
> (Updated April 27, 2016, 7:43 a.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15612
> https://issues.apache.org/jira/browse/AMBARI-15612
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add Livy to HDP 2.5 as slave component of Spark.
> 
> Move the spark stack definiation to commoon-services and then
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
>  a64f36f 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
>  19c9a0e 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  1a6552f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
>  2dda4bb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  4f2563b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
> a5a19f6 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> f7e15aa 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
> 107ca93 
> 
> Diff: https://reviews.apache.org/r/46727/diff/
> 
> 
> Testing
> ---
> 
> Manully verify it. Livy is installed correctly in non-secure and secure 
> cluster.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Review Request 46727: [AMBARI-15612] Add Livy to HDP 2.5 as slave component of Spark

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46727/
---

Review request for Ambari and Sumit Mohanty.


Bugs: AMBARI-15612
https://issues.apache.org/jira/browse/AMBARI-15612


Repository: ambari


Description
---

Add Livy to HDP 2.5 as slave component of Spark


Diffs
-

  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_server.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/livy_service.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
 a64f36f 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/service_check.py
 694f046 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_livy.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/status_params.py
 19c9a0e 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-fairscheduler.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/configuration/spark-thrift-sparkconf.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.4/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-defaults.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/configuration/livy-log4j-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/common-services/SPARK/1.6.x.2.5/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
 1a6552f 
  
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
 2dda4bb 
  
ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
 4f2563b 
  ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 
a5a19f6 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
f7e15aa 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/metainfo.xml 
107ca93 

Diff: https://reviews.apache.org/r/46727/diff/


Testing
---

Manully verify it. Livy is installed correctly in non-secure and secure cluster.


Thanks,

Jeff Zhang