Re: Review Request 50239: Spark Livy should wait for ATS start for BP deployments

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50239/
---

(Updated Июль 21, 2016, 9:49 д.п.)


Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and Sumit 
Mohanty.


Bugs: AMBARI-17814
https://issues.apache.org/jira/browse/AMBARI-17814


Repository: ambari


Description
---

Spark Livy should wait for ATS start for BP deployments
We should do something like:

 if params.has_ats:
  Logger.info("Verifying DFS directories where ATS stores time line data 
for active and completed applications.")
  self.wait_for_dfs_directories_created(params.entity_groupfs_store_dir, 
params.entity_groupfs_active_dir)


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 1e859a8 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 61f73fb 
  ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py ed331ea 

Diff: https://reviews.apache.org/r/50239/diff/


Testing
---

[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Main ... SUCCESS [3.566s]
[INFO] Apache Ambari Project POM . SUCCESS [0.038s]
[INFO] Ambari Views .. SUCCESS [3.311s]
[INFO] ambari-metrics  SUCCESS [0.469s]
[INFO] Ambari Metrics Common . SUCCESS [0.681s]
[INFO] Ambari Server . SUCCESS [1:06.033s]
[INFO] 
[INFO] BUILD SUCCESS


Thanks,

Dmytro Sen

Re: Review Request 50239: Spark Livy should wait for ATS start for BP deployments

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50239/#review142987
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 (line 74)


Pass a list of dirs instead



ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 (line 190)


Keep it simple, len(ats_hosts) > 0


- Alejandro Fernandez


On July 20, 2016, 3:40 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50239/
> ---
> 
> (Updated July 20, 2016, 3:40 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17814
> https://issues.apache.org/jira/browse/AMBARI-17814
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Spark Livy should wait for ATS start for BP deployments
> We should do something like:
> 
>  if params.has_ats:
>   Logger.info("Verifying DFS directories where ATS stores time line data 
> for active and completed applications.")
>   self.wait_for_dfs_directories_created(params.entity_groupfs_store_dir, 
> params.entity_groupfs_active_dir)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  1e859a8 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  61f73fb 
>   ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py ed331ea 
> 
> Diff: https://reviews.apache.org/r/50239/diff/
> 
> 
> Testing
> ---
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Ambari Main ... SUCCESS [3.566s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Views .. SUCCESS [3.311s]
> [INFO] ambari-metrics  SUCCESS [0.469s]
> [INFO] Ambari Metrics Common . SUCCESS [0.681s]
> [INFO] Ambari Server . SUCCESS [1:06.033s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 50239: Spark Livy should wait for ATS start for BP deployments

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50239/#review142948
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On July 20, 2016, 6:40 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50239/
> ---
> 
> (Updated July 20, 2016, 6:40 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17814
> https://issues.apache.org/jira/browse/AMBARI-17814
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Spark Livy should wait for ATS start for BP deployments
> We should do something like:
> 
>  if params.has_ats:
>   Logger.info("Verifying DFS directories where ATS stores time line data 
> for active and completed applications.")
>   self.wait_for_dfs_directories_created(params.entity_groupfs_store_dir, 
> params.entity_groupfs_active_dir)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
>  1e859a8 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  61f73fb 
>   ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py ed331ea 
> 
> Diff: https://reviews.apache.org/r/50239/diff/
> 
> 
> Testing
> ---
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Ambari Main ... SUCCESS [3.566s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Views .. SUCCESS [3.311s]
> [INFO] ambari-metrics  SUCCESS [0.469s]
> [INFO] Ambari Metrics Common . SUCCESS [0.681s]
> [INFO] Ambari Server . SUCCESS [1:06.033s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 50239: Spark Livy should wait for ATS start for BP deployments

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50239/
---

Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and Sumit 
Mohanty.


Bugs: AMBARI-17814
https://issues.apache.org/jira/browse/AMBARI-17814


Repository: ambari


Description
---

Spark Livy should wait for ATS start for BP deployments
We should do something like:

 if params.has_ats:
  Logger.info("Verifying DFS directories where ATS stores time line data 
for active and completed applications.")
  self.wait_for_dfs_directories_created(params.entity_groupfs_store_dir, 
params.entity_groupfs_active_dir)


Diffs
-

  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
 1e859a8 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 61f73fb 
  ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py ed331ea 

Diff: https://reviews.apache.org/r/50239/diff/


Testing
---

[INFO] Reactor Summary:
[INFO] 
[INFO] Ambari Main ... SUCCESS [3.566s]
[INFO] Apache Ambari Project POM . SUCCESS [0.038s]
[INFO] Ambari Views .. SUCCESS [3.311s]
[INFO] ambari-metrics  SUCCESS [0.469s]
[INFO] Ambari Metrics Common . SUCCESS [0.681s]
[INFO] Ambari Server . SUCCESS [1:06.033s]
[INFO] 
[INFO] BUILD SUCCESS


Thanks,

Dmytro Sen