Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-19 Thread Di Li 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review178255
---


Ship it!




Ship It!

- Di Li


On June 13, 2017, 9:56 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 13, 2017, 9:56 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/4/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-13 Thread Tim Thorpe 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/
---

(Updated June 13, 2017, 9:56 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
Hurley, Jayush Luniya, and Nate Cole.


Changes
---

Updated diff to fix the UI issue: 
  File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
line 901, in sortServicesByDependencies
for component in service["components"]:
TypeError: 'NoneType' object is unsubscriptable


Bugs: AMBARI-20122
https://issues.apache.org/jira/browse/AMBARI-20122


Repository: ambari


Description
---

After resolution of AMBARI-19685, stack advisor validates if stack defined 
dependency is not satisfied but recommendation API does not account for this.

Stack defined dependencies are service/component based and has a scope 
CLUSTER|HOST.  

During recommendation the services to install have already been selected.  We 
can't really utilize the cluster scope because either the dependent service was 
selected or it was not.  If it was not selected it will be caught during 
validation.  We can only recommend based on HOST scope.

This JIRA is also limited to only handling those which don't have conditional 
dependencies.


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
41c57f6 


Diff: https://reviews.apache.org/r/59838/diff/4/

Changes: https://reviews.apache.org/r/59838/diff/3-4/


Testing
---

Added new test case to test the recommendation of dependencies for slaves and 
masters

mvn clean test -DskipSurefireTests


New trunk without my change

Total run:1168
Total errors:13
Total failures:0

New trunk with my change

Total run:1169
Total errors:13
Total failures:0


Thanks,

Tim Thorpe

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-13 Thread Tim Thorpe 


> On June 13, 2017, 7:09 p.m., Nate Cole wrote:
> > Ship It!

Hi Nate, still working on this to fix a deploy with UI issue will hopefully 
have another patch up soon.  Thanks


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review177782
---


On June 7, 2017, 12:26 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 7, 2017, 12:26 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/3/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-13 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review177782
---


Ship it!




Ship It!

- Nate Cole


On June 7, 2017, 8:26 a.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 7, 2017, 8:26 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/3/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-07 Thread Di Li 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review177170
---


Ship it!




Ship It!

- Di Li


On June 7, 2017, 12:26 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 7, 2017, 12:26 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/3/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-07 Thread Tim Thorpe 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/
---

(Updated June 7, 2017, 12:26 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
Hurley, Jayush Luniya, and Nate Cole.


Changes
---

Removed white space


Bugs: AMBARI-20122
https://issues.apache.org/jira/browse/AMBARI-20122


Repository: ambari


Description
---

After resolution of AMBARI-19685, stack advisor validates if stack defined 
dependency is not satisfied but recommendation API does not account for this.

Stack defined dependencies are service/component based and has a scope 
CLUSTER|HOST.  

During recommendation the services to install have already been selected.  We 
can't really utilize the cluster scope because either the dependent service was 
selected or it was not.  If it was not selected it will be caught during 
validation.  We can only recommend based on HOST scope.

This JIRA is also limited to only handling those which don't have conditional 
dependencies.


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
41c57f6 


Diff: https://reviews.apache.org/r/59838/diff/3/

Changes: https://reviews.apache.org/r/59838/diff/2-3/


Testing
---

Added new test case to test the recommendation of dependencies for slaves and 
masters

mvn clean test -DskipSurefireTests


New trunk without my change

Total run:1168
Total errors:13
Total failures:0

New trunk with my change

Total run:1169
Total errors:13
Total failures:0


Thanks,

Tim Thorpe

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-07 Thread Tim Thorpe 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/
---

(Updated June 7, 2017, 12:23 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
Hurley, Jayush Luniya, and Nate Cole.


Changes
---

Updated patch to include documenting new functions.


Bugs: AMBARI-20122
https://issues.apache.org/jira/browse/AMBARI-20122


Repository: ambari


Description
---

After resolution of AMBARI-19685, stack advisor validates if stack defined 
dependency is not satisfied but recommendation API does not account for this.

Stack defined dependencies are service/component based and has a scope 
CLUSTER|HOST.  

During recommendation the services to install have already been selected.  We 
can't really utilize the cluster scope because either the dependent service was 
selected or it was not.  If it was not selected it will be caught during 
validation.  We can only recommend based on HOST scope.

This JIRA is also limited to only handling those which don't have conditional 
dependencies.


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
41c57f6 


Diff: https://reviews.apache.org/r/59838/diff/2/

Changes: https://reviews.apache.org/r/59838/diff/1-2/


Testing
---

Added new test case to test the recommendation of dependencies for slaves and 
masters

mvn clean test -DskipSurefireTests


New trunk without my change

Total run:1168
Total errors:13
Total failures:0

New trunk with my change

Total run:1169
Total errors:13
Total failures:0


Thanks,

Tim Thorpe

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-06 Thread Di Li 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review177078
---


Ship it!




Ship It!

- Di Li


On June 6, 2017, 11:33 a.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 6, 2017, 11:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/1/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-06 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review177072
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 6, 2017, 11:33 a.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 6, 2017, 11:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/1/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-06 Thread Tim Thorpe 


> On June 6, 2017, 5:38 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/stack_advisor.py
> > Lines 759 (patched)
> > 
> >
> > Can you provide an example of why this has to be sorted?

Hi Alejandro,

In the function def getFilteredHostsBasedOnDependencies, you can see that it 
uses the hostsComponentsMap to determine which hosts a given component will be 
installed on.  If the services are not processed in dependency order then you 
might process HDFS before ZOOKEEPER and there would be no hosts where ZOOKEEPER 
will be installed.

for host, hostComponents in hostsComponentsMap.iteritems():
  isRequiredIncluded = False
  for component in hostComponents:
currentComponentName = None if "name" not in component else 
component["name"]
if requiredComponentName == currentComponentName:
  isRequiredIncluded = True
  if not isRequiredIncluded:
removeHosts.append(host)


- Tim


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review177055
---


On June 6, 2017, 11:33 a.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 6, 2017, 11:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/1/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters

2017-06-06 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59838/#review177055
---




ambari-server/src/main/resources/stacks/stack_advisor.py
Lines 759 (patched)


Can you provide an example of why this has to be sorted?



ambari-server/src/main/resources/stacks/stack_advisor.py
Lines 882 (patched)


Add doc to this method that processedServices and serotedServices are 
modified.



ambari-server/src/main/resources/stacks/stack_advisor.py
Lines 902 (patched)


Add documentation to this method


- Alejandro Fernandez


On June 6, 2017, 11:33 a.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59838/
> ---
> 
> (Updated June 6, 2017, 11:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-20122
> https://issues.apache.org/jira/browse/AMBARI-20122
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After resolution of AMBARI-19685, stack advisor validates if stack defined 
> dependency is not satisfied but recommendation API does not account for this.
> 
> Stack defined dependencies are service/component based and has a scope 
> CLUSTER|HOST.  
> 
> During recommendation the services to install have already been selected.  We 
> can't really utilize the cluster scope because either the dependent service 
> was selected or it was not.  If it was not selected it will be caught during 
> validation.  We can only recommend based on HOST scope.
> 
> This JIRA is also limited to only handling those which don't have conditional 
> dependencies.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 41c57f6 
> 
> 
> Diff: https://reviews.apache.org/r/59838/diff/1/
> 
> 
> Testing
> ---
> 
> Added new test case to test the recommendation of dependencies for slaves and 
> masters
> 
> mvn clean test -DskipSurefireTests
> 
> 
> New trunk without my change
> 
> Total run:1168
> Total errors:13
> Total failures:0
> 
> New trunk with my change
> 
> Total run:1169
> Total errors:13
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>