Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-05-13 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/#review133132
---


Ship it!




Ship It!

- Sumit Mohanty


On May 13, 2016, 3:25 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46583/
> ---
> 
> (Updated May 13, 2016, 3:25 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
> Sumit Mohanty, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-16072
> https://issues.apache.org/jira/browse/AMBARI-16072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding a service to a Kerberized cluster and click install nothing 
> happens on the UI and i see the following error in the ambari server logs
> 
> ```
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for JOURNALNODE to auth to local 
> mapping
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for METRICS_COLLECTOR to auth to 
> local mapping
> 20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 
> - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
> actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
> command=recommend-configurations
> 20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 
> - Stack-advisor 
> output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
> error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 
> - Stack advisor output files
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 
> - advisor script stdout: StackAdvisor implementation for stack HDP, 
> version 2.0.6 was loaded
> StackAdvisor implementation for stack HDP, version 2.1 was loaded
> StackAdvisor implementation for stack HDP, version 2.2 was loaded
> StackAdvisor implementation for stack HDP, version 2.3 was loaded
> StackAdvisor implementation for stack HDP, version 2.4 was loaded
> Returning HDP24StackAdvisor implementation
> Error occured in stack advisor.
> Error details: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 
> - advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 570, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 627, in recommendAmsConfigurations
> if set(amsCollectorHosts).intersection(dn_hosts):
> TypeError: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
> AbstractResourceProvider:802 - Caught an exception while updating host 
> components, retrying : org.apache.ambari.server.AmbariException: Stack 
> Advisor reported an error: TypeError: 'NoneType' object is not iterable
> StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out
> 
> StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> ```
> 
> # Solution
> Pass to the stack advisor information about all installed services where each 
> component is installed (component host map)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  fe466d5 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> 9c0052f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  13eea58 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  e03ed86 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
>  7dc8bf8 
> 
> Diff: https://reviews.apache.org/r/46583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested using the UI and BP.
> 
> # Local test results: 
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 59:18.081s
> [INFO] Finished at: Sat Apr 23 18:37:45 EDT 2016
> [INFO] Final Memory: 70M/1720M
> [INFO] 
> 

Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-05-13 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/
---

(Updated May 13, 2016, 11:25 a.m.)


Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
Sumit Mohanty, and Srimanth Gunturi.


Changes
---

Uploading new patch to account for merge changes... nothing was altered in the 
pathed code.


Bugs: AMBARI-16072
https://issues.apache.org/jira/browse/AMBARI-16072


Repository: ambari


Description
---

When adding a service to a Kerberized cluster and click install nothing happens 
on the UI and i see the following error in the ambari server logs

```
20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] KerberosHelperImpl:735 
- Adding identity for JOURNALNODE to auth to local mapping
20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] KerberosHelperImpl:735 
- Adding identity for METRICS_COLLECTOR to auth to local mapping
20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 - 
Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
command=recommend-configurations
20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 - 
Stack-advisor 
output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 - 
Stack advisor output files
20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 - 
advisor script stdout: StackAdvisor implementation for stack HDP, version 
2.0.6 was loaded
StackAdvisor implementation for stack HDP, version 2.1 was loaded
StackAdvisor implementation for stack HDP, version 2.2 was loaded
StackAdvisor implementation for stack HDP, version 2.3 was loaded
StackAdvisor implementation for stack HDP, version 2.4 was loaded
Returning HDP24StackAdvisor implementation
Error occured in stack advisor.
Error details: 'NoneType' object is not iterable
20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 - 
advisor script stderr: Traceback (most recent call last):
  File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
in 
main(sys.argv)
  File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
in main
result = stackAdvisor.recommendConfigurations(services, hosts)
  File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
line 570, in recommendConfigurations
calculation(configurations, clusterSummary, services, hosts)
  File 
"/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
 line 627, in recommendAmsConfigurations
if set(amsCollectorHosts).intersection(dn_hosts):
TypeError: 'NoneType' object is not iterable
20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
AbstractResourceProvider:802 - Caught an exception while updating host 
components, retrying : org.apache.ambari.server.AmbariException: Stack Advisor 
reported an error: TypeError: 'NoneType' object is not iterable
StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out

StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
```

# Solution
Pass to the stack advisor information about all installed services where each 
component is installed (component host map)


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 fe466d5 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
9c0052f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 13eea58 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 e03ed86 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
 7dc8bf8 

Diff: https://reviews.apache.org/r/46583/diff/


Testing
---

Manually tested using the UI and BP.

# Local test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 59:18.081s
[INFO] Finished at: Sat Apr 23 18:37:45 EDT 2016
[INFO] Final Memory: 70M/1720M
[INFO] 

# Jenkins test results: 

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 01:48 h
[INFO] Finished at: 2016-04-24T03:13:41+00:00
[INFO] Final Memory: 144M/469M
[INFO] 



Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-04-23 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/
---

(Updated April 23, 2016, 11:22 a.m.)


Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
Sumit Mohanty, and Srimanth Gunturi.


Changes
---

Moved `KerberosHelperImpl#buildComponentHostMap` to 
`Cluster.getComponentHostMap` and updated unit tests.


Bugs: AMBARI-16072
https://issues.apache.org/jira/browse/AMBARI-16072


Repository: ambari


Description
---

When adding a service to a Kerberized cluster and click install nothing happens 
on the UI and i see the following error in the ambari server logs

```
20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] KerberosHelperImpl:735 
- Adding identity for JOURNALNODE to auth to local mapping
20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] KerberosHelperImpl:735 
- Adding identity for METRICS_COLLECTOR to auth to local mapping
20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 - 
Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
command=recommend-configurations
20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 - 
Stack-advisor 
output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 - 
Stack advisor output files
20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 - 
advisor script stdout: StackAdvisor implementation for stack HDP, version 
2.0.6 was loaded
StackAdvisor implementation for stack HDP, version 2.1 was loaded
StackAdvisor implementation for stack HDP, version 2.2 was loaded
StackAdvisor implementation for stack HDP, version 2.3 was loaded
StackAdvisor implementation for stack HDP, version 2.4 was loaded
Returning HDP24StackAdvisor implementation
Error occured in stack advisor.
Error details: 'NoneType' object is not iterable
20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 - 
advisor script stderr: Traceback (most recent call last):
  File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
in 
main(sys.argv)
  File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
in main
result = stackAdvisor.recommendConfigurations(services, hosts)
  File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
line 570, in recommendConfigurations
calculation(configurations, clusterSummary, services, hosts)
  File 
"/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
 line 627, in recommendAmsConfigurations
if set(amsCollectorHosts).intersection(dn_hosts):
TypeError: 'NoneType' object is not iterable
20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
AbstractResourceProvider:802 - Caught an exception while updating host 
components, retrying : org.apache.ambari.server.AmbariException: Stack Advisor 
reported an error: TypeError: 'NoneType' object is not iterable
StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out

StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
```

# Solution
Pass to the stack advisor information about all installed services where each 
component is installed (component host map)


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 dca81c5 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 84f0bac 

Diff: https://reviews.apache.org/r/46583/diff/


Testing
---

Manually tested using the UI and BP.

# Local test results: PENDING

# Jenkins test results: PENDING


Thanks,

Robert Levas



Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-04-22 Thread Sumit Mohanty

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/#review130241
---


Ship it!




Ship It!

- Sumit Mohanty


On April 22, 2016, 9:12 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46583/
> ---
> 
> (Updated April 22, 2016, 9:12 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
> Sumit Mohanty, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-16072
> https://issues.apache.org/jira/browse/AMBARI-16072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding a service to a Kerberized cluster and click install nothing 
> happens on the UI and i see the following error in the ambari server logs
> 
> ```
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for JOURNALNODE to auth to local 
> mapping
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for METRICS_COLLECTOR to auth to 
> local mapping
> 20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 
> - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
> actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
> command=recommend-configurations
> 20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 
> - Stack-advisor 
> output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
> error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 
> - Stack advisor output files
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 
> - advisor script stdout: StackAdvisor implementation for stack HDP, 
> version 2.0.6 was loaded
> StackAdvisor implementation for stack HDP, version 2.1 was loaded
> StackAdvisor implementation for stack HDP, version 2.2 was loaded
> StackAdvisor implementation for stack HDP, version 2.3 was loaded
> StackAdvisor implementation for stack HDP, version 2.4 was loaded
> Returning HDP24StackAdvisor implementation
> Error occured in stack advisor.
> Error details: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 
> - advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 570, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 627, in recommendAmsConfigurations
> if set(amsCollectorHosts).intersection(dn_hosts):
> TypeError: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
> AbstractResourceProvider:802 - Caught an exception while updating host 
> components, retrying : org.apache.ambari.server.AmbariException: Stack 
> Advisor reported an error: TypeError: 'NoneType' object is not iterable
> StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out
> 
> StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> ```
> 
> # Solution
> Pass to the stack advisor information about all installed services where each 
> component is installed (component host map)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  dca81c5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  84f0bac 
> 
> Diff: https://reviews.apache.org/r/46583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested using the UI and BP.
> 
> # Local test results: PENDING
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>



Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-04-22 Thread Robert Levas


> On April 22, 2016, 4:58 p.m., Aravindan Vijayan wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java,
> >  line 2433
> > 
> >
> > I meant this method.
> 
> Robert Levas wrote:
> I did a bit of searching in the code and didn't see anything obvious.  I 
> search for methods the returned `Map` and none seemd to 
> return a map of components to host names given the cluster object. The 
> closest method that resembles it is 
> `org.apache.ambari.server.api.services.stackadvisor.StackAdvisorBlueprintProcessor#gatherComponentsHostsMap`
>  but it builds the map using data from a Blueprint, not from the existing 
> cluster data. 
> 
> I suppose I can move this method somewhere, but I am not sure where... 
> maybe the Cluster object?
> 
> Aravindan Vijayan wrote:
> I am ok with the method staying in this class.

After thinking about it... I'll move it.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/#review130178
---


On April 22, 2016, 5:12 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46583/
> ---
> 
> (Updated April 22, 2016, 5:12 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
> Sumit Mohanty, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-16072
> https://issues.apache.org/jira/browse/AMBARI-16072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding a service to a Kerberized cluster and click install nothing 
> happens on the UI and i see the following error in the ambari server logs
> 
> ```
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for JOURNALNODE to auth to local 
> mapping
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for METRICS_COLLECTOR to auth to 
> local mapping
> 20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 
> - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
> actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
> command=recommend-configurations
> 20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 
> - Stack-advisor 
> output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
> error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 
> - Stack advisor output files
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 
> - advisor script stdout: StackAdvisor implementation for stack HDP, 
> version 2.0.6 was loaded
> StackAdvisor implementation for stack HDP, version 2.1 was loaded
> StackAdvisor implementation for stack HDP, version 2.2 was loaded
> StackAdvisor implementation for stack HDP, version 2.3 was loaded
> StackAdvisor implementation for stack HDP, version 2.4 was loaded
> Returning HDP24StackAdvisor implementation
> Error occured in stack advisor.
> Error details: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 
> - advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 570, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 627, in recommendAmsConfigurations
> if set(amsCollectorHosts).intersection(dn_hosts):
> TypeError: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
> AbstractResourceProvider:802 - Caught an exception while updating host 
> components, retrying : org.apache.ambari.server.AmbariException: Stack 
> Advisor reported an error: TypeError: 'NoneType' object is not iterable
> StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out
> 
> StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> ```
> 
> # Solution
> Pass to the stack advisor information about all installed services where each 
> component is installed (component host map)
> 
> 
> Diffs
> -
> 
>   
> 

Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-04-22 Thread Aravindan Vijayan


> On April 22, 2016, 8:58 p.m., Aravindan Vijayan wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java,
> >  line 2433
> > 
> >
> > I meant this method.
> 
> Robert Levas wrote:
> I did a bit of searching in the code and didn't see anything obvious.  I 
> search for methods the returned `Map` and none seemd to 
> return a map of components to host names given the cluster object. The 
> closest method that resembles it is 
> `org.apache.ambari.server.api.services.stackadvisor.StackAdvisorBlueprintProcessor#gatherComponentsHostsMap`
>  but it builds the map using data from a Blueprint, not from the existing 
> cluster data. 
> 
> I suppose I can move this method somewhere, but I am not sure where... 
> maybe the Cluster object?

I am ok with the method staying in this class.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/#review130178
---


On April 22, 2016, 9:12 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46583/
> ---
> 
> (Updated April 22, 2016, 9:12 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
> Sumit Mohanty, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-16072
> https://issues.apache.org/jira/browse/AMBARI-16072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding a service to a Kerberized cluster and click install nothing 
> happens on the UI and i see the following error in the ambari server logs
> 
> ```
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for JOURNALNODE to auth to local 
> mapping
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for METRICS_COLLECTOR to auth to 
> local mapping
> 20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 
> - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
> actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
> command=recommend-configurations
> 20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 
> - Stack-advisor 
> output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
> error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 
> - Stack advisor output files
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 
> - advisor script stdout: StackAdvisor implementation for stack HDP, 
> version 2.0.6 was loaded
> StackAdvisor implementation for stack HDP, version 2.1 was loaded
> StackAdvisor implementation for stack HDP, version 2.2 was loaded
> StackAdvisor implementation for stack HDP, version 2.3 was loaded
> StackAdvisor implementation for stack HDP, version 2.4 was loaded
> Returning HDP24StackAdvisor implementation
> Error occured in stack advisor.
> Error details: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 
> - advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 570, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 627, in recommendAmsConfigurations
> if set(amsCollectorHosts).intersection(dn_hosts):
> TypeError: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
> AbstractResourceProvider:802 - Caught an exception while updating host 
> components, retrying : org.apache.ambari.server.AmbariException: Stack 
> Advisor reported an error: TypeError: 'NoneType' object is not iterable
> StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out
> 
> StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> ```
> 
> # Solution
> Pass to the stack advisor information about all installed services where each 
> component is installed (component host map)
> 
> 
> Diffs
> -
> 
>   
> 

Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-04-22 Thread Robert Levas


> On April 22, 2016, 4:58 p.m., Aravindan Vijayan wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java,
> >  line 2433
> > 
> >
> > I meant this method.

I did a bit of searching in the code and didn't see anything obvious.  I search 
for methods the returned `Map` and none seemd to return a 
map of components to host names given the cluster object. The closest method 
that resembles it is 
`org.apache.ambari.server.api.services.stackadvisor.StackAdvisorBlueprintProcessor#gatherComponentsHostsMap`
 but it builds the map using data from a Blueprint, not from the existing 
cluster data. 

I suppose I can move this method somewhere, but I am not sure where... maybe 
the Cluster object?


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/#review130178
---


On April 22, 2016, 5:12 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46583/
> ---
> 
> (Updated April 22, 2016, 5:12 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
> Sumit Mohanty, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-16072
> https://issues.apache.org/jira/browse/AMBARI-16072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding a service to a Kerberized cluster and click install nothing 
> happens on the UI and i see the following error in the ambari server logs
> 
> ```
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for JOURNALNODE to auth to local 
> mapping
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for METRICS_COLLECTOR to auth to 
> local mapping
> 20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 
> - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
> actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
> command=recommend-configurations
> 20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 
> - Stack-advisor 
> output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
> error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 
> - Stack advisor output files
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 
> - advisor script stdout: StackAdvisor implementation for stack HDP, 
> version 2.0.6 was loaded
> StackAdvisor implementation for stack HDP, version 2.1 was loaded
> StackAdvisor implementation for stack HDP, version 2.2 was loaded
> StackAdvisor implementation for stack HDP, version 2.3 was loaded
> StackAdvisor implementation for stack HDP, version 2.4 was loaded
> Returning HDP24StackAdvisor implementation
> Error occured in stack advisor.
> Error details: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 
> - advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 570, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 627, in recommendAmsConfigurations
> if set(amsCollectorHosts).intersection(dn_hosts):
> TypeError: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
> AbstractResourceProvider:802 - Caught an exception while updating host 
> components, retrying : org.apache.ambari.server.AmbariException: Stack 
> Advisor reported an error: TypeError: 'NoneType' object is not iterable
> StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out
> 
> StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> ```
> 
> # Solution
> Pass to the stack advisor information about all installed services where each 
> component is installed (component host map)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  dca81c5 
>   
> 

Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-04-22 Thread Aravindan Vijayan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/#review130178
---




ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 (line 2433)


I meant this method.


- Aravindan Vijayan


On April 22, 2016, 8:47 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46583/
> ---
> 
> (Updated April 22, 2016, 8:47 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16072
> https://issues.apache.org/jira/browse/AMBARI-16072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding a service to a Kerberized cluster and click install nothing 
> happens on the UI and i see the following error in the ambari server logs
> 
> ```
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for JOURNALNODE to auth to local 
> mapping
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for METRICS_COLLECTOR to auth to 
> local mapping
> 20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 
> - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
> actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
> command=recommend-configurations
> 20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 
> - Stack-advisor 
> output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
> error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 
> - Stack advisor output files
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 
> - advisor script stdout: StackAdvisor implementation for stack HDP, 
> version 2.0.6 was loaded
> StackAdvisor implementation for stack HDP, version 2.1 was loaded
> StackAdvisor implementation for stack HDP, version 2.2 was loaded
> StackAdvisor implementation for stack HDP, version 2.3 was loaded
> StackAdvisor implementation for stack HDP, version 2.4 was loaded
> Returning HDP24StackAdvisor implementation
> Error occured in stack advisor.
> Error details: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 
> - advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 570, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 627, in recommendAmsConfigurations
> if set(amsCollectorHosts).intersection(dn_hosts):
> TypeError: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
> AbstractResourceProvider:802 - Caught an exception while updating host 
> components, retrying : org.apache.ambari.server.AmbariException: Stack 
> Advisor reported an error: TypeError: 'NoneType' object is not iterable
> StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out
> 
> StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> ```
> 
> # Solution
> Pass to the stack advisor information about all installed services where each 
> component is installed (component host map)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  dca81c5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  84f0bac 
> 
> Diff: https://reviews.apache.org/r/46583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested using the UI and BP.
> 
> # Local test results: PENDING
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>



Re: Review Request 46583: Stack Advisor issue when adding service to Kerberized cluster

2016-04-22 Thread Aravindan Vijayan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46583/#review130176
---


Ship it!




This looks like a general utility method. Can we check if something similar is 
not already present in StackAdvisorResourceProvider or somewhere else?

If there is no such method, we can go with this.

- Aravindan Vijayan


On April 22, 2016, 8:47 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46583/
> ---
> 
> (Updated April 22, 2016, 8:47 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16072
> https://issues.apache.org/jira/browse/AMBARI-16072
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding a service to a Kerberized cluster and click install nothing 
> happens on the UI and i see the following error in the ambari server logs
> 
> ```
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for JOURNALNODE to auth to local 
> mapping
> 20 Apr 2016 16:03:56,818  INFO [qtp-ambari-client-2764] 
> KerberosHelperImpl:735 - Adding identity for METRICS_COLLECTOR to auth to 
> local mapping
> 20 Apr 2016 16:03:56,857  INFO [qtp-ambari-client-2764] StackAdvisorRunner:47 
> - Script=/var/lib/ambari-server/resources/scripts/stack_advisor.py, 
> actionDirectory=/var/run/ambari-server/stack-recommendations/323, 
> command=recommend-configurations
> 20 Apr 2016 16:03:56,860  INFO [qtp-ambari-client-2764] StackAdvisorRunner:61 
> - Stack-advisor 
> output=/var/run/ambari-server/stack-recommendations/323/stackadvisor.out, 
> error=/var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:69 
> - Stack advisor output files
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:70 
> - advisor script stdout: StackAdvisor implementation for stack HDP, 
> version 2.0.6 was loaded
> StackAdvisor implementation for stack HDP, version 2.1 was loaded
> StackAdvisor implementation for stack HDP, version 2.2 was loaded
> StackAdvisor implementation for stack HDP, version 2.3 was loaded
> StackAdvisor implementation for stack HDP, version 2.4 was loaded
> Returning HDP24StackAdvisor implementation
> Error occured in stack advisor.
> Error details: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,917  INFO [qtp-ambari-client-2764] StackAdvisorRunner:71 
> - advisor script stderr: Traceback (most recent call last):
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 158, 
> in 
> main(sys.argv)
>   File "/var/lib/ambari-server/resources/scripts/stack_advisor.py", line 109, 
> in main
> result = stackAdvisor.recommendConfigurations(services, hosts)
>   File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", 
> line 570, in recommendConfigurations
> calculation(configurations, clusterSummary, services, hosts)
>   File 
> "/var/lib/ambari-server/resources/scripts/./../stacks/HDP/2.0.6/services/stack_advisor.py",
>  line 627, in recommendAmsConfigurations
> if set(amsCollectorHosts).intersection(dn_hosts):
> TypeError: 'NoneType' object is not iterable
> 20 Apr 2016 16:03:56,918  INFO [qtp-ambari-client-2764] 
> AbstractResourceProvider:802 - Caught an exception while updating host 
> components, retrying : org.apache.ambari.server.AmbariException: Stack 
> Advisor reported an error: TypeError: 'NoneType' object is not iterable
> StdOut file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.out
> 
> StdErr file: /var/run/ambari-server/stack-recommendations/323/stackadvisor.err
> ```
> 
> # Solution
> Pass to the stack advisor information about all installed services where each 
> component is installed (component host map)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  dca81c5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  84f0bac 
> 
> Diff: https://reviews.apache.org/r/46583/diff/
> 
> 
> Testing
> ---
> 
> Manually tested using the UI and BP.
> 
> # Local test results: PENDING
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>