date:20160912


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51833/
---

(Updated Sept. 13, 2016, 1:53 a.m.)


Review request for Ambari, Sumit Mohanty and Swapan Shridhar.


Bugs: AMBARI-18368
https://issues.apache.org/jira/browse/AMBARI-18368


Repository: ambari


Description
---

Steps to Reproduce:
* Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
* Kerberize the cluster
* Perform EU/RU to HDP 2.5
* Add Atlas Service

Atlas Server log contains,
```
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at 
http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not find 
the specified config set: vertex_index  
```

Fix was to change ownership of users-credentials.properties and 
policy-store.txt to atlas:hadoop


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 0c07018 

Diff: https://reviews.apache.org/r/51833/diff/


Testing
---

Verified on a live cluster.

Python unit tests passed.
Total run:1124
Total errors:0
Total failures:0
OK


Thanks,

Alejandro Fernandez

Re: Review Request 51833: Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51833/#review148636
---




ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 (line 100)


Will remove #TODO after performing another test on a live cluster


- Alejandro Fernandez


On Sept. 13, 2016, 1:49 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51833/
> ---
> 
> (Updated Sept. 13, 2016, 1:49 a.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty and Swapan Shridhar.
> 
> 
> Bugs: AMBARI-18368
> https://issues.apache.org/jira/browse/AMBARI-18368
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps to Reproduce:
> * Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
> * Kerberize the cluster
> * Perform EU/RU to HDP 2.5
> * Add Atlas Service
> 
> Atlas Server log contains,
> ```
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at 
> http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not 
> find the specified config set: vertex_index  
> ```
> 
> Fix was to change ownership of users-credentials.properties and 
> policy-store.txt to atlas:hadoop
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  0c07018 
> 
> Diff: https://reviews.apache.org/r/51833/diff/
> 
> 
> Testing
> ---
> 
> Verified on a live cluster.
> 
> Python unit tests passed.
> Total run:1124
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 51833: Atlas web UI alert after performing stack upgrade to HDP 2.5 and adding Atlas Service


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51833/
---

Review request for Ambari, Sumit Mohanty and Swapan Shridhar.


Bugs: AMBARI-18368
https://issues.apache.org/jira/browse/AMBARI-18368


Repository: ambari


Description
---

Steps to Reproduce:
* Install Ambari 2.2.2 with HDP 2.4 (HBase, Solr)
* Kerberize the cluster
* Perform EU/RU to HDP 2.5
* Add Atlas Service

Atlas Server log contains,
```
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at 
http://natu146-ehbs-dgm10toeriesec-u14-1.openstacklocal:8886/solr: Can not find 
the specified config set: vertex_index  
```

Fix was to change ownership of users-credentials.properties and 
policy-store.txt to atlas:hadoop


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 0c07018 

Diff: https://reviews.apache.org/r/51833/diff/


Testing
---

Verified on a live cluster.

Python unit tests passed.
Total run:1124
Total errors:0
Total failures:0
OK


Thanks,

Alejandro Fernandez

Re: Review Request 51705: Password in the configurations.json file in the ambari-agent cache is not encrypted

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51705/
---

(Updated Sept. 12, 2016, 10:48 p.m.)

Review request for Ambari, Di Li, Jonathan Hurley, and Nate Cole.

Bugs: AMBARI-18334
https://issues.apache.org/jira/browse/AMBARI-18334

Repository: ambari

Description
---

The configurations.json file loaded in the ambari-agent cache located at
/var/lib/ambari-agent/cache/cluster_configuration contains password details in
plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password
etc.). The values are loaded both in the memory cache and file cache, the file
seems to be used only for debugging purposes, so it would be a better approach
to mask the passwords in the file.

Approach:

The password_config_type is included in the heartbeat response for alert
definition command and execution command, for which the values are dumped into
the json file. The password_config_type contains the information on which
properties in the configurations has the propertyType password. Based on the
response, the json is parsed and the password values are masked before dumping
it into the configurations.json file.

Diffs (updated)
-

ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py 72b87be
ambari-agent/src/test/python/ambari_agent/TestAlerts.py 2bddc43
ambari-agent/src/test/python/ambari_agent/TestClusterConfigurationCache.py
a418f6d

ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java
0562c15

ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java
4d2e048

ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
29737ee
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
70c24f9

Diff: https://reviews.apache.org/r/51705/diff/

Testing
---

Updated the test cases.
Ran mvn test.

Manually tested by setting up a cluster, the password fields in the
configurations.json is masked. During testing, everytime the ambari agent is
restarted, it registers with the server and the memory cache and file cache are
updated, the alerts in turn uses the value from the memory cache.

Thanks,

Anita Jebaraj

Re: Review Request 51724: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor

2016-09-12 Thread Aravindan Vijayan



On Sept. 8, 2016, 6:13 p.m., Qin Liu wrote:
> > We don't have unit test whcih setup secure cluster so we should definitely 
> > test this on trunk build, which should be working now.
> > The changes overall look good just want to make sure we have done some 
> > manual testing before commit.
> 
> Qin Liu wrote:
> Hi Sid,
> I tried to test it on HDP2.5.0.0/trunk. but I was not able to bring the 
> metrics up. Do you have this experience? 
> But I will try to test it on HDP2.4/trunk.
> 
> Qin Liu wrote:
> I got latest version of trunk yesterday and setup to run AMS distributed 
> mode without kerberos enabled but I was unable to start monitor and 
> collector. I got:
> 
> 1) Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 152, in 
> AmsCollector().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 281, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 734, in restart
> self.status(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 61, in status
> check_service_status(name='collector')
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py",
>  line 36, in check_service_status
> env.set_params(status_params)
> NameError: global name 'env' is not defined >introduced by 
> AMBARI-17159
> 
> then I switched back to previous version of status.py and was able to 
> start monitors.
> 
> Then, I got:
> 2) Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 152, in 
> AmsCollector().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 284, in execute
> self.post_start()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 321, in post_start
> pid_files = self.get_pid_files()
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 73, in get_pid_files
> return status.get_collector_pid_files()
> AttributeError: 'module' object has no attribute 'get_collector_pid_files'
> 
> then, I hardcoded to "return status.get_collector_pid_files()" with 
> "return []" and was able to start collector with a warning. But monitor got " 
> Error sending metrics to server. [Errno -2] Name or service not known" then " 
> Metric collector host ['myhost.example.com'] was blacklisted." I didn't get a 
> chance to look into it (it might be caused by 
> status.get_collector_pid_files() "return []").
> 
> Any way, anyone saw above issues? Thank you!
> 
> Sid Wagle wrote:
> Hi Qin Liu, Can you please open a jira for this in Apache for the errors? 
> Will get this resovled ASAP.

Hi Qin Liu,

On trunk deploy, AMS components are in fact running. But the bug is that their 
statuses are not being reported to Ambari correctly. I have fixed it in 
https://reviews.apache.org/r/51711/. I will check it in today.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51724/#review148224
---


On Sept. 8, 2016, 5:27 p.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51724/
> ---
> 
> (Updated Sept. 8, 2016, 5:27 p.m.)
> 
> 
> Review request for Ambari, Di Li, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ()This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
> authenticate".
> 
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari 
> Metrics Collector (in AMS distributed mode) web-console will be Kerberos HTTP 
> SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics 
> Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> 
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: 
> http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671

Re: Review Request 51724: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor



> On Sept. 8, 2016, 6:13 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/spnego_kerberos_auth.py,
> >  line 25
> > 
> >
> > I do not see a "import kerberos" anywhere in Ambari deployed cluster 
> > under: /usr/lib/python2.6/site-packages/
> > 
> > Where does this dependency get fulfiled ?
> 
> Qin Liu wrote:
> It is in /usr/lib/python2.6/site-packages/ipalib/plugins on all of my 
> clusters. I will include ipa-python in ambari-metrics-assembly/pom.xml. 
> Thanks Sid!

This seems to be ipa only dependency, does addinig it as component build dep 
make sense?
What about MIT kereberos?


On Sept. 8, 2016, 6:13 p.m., Qin Liu wrote:
> > We don't have unit test whcih setup secure cluster so we should definitely 
> > test this on trunk build, which should be working now.
> > The changes overall look good just want to make sure we have done some 
> > manual testing before commit.
> 
> Qin Liu wrote:
> Hi Sid,
> I tried to test it on HDP2.5.0.0/trunk. but I was not able to bring the 
> metrics up. Do you have this experience? 
> But I will try to test it on HDP2.4/trunk.
> 
> Qin Liu wrote:
> I got latest version of trunk yesterday and setup to run AMS distributed 
> mode without kerberos enabled but I was unable to start monitor and 
> collector. I got:
> 
> 1) Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 152, in 
> AmsCollector().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 281, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 734, in restart
> self.status(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 61, in status
> check_service_status(name='collector')
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py",
>  line 36, in check_service_status
> env.set_params(status_params)
> NameError: global name 'env' is not defined >introduced by 
> AMBARI-17159
> 
> then I switched back to previous version of status.py and was able to 
> start monitors.
> 
> Then, I got:
> 2) Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 152, in 
> AmsCollector().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 284, in execute
> self.post_start()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 321, in post_start
> pid_files = self.get_pid_files()
>   File 
> "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
>  line 73, in get_pid_files
> return status.get_collector_pid_files()
> AttributeError: 'module' object has no attribute 'get_collector_pid_files'
> 
> then, I hardcoded to "return status.get_collector_pid_files()" with 
> "return []" and was able to start collector with a warning. But monitor got " 
> Error sending metrics to server. [Errno -2] Name or service not known" then " 
> Metric collector host ['myhost.example.com'] was blacklisted." I didn't get a 
> chance to look into it (it might be caused by 
> status.get_collector_pid_files() "return []").
> 
> Any way, anyone saw above issues? Thank you!

Hi Qin Liu, Can you please open a jira for this in Apache for the errors? Will 
get this resovled ASAP.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51724/#review148224
---


On Sept. 8, 2016, 5:27 p.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51724/
> ---
> 
> (Updated Sept. 8, 2016, 5:27 p.m.)
> 
> 
> Review request for Ambari, Di Li, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ()This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
> authenticate".
> 
> In a Kerberos enabled cluster with SPNEGO enabled on

Re: Review Request 51724: Add Kerberos HTTP SPNEGO authentication support to Ambari Metrics Monitor

2016-09-12 Thread Qin Liu



On Sept. 8, 2016, 6:13 p.m., Qin Liu wrote:
> > We don't have unit test whcih setup secure cluster so we should definitely 
> > test this on trunk build, which should be working now.
> > The changes overall look good just want to make sure we have done some 
> > manual testing before commit.
> 
> Qin Liu wrote:
> Hi Sid,
> I tried to test it on HDP2.5.0.0/trunk. but I was not able to bring the 
> metrics up. Do you have this experience? 
> But I will try to test it on HDP2.4/trunk.

I got latest version of trunk yesterday and setup to run AMS distributed mode 
without kerberos enabled but I was unable to start monitor and collector. I got:

1) Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
 line 152, in 
AmsCollector().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 281, in execute
method(env)
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 734, in restart
self.status(env)
  File 
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
 line 61, in status
check_service_status(name='collector')
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py",
 line 36, in check_service_status
env.set_params(status_params)
NameError: global name 'env' is not defined >introduced by AMBARI-17159

then I switched back to previous version of status.py and was able to start 
monitors.

Then, I got:
2) Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
 line 152, in 
AmsCollector().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 284, in execute
self.post_start()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 321, in post_start
pid_files = self.get_pid_files()
  File 
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
 line 73, in get_pid_files
return status.get_collector_pid_files()
AttributeError: 'module' object has no attribute 'get_collector_pid_files'

then, I hardcoded to "return status.get_collector_pid_files()" with "return []" 
and was able to start collector with a warning. But monitor got " Error sending 
metrics to server. [Errno -2] Name or service not known" then " Metric 
collector host ['myhost.example.com'] was blacklisted." I didn't get a chance 
to look into it (it might be caused by status.get_collector_pid_files() "return 
[]").

Any way, anyone saw above issues? Thank you!


- Qin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51724/#review148224
---


On Sept. 8, 2016, 5:27 p.m., Qin Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51724/
> ---
> 
> (Updated Sept. 8, 2016, 5:27 p.m.)
> 
> 
> Review request for Ambari, Di Li, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17898
> https://issues.apache.org/jira/browse/AMBARI-17898
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ()This is a subtask of AMBARI-14384 "Ambari Metrics doesn't use SPNEGO to 
> authenticate".
> 
> In a Kerberos enabled cluster with SPNEGO enabled on Hadoop APIs, Ambari 
> Metrics Collector (in AMS distributed mode) web-console will be Kerberos HTTP 
> SPNEGO enabled too. But Ambari Metrics Monitor, a client of Ambari Metrics 
> Collector, currently does not support Kerberos HTTP SPNEGO authentication.
> 
> /var/log/ambari-metrics-monitor/ambari-metrics-monitor.out:
> 2015-12-15 13:26:30,663 [INFO] emitter.py:101 - server: 
> http://metrics-collector:6188/ws/v1/timeline/metrics
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:84 - Error sending metrics to 
> server. HTTP Error 401: Authentication required
> 2015-12-15 13:26:30,671 [WARNING] emitter.py:90 - Retrying after 5 ...
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-host-monitoring/conf/unix/metric_monitor.ini 
> 3e5d861 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/config_reader.py
>  02f0ce3 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/emitter.py 
> 6997108 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/faked_kerberos.py
>  PRE-CREATION 
>   
>

Re: Review Request 51818: YAML Maps For Storm Are Not Being Escaped Correctly

2016-09-12 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51818/#review148582
---


Ship it!




Ship It!

- Robert Levas


On Sept. 12, 2016, 3:13 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51818/
> ---
> 
> (Updated Sept. 12, 2016, 3:13 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18366
> https://issues.apache.org/jira/browse/AMBARI-18366
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When attempting to add a custom {{storm-site}} property which is a YAML map 
> of maps, the {{storm.yaml}} file being generated is incorrectly escaping the 
> map. 
> 
> {code:title="Expected YAML"}
> nimbus.impersonation.acl:
> storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]
> {code}
> 
> {code:title="Actual YAML"}
> nimbus.impersonation.acl: 'storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]'
> {code}
> 
> - We should not be escaping YAML maps
> - YAML maps must being on a newline with no leading whitespace
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/yaml_utils.py bb05c8a 
>   ambari-server/src/test/python/TestYAMLUtils.py bdbb11f 
> 
> Diff: https://reviews.apache.org/r/51818/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1125
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services



> On Sept. 7, 2016, 11:48 a.m., Tim Thorpe wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java,
> >  line 1133
> > 
> >
> > This won't include extension services.  You would need to loop through 
> > all the ServiceInfo objects and check to see if they have repos.

Added a new patch that handles it.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review148002
---


On Sept. 12, 2016, 8:48 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 12, 2016, 8:48 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  7403dba 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> bc94104 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested
> - Wrote new unit tests
> - All unit tests passed except two which were failing in CI builds as well.
> 
> 
> Thanks,
> 
> Balázs Bence

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/
---

(Updated Sept. 12, 2016, 8:48 p.m.)


Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Fixed issues with stack inheritance and extension services.


Bugs: AMBARI-15538
https://issues.apache.org/jira/browse/AMBARI-15538


Repository: ambari


Description
---

Implementation contains the following things:
- On startup, add-on service repos are loaded into the stack model
- On startup, add-on service repos are merged into the VDF's downloaded by 
LatestRepoCallable
- On startup, if the is an existing cluster, it's repository version entity is 
potentially updated with new add-on service repos.
- Repository definitions contain two new optional fields: service_name and 
service_version. (null for stack repositories) 
- Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
- New unit tests


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
 7403dba 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 b1fd592 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 d20b1d7 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
 02fc2ec 
  ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
bc94104 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java 
3acc617 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
23b0218 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
 7bcd08b 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java 
29776ed 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
14ff9de 
  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
 50d6028 
  ambari-server/src/main/resources/version_definition.xsd bd49028 
  ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
 1d73ff3 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
 PRE-CREATION 
  
ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
 PRE-CREATION 
  ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
 PRE-CREATION 
  
ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
 PRE-CREATION 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
 61ea96b 
  
contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51238/diff/


Testing
---

- Manually tested
- Wrote new unit tests
- All unit tests passed except two which were failing in CI builds as well.


Thanks,

Balázs Bence Sári

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services



> On Aug. 29, 2016, 5:54 p.m., Jayush Luniya wrote:
> > contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml,
> >  line 24
> > 
> >
> > A management pack can have one than one custom services. For example 
> > for HAWQ and PXF release artifacts for both the services are in the same 
> > repo.
> 
> Balázs Bence Sári wrote:
> Service name is removed

Also, if two services declare exactly the same repo (all fields are the same), 
the repo will be loaded and not treated as duplicate. If multiple 
extension/mpack services are delivered together and use the same repo, all 
services should declare the repo the same way.


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147167
---


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services



> On Aug. 29, 2016, 6:14 p.m., Jayush Luniya wrote:
> > contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml,
> >  line 24
> > 
> >
> > Is it mandatory to have servicename and serviceversion when defining a 
> > repoinfo at service-level? If, so can we add validations for it?
> 
> Balázs Bence Sári wrote:
> How about logginng a warning and not loading the repo in this case?

Deleted service_name and service_version


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147175
---


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> f90ccce 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested

Review Request 51822: Add Ambari configuration options to support Kerberos token authentication

2016-09-12 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51822/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-18365
https://issues.apache.org/jira/browse/AMBARI-18365


Repository: ambari


Description
---

Add the followng Ambari configuration options to support Kerberos token 
authentication

- `authentication.kerberos.enabled`
-- Determines whether to use Kerberos (SPNEGO) authentication when connecting 
Ambari:  {{true}} to enable this feature; {{false}}, otherwise
- `authentication.kerberos.spnego.principal`
-- The Kerberos principal name to use when verifying user-supplied Kerberos 
tokens for authentication via SPNEGO
- `authentication.kerberos.spnego.keytab.file`
-- The Kerberos keytab file to use when verifying user-supplied Kerberos tokens 
for authentication via SPNEGO
- `authentication.kerberos.user.types`
-- A comma-delimited (ordered) list of preferred user types to use when finding 
the Ambari user account for the user-supplied Kerberos identity during 
authentication via SPNEGO
- `authentication.kerberos.auth_to_local.rules`
-- The auth-to-local rules set to use when translating a user's principal name 
to a local user name during authentication via SPNEGO.

NOTE: These properties are in the {{ambari.properties}} file since this feature 
may be enabled whether the rest of the cluster has Kerberos enabled or not. 

See https://issues.apache.org/jira/browse/AMBARI-18364 for an overview 
requiring the need for this patch.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 ee73b8d 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationProperties.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 f429a36 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authentication/kerberos/AmbariKerberosAuthenticationPropertiesTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/51822/diff/


Testing
---

Manually tested...

# Local test results: PENDING

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 51705: Password in the configurations.json file in the ambari-agent cache is not encrypted



> On Sept. 8, 2016, 5:42 p.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py, line 103
> > 
> >
> > This will cause problems since you're storing the obfuscated 
> > properties. If a command needed the property from the in-memory cache, it 
> > would not have the correct value anymore.
> > 
> > Perhaps this approach is flawed in that there's no credential store 
> > being used for the in-memory passwords.

Thanks for the catch, I updated the code to mask passwords only while dumping 
in the file cache, but missed to remove the code. Was not able to capture this 
in the testing. Thank you.


> On Sept. 8, 2016, 5:42 p.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py, line 174
> > 
> >
> > `i` is not a very good name here; maybe something a little clearer.

Updated the code


> On Sept. 8, 2016, 5:42 p.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py, lines 
> > 179-181
> > 
> >
> > That's a lot of iterating - over the entire config dictionary per-key 
> > that needs replacing - also you never exit once you have a hit.
> > 
> > This function needs to be cleaned up for better performance.

Updated the code with a different approach


> On Sept. 8, 2016, 5:42 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java,
> >  lines 182-183
> > 
> >
> > What about status commands - those send down configurations too?

The configurations in the status commands are not dumped into the 
configurations.json file. The values are dumped only for alert definition and 
execution commands.


> On Sept. 8, 2016, 5:42 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java,
> >  lines 252-275
> > 
> >
> > This seems like it should be cached once after calculation for the 
> > cluster so that it doesn't need to keep generating it over and over.

Updated the code with a different approach. Instead of getting the value from 
the Ambari Metainfo, I am getting the value from the config based on the 
desired Tags. Similar approach is being used for getting the required config 
properties for the response.


> On Sept. 8, 2016, 5:42 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java,
> >  line 270
> > 
> >
> > You call this property "password config types" and bind a name to a 
> > type, but then never check the type for PASSWORD in python. I'd say either 
> > make this a flat list of password-protected fields or enhance the python 
> > code to extract properties of type PASSWORD to check on.

Yes you are right, key-value pair is not required in this case. I have updated 
the code


- Anita


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51705/#review148212
---


On Sept. 12, 2016, 8:20 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51705/
> ---
> 
> (Updated Sept. 12, 2016, 8:20 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-18334
> https://issues.apache.org/jira/browse/AMBARI-18334
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The configurations.json file loaded in the ambari-agent cache located at 
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.
> 
> Approach:
> 
> The password_config_type is included in the heartbeat response for alert 
> definition command and execution command, for which the values are dumped 
> into the json file. The password_config_type contains the information on 
> which properties in the configurations has the propertyType password. Based 
> on the response, the json is parsed and the password values are masked before 
> dumping it into the configurations.json file.
>

Re: Review Request 51705: Password in the configurations.json file in the ambari-agent cache is not encrypted



> On Sept. 8, 2016, 3:25 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java,
> >  lines 252-257
> > 
> >
> > Will this be a performance issue?  On large clusters this can be a lot 
> > of iterating for alert and execution commands.

I have changed the approach for finding the property Types. Instead of getting 
the value from the Ambari Metainfo, I am getting the value from the Config 
based on the desired Tags. Similar approach is being used for getting the 
required config properties for the response.


- Anita


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51705/#review148191
---


On Sept. 12, 2016, 8:20 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51705/
> ---
> 
> (Updated Sept. 12, 2016, 8:20 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-18334
> https://issues.apache.org/jira/browse/AMBARI-18334
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The configurations.json file loaded in the ambari-agent cache located at 
> /var/lib/ambari-agent/cache/cluster_configuration contains password details 
> in plaintext (Ex: ssl.client.keystore.password,ssl.client.truststore.password 
> etc.). The values are loaded both in the memory cache and file cache, the 
> file seems to be used only for debugging purposes, so it would be a better 
> approach to mask the passwords in the file.
> 
> Approach:
> 
> The password_config_type is included in the heartbeat response for alert 
> definition command and execution command, for which the values are dumped 
> into the json file. The password_config_type contains the information on 
> which properties in the configurations has the propertyType password. Based 
> on the response, the json is parsed and the password values are masked before 
> dumping it into the configurations.json file.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ClusterConfiguration.py 72b87be 
>   ambari-agent/src/test/python/ambari_agent/TestAlerts.py 2bddc43 
>   ambari-agent/src/test/python/ambari_agent/TestClusterConfigurationCache.py 
> a418f6d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java
>  0562c15 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java
>  4d2e048 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  29737ee 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
> 70c24f9 
> 
> Diff: https://reviews.apache.org/r/51705/diff/
> 
> 
> Testing
> ---
> 
> Updated the test cases.
> Ran mvn test.
> 
> Manually tested by setting up a cluster, the password fields in the 
> configurations.json is masked. During testing, everytime the ambari agent is 
> restarted, it registers with the server and the memory cache and file cache 
> are updated, the alerts in turn uses the value from the memory cache.
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 51705: Password in the configurations.json file in the ambari-agent cache is not encrypted

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51705/
---

(Updated Sept. 12, 2016, 8:20 p.m.)

Review request for Ambari, Di Li, Jonathan Hurley, and Nate Cole.

Bugs: AMBARI-18334
https://issues.apache.org/jira/browse/AMBARI-18334

Repository: ambari

Description
---

Approach:

Diffs (updated)
-

ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java
0562c15

ambari-server/src/main/java/org/apache/ambari/server/agent/AlertDefinitionCommand.java
4d2e048

ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
29737ee
ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
70c24f9

Diff: https://reviews.apache.org/r/51705/diff/

Testing
---

Updated the test cases.
Ran mvn test.

Thanks,

Anita Jebaraj

Re: Review Request 51818: YAML Maps For Storm Are Not Being Escaped Correctly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51818/#review148562
---


Ship it!




Ship It!

- Alejandro Fernandez


On Sept. 12, 2016, 7:13 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51818/
> ---
> 
> (Updated Sept. 12, 2016, 7:13 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18366
> https://issues.apache.org/jira/browse/AMBARI-18366
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When attempting to add a custom {{storm-site}} property which is a YAML map 
> of maps, the {{storm.yaml}} file being generated is incorrectly escaping the 
> map. 
> 
> {code:title="Expected YAML"}
> nimbus.impersonation.acl:
> storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]
> {code}
> 
> {code:title="Actual YAML"}
> nimbus.impersonation.acl: 'storm:
>   hosts:
> [c6401.ambari.apache.org, c6402.ambari.apache.org]
>   groups:
> [hadoop, foo]'
> {code}
> 
> - We should not be escaping YAML maps
> - YAML maps must being on a newline with no leading whitespace
> 
> 
> Diffs
> -
> 
>   ambari-common/src/main/python/ambari_commons/yaml_utils.py bb05c8a 
>   ambari-server/src/test/python/TestYAMLUtils.py bdbb11f 
> 
> Diff: https://reviews.apache.org/r/51818/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1125
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 51818: YAML Maps For Storm Are Not Being Escaped Correctly

2016-09-12 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51818/
---

Review request for Ambari, Nate Cole and Robert Levas.


Bugs: AMBARI-18366
https://issues.apache.org/jira/browse/AMBARI-18366


Repository: ambari


Description
---

When attempting to add a custom {{storm-site}} property which is a YAML map of 
maps, the {{storm.yaml}} file being generated is incorrectly escaping the map. 

{code:title="Expected YAML"}
nimbus.impersonation.acl:
storm:
  hosts:
[c6401.ambari.apache.org, c6402.ambari.apache.org]
  groups:
[hadoop, foo]
{code}

{code:title="Actual YAML"}
nimbus.impersonation.acl: 'storm:
  hosts:
[c6401.ambari.apache.org, c6402.ambari.apache.org]
  groups:
[hadoop, foo]'
{code}

- We should not be escaping YAML maps
- YAML maps must being on a newline with no leading whitespace


Diffs
-

  ambari-common/src/main/python/ambari_commons/yaml_utils.py bb05c8a 
  ambari-server/src/test/python/TestYAMLUtils.py bdbb11f 

Diff: https://reviews.apache.org/r/51818/diff/


Testing
---

--
Total run:1125
Total errors:0
Total failures:0
OK


Thanks,

Jonathan Hurley

Re: Review Request 51801: Get LDAP tests to run in concurrent forked JVMs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51801/#review148552
---


Ship it!




Ship It!

- Oliver Szabo


On Sept. 12, 2016, 1:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51801/
> ---
> 
> (Updated Sept. 12, 2016, 1:01 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Sumit Mohanty, Sebastian Toader, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-18359
> https://issues.apache.org/jira/browse/AMBARI-18359
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
> increasing forkCount to speed up the unit tests.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java
>  1a18ff2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDuplicateUserTest.java
>  c0086d6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java
>  779b0a2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  1716c97 
> 
> Diff: https://reviews.apache.org/r/51801/diff/
> 
> 
> Testing
> ---
> 
> * Added forkCount with various values (1, 2, 4, 8, 16) to 
> `maven-surefire-plugin` configuration in `ambari-server/pom.xml`, eg.:
> ```
>   4
>   false
> ```
>  * Ran `mvn -f ambari-server/pom.xml test -DskipPythonTests -Drat.skip 
> -Dtest='*Ldap*Test'`
>  * Verified test failures/errors that occur without the patch are resolved
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 51711: AMBARI-17458 : Add support for distributed collector to Ambari REST API.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51711/#review148543
---


Ship it!




Ship It!

- Sid Wagle


On Sept. 11, 2016, 2:10 a.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51711/
> ---
> 
> (Updated Sept. 11, 2016, 2:10 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17458
> https://issues.apache.org/jira/browse/AMBARI-17458
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support Multiple collectors in Ambari Server, managed through a separate 
> MetricsHAManager module. Round robin scheduling with 5mins time slice is used 
> for toggling between available collectors. Ambari event framework is used to 
> notify the MetricsHAManager about a down collector.
> 
> This patch also fixes the bug in reporting AMS components' process status.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  9da6fd4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1d82928 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  b6cbed5 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStatusHelper.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsCollectorHAClusterState.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/MetricsCollectorHAManager.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
>  9e81df4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSReportPropertyProvider.java
>  3688742 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/MetricsRequestHelper.java
>  1df0f6a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/cache/TimelineMetricCache.java
>  b5fe05e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 912c441 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/MetricsCollectorHostDownEvent.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py
>  2c7119b 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_grafana.py
>  747a687 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py
>  d53c1fc 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py
>  14af3ad 
> 
> Diff: https://reviews.apache.org/r/51711/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 51813: Modify the AMS stack scripts to support distributed collector


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51813/#review148533
---




ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
 (line 42)


Aggree that 30 is too high but 5 is too low, how about 10 ?



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
 (line 45)


I think this should be 10 secs linke server sice default. Rest of the 
settings make sense.



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
 (line 76)


Can we do this in parallel?



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
 (line 153)


We should log the same output as the exception since plain old message 
would not be meaningful in the log.


- Sid Wagle


On Sept. 12, 2016, 3:59 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51813/
> ---
> 
> (Updated Sept. 12, 2016, 3:59 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-17457
> https://issues.apache.org/jira/browse/AMBARI-17457
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix service check to andle AMS HA
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
>  ddd3e42 
> 
> Diff: https://reviews.apache.org/r/51813/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 51814: Update sinks to read multiple collector hostnames from configs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51814/#review148527
---




ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 (line 61)


Shouldn't the constructor be modified for getting multiple hosts? not sure 
what this returns null for everything but looks like the author of the sink 
left this a TODO for later.



ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
 (line 49)


Ditto here, if there is no such config we should add it.



ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 (line 335)


Null check seems redundant maybe reorder as StringUtil.isEmpty followed by 
trim.



ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 (line 473)


Effectively with this change user cannot configure separate port on 
different hosts. Ambari allows this by the means of Config groups. Is this 
change necessary / practical? I am not flagging this as an issue since 
technically user can change the port for both collectors if there is a conflict 
on one of the hosts but wanted to make sure we have a reason to seggregate this 
info.


- Sid Wagle


On Sept. 12, 2016, 4:09 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51814/
> ---
> 
> (Updated Sept. 12, 2016, 4:09 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-18362
> https://issues.apache.org/jira/browse/AMBARI-18362
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update sinks to read multiple collector hostnames from configs
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
>  da61d83 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
>  85ea69d 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
>  426eb42 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
>  7fadeb2 
>   
> ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
>  ccaa574 
>   
> ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java
>  1b36e9a 
>   
> ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
>  c534121 
>   
> ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
>  ea7f72d 
>   
> ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
>  9c16564 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  73381d9 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  0d3b770 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  9082e70 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  f6531c8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
>  4618618 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
>  4c111f1 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/templates/hadoop-metrics2-accumulo.properties.j2
>  3cd535e 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/hadoop-metrics2-hbase.properties.j2
>  586cd47 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py
>  b839eb8 
>   
> ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
>  d3a9294 
>   
>

Review Request 51815: Introduce conditional dependencies in stack defition to handle blueprint validation gracefully

2016-09-12 Thread Amruta Borkar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51815/
---

Review request for Ambari, Di Li, Juanjo  Marron, Laszlo Puskas, and Robert 
Nettleton.


Bugs: AMBARI-18355
https://issues.apache.org/jira/browse/AMBARI-18355


Repository: ambari


Description
---

Currently stack definitions do not list conditional dependencies, adding those 
to the stack definitions would make it easy to validate errors in case of 
blueprint deployment. Please refer to document attached to Jira


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/DependencyConditionInfo.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java 
e3db662 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java
 a5f33ff 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
65d166a 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java
 b1de8ef 

Diff: https://reviews.apache.org/r/51815/diff/


Testing
---

Written Junit test cases. Perfomred manual testing to check the Namenode HA 
component dependency. Was able to proceed with the installation for valid 
blueprint. and got validation error message while registering blueprint when 
the Blueprint did not satisfy the conditional dependencies.


Thanks,

Amruta Borkar

Re: Review Request 51801: Get LDAP tests to run in concurrent forked JVMs


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51801/#review148520
---


Ship it!




Fantastic!

- Alejandro Fernandez


On Sept. 12, 2016, 1:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51801/
> ---
> 
> (Updated Sept. 12, 2016, 1:01 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Sumit Mohanty, Sebastian Toader, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-18359
> https://issues.apache.org/jira/browse/AMBARI-18359
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
> increasing forkCount to speed up the unit tests.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java
>  1a18ff2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDuplicateUserTest.java
>  c0086d6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java
>  779b0a2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  1716c97 
> 
> Diff: https://reviews.apache.org/r/51801/diff/
> 
> 
> Testing
> ---
> 
> * Added forkCount with various values (1, 2, 4, 8, 16) to 
> `maven-surefire-plugin` configuration in `ambari-server/pom.xml`, eg.:
> ```
>   4
>   false
> ```
>  * Ran `mvn -f ambari-server/pom.xml test -DskipPythonTests -Drat.skip 
> -Dtest='*Ldap*Test'`
>  * Verified test failures/errors that occur without the patch are resolved
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 51811: All classes recompiled due to Maven bug, even if none changed

2016-09-12 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51811/#review148513
---


Ship it!




Ship It!

- Jayush Luniya


On Sept. 12, 2016, 3:56 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51811/
> ---
> 
> (Updated Sept. 12, 2016, 3:56 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Laszlo Puskas, Nate Cole, Oliver 
> Szabo, Sebastian Toader, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18361
> https://issues.apache.org/jira/browse/AMBARI-18361
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix:
> 
> 1. Upgrade to maven-compiler-plugin from 3.0 (because 
> https://issues.apache.org/jira/browse/MCOMPILER-187 was only fixed in 3.1+) 
> to 3.2 (because it is already used by ambari-logsearch and ambari-metrics)
> 
> 2. Set useIncrementalCompilation flag to false (defaults to true) to work 
> around https://issues.apache.org/jira/browse/MCOMPILER-209
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml 45d3ce4 
> 
> Diff: https://reviews.apache.org/r/51811/diff/
> 
> 
> Testing
> ---
> 
> 1. Verify Maven reports "Nothing to compile - all classes are up to date" if 
> no class was changed.
> 
> ```
> mvn -pl ambari-server compile
> mvn -pl ambari-server compile
> ```
> 
> 2. Verify Maven only reports compilation of 3 source files after 3 source 
> files updated.
> 
> ```
> mvn -pl ambari-server compile
> find ambari-server/src/main/java -type f | head -3 | xargs touch
> mvn -pl ambari-server compile
> ```
> 
> 3. Verify Maven compiles all Java sources after removing class files.
> 
> ```
> mvn -pl ambari-server compile
> find ambari-server -name '*.class' | xargs rm
> mvn -pl ambari-server compile
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 51811: All classes recompiled due to Maven bug, even if none changed

2016-09-12 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51811/#review148510
---


Ship it!




Ship It!

- Sebastian Toader


On Sept. 12, 2016, 5:56 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51811/
> ---
> 
> (Updated Sept. 12, 2016, 5:56 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Laszlo Puskas, Nate Cole, Oliver 
> Szabo, Sebastian Toader, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18361
> https://issues.apache.org/jira/browse/AMBARI-18361
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix:
> 
> 1. Upgrade to maven-compiler-plugin from 3.0 (because 
> https://issues.apache.org/jira/browse/MCOMPILER-187 was only fixed in 3.1+) 
> to 3.2 (because it is already used by ambari-logsearch and ambari-metrics)
> 
> 2. Set useIncrementalCompilation flag to false (defaults to true) to work 
> around https://issues.apache.org/jira/browse/MCOMPILER-209
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml 45d3ce4 
> 
> Diff: https://reviews.apache.org/r/51811/diff/
> 
> 
> Testing
> ---
> 
> 1. Verify Maven reports "Nothing to compile - all classes are up to date" if 
> no class was changed.
> 
> ```
> mvn -pl ambari-server compile
> mvn -pl ambari-server compile
> ```
> 
> 2. Verify Maven only reports compilation of 3 source files after 3 source 
> files updated.
> 
> ```
> mvn -pl ambari-server compile
> find ambari-server/src/main/java -type f | head -3 | xargs touch
> mvn -pl ambari-server compile
> ```
> 
> 3. Verify Maven compiles all Java sources after removing class files.
> 
> ```
> mvn -pl ambari-server compile
> find ambari-server -name '*.class' | xargs rm
> mvn -pl ambari-server compile
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 51238: Implementation for AMBARI-15538: Support service-specific repo for add-on services



> On Aug. 29, 2016, 5:27 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java,
> >  line 562
> > 
> >
> > Weird that it's setOperatingSystems() but getting RepositoryInfos()

Renamed method to original name: serializeOperatingSystems()


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51238/#review147153
---


On Sept. 5, 2016, 1:26 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51238/
> ---
> 
> (Updated Sept. 5, 2016, 1:26 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Nate Cole, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-15538
> https://issues.apache.org/jira/browse/AMBARI-15538
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Implementation contains the following things:
> - On startup, add-on service repos are loaded into the stack model
> - On startup, add-on service repos are merged into the VDF's downloaded by 
> LatestRepoCallable
> - On startup, if the is an existing cluster, it's repository version entity 
> is potentially updated with new add-on service repos.
> - Repository definitions contain two new optional fields: service_name and 
> service_version. (null for stack repositories) 
> - Small changes on the Mictrosoft-R mpack (supoorts HDP-2.4 and 2.5)
> - New unit tests
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  9f7419c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  b1fd592 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  d20b1d7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/VersionDefinitionResourceProvider.java
>  02fc2ec 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/RepoUtil.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
>  3acc617 
>   ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
> 23b0218 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
>  7bcd08b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartup.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  29776ed 
>   ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
> 14ff9de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  50d6028 
>   ambari-server/src/main/resources/version_definition.xsd bd49028 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/RepoUtilTest.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java
>  1d73ff3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/StackModuleTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/common-services/ADDON/1.0/configuration/addon-env.xml
>  PRE-CREATION 
>   ambari-server/src/test/resources/common-services/ADDON/1.0/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/test/resources/org/apache/ambari/server/stack/UpdateActiveRepoVersionOnStartupTest_initialRepos.json
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/resources/stacks_with_common_services/HDP/0.2/services/ADDON/repos/repoinfo.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/configuration/microsoft-r-env.xml
>  PRE-CREATION 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/common-services/MICROSOFT_R/8.0.0/package/scripts/microsoft_r.py
>  61ea96b 
>   
> contrib/management-packs/microsoft-r_mpack/src/main/resources/custom-services/MICROSOFT_R/8.0.0/repos/repoinfo.xml
>  PRE-CREATION 
>   contrib/management-packs/microsoft-r_mpack/src/main/resources/mpack.json 
> f90ccce 
> 
> Diff: https://reviews.apache.org/r/51238/diff/
> 
> 
> Testing
> ---
> 
> - Manually tested
> - Wrote new unit tests
> - All unit tests passed except two which were failing in CI builds as well.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 51811: All classes recompiled due to Maven bug, even if none changed

2016-09-12 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51811/#review148503
---


Ship it!




Ship It!

- Nate Cole


On Sept. 12, 2016, 11:56 a.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51811/
> ---
> 
> (Updated Sept. 12, 2016, 11:56 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Laszlo Puskas, Nate Cole, Oliver 
> Szabo, Sebastian Toader, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18361
> https://issues.apache.org/jira/browse/AMBARI-18361
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix:
> 
> 1. Upgrade to maven-compiler-plugin from 3.0 (because 
> https://issues.apache.org/jira/browse/MCOMPILER-187 was only fixed in 3.1+) 
> to 3.2 (because it is already used by ambari-logsearch and ambari-metrics)
> 
> 2. Set useIncrementalCompilation flag to false (defaults to true) to work 
> around https://issues.apache.org/jira/browse/MCOMPILER-209
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml 45d3ce4 
> 
> Diff: https://reviews.apache.org/r/51811/diff/
> 
> 
> Testing
> ---
> 
> 1. Verify Maven reports "Nothing to compile - all classes are up to date" if 
> no class was changed.
> 
> ```
> mvn -pl ambari-server compile
> mvn -pl ambari-server compile
> ```
> 
> 2. Verify Maven only reports compilation of 3 source files after 3 source 
> files updated.
> 
> ```
> mvn -pl ambari-server compile
> find ambari-server/src/main/java -type f | head -3 | xargs touch
> mvn -pl ambari-server compile
> ```
> 
> 3. Verify Maven compiles all Java sources after removing class files.
> 
> ```
> mvn -pl ambari-server compile
> find ambari-server -name '*.class' | xargs rm
> mvn -pl ambari-server compile
> ```
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Review Request 51814: Update sinks to read multiple collector hostnames from configs

2016-09-12 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51814/
---

Review request for Ambari, Aravindan Vijayan and Sid Wagle.


Bugs: AMBARI-18362
https://issues.apache.org/jira/browse/AMBARI-18362


Repository: ambari


Description
---

Update sinks to read multiple collector hostnames from configs


Diffs
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 da61d83 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrAmsClient.java
 85ea69d 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 426eb42 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricCollectorHATest.java
 7fadeb2 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/cache/HandleConnectExceptionTest.java
 ccaa574 
  
ambari-metrics/ambari-metrics-flume-sink/src/main/java/org/apache/hadoop/metrics2/sink/flume/FlumeTimelineMetricsSink.java
 1b36e9a 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/main/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSink.java
 c534121 
  
ambari-metrics/ambari-metrics-hadoop-sink/src/test/java/org/apache/hadoop/metrics2/sink/timeline/HadoopTimelineMetricsSinkTest.java
 ea7f72d 
  
ambari-metrics/ambari-metrics-kafka-sink/src/main/java/org/apache/hadoop/metrics2/sink/kafka/KafkaTimelineMetricsReporter.java
 9c16564 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 73381d9 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 0d3b770 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 9082e70 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 f6531c8 
  
ambari-server/src/main/java/org/apache/ambari/server/metrics/system/impl/AmbariMetricSinkImpl.java
 4618618 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py
 4c111f1 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/templates/hadoop-metrics2-accumulo.properties.j2
 3cd535e 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/templates/hadoop-metrics2-hbase.properties.j2
 586cd47 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py
 b839eb8 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/params.py
 d3a9294 
  
ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/templates/flume-metrics2.properties.j2
 b960296 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 49b684b 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2
 e62ce9e 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/templates/hadoop-metrics2-hbase.properties-GANGLIA-RS.j2
 818 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 56e3ce6 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-hivemetastore.properties.j2
 88be81b 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-hiveserver2.properties.j2
 3d71867 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-llapdaemon.j2
 ba44af5 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/templates/hadoop-metrics2-llaptaskscheduler.j2
 77f128b 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/configuration/kafka-broker.xml
 ab4d701 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 6cc85f4 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
 f631ac9 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 f96aeeb 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/config.yaml.j2
 a9760cb 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
 1f0875f 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
 45eab2f

Re: Review Request 51806: ambari-agent check for unset variables (AMBARI-18317)

2016-09-12 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51806/#review148502
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Sept. 12, 2016, 4:17 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51806/
> ---
> 
> (Updated Sept. 12, 2016, 4:17 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18360
> https://issues.apache.org/jira/browse/AMBARI-18360
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Reported in community:
> 
> 
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent 978e5c6 
>   ambari-server/sbin/ambari-server 8064c45 
> 
> Diff: https://reviews.apache.org/r/51806/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 51813: Modify the AMS stack scripts to support distributed collector

2016-09-12 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51813/
---

Review request for Ambari, Aravindan Vijayan and Sid Wagle.


Bugs: AMBARI-17457
https://issues.apache.org/jira/browse/AMBARI-17457


Repository: ambari


Description
---

Fix service check to andle AMS HA


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
 ddd3e42 

Diff: https://reviews.apache.org/r/51813/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Review Request 51811: All classes recompiled due to Maven bug, even if none changed

2016-09-12 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51811/
---

Review request for Ambari, Jayush Luniya, Laszlo Puskas, Oliver Szabo, and 
Sebastian Toader.


Bugs: AMBARI-18361
https://issues.apache.org/jira/browse/AMBARI-18361


Repository: ambari


Description
---

Fix:

1. Upgrade to maven-compiler-plugin from 3.0 (because 
https://issues.apache.org/jira/browse/MCOMPILER-187 was only fixed in 3.1+) to 
3.2 (because it is already used by ambari-logsearch and ambari-metrics)

2. Set useIncrementalCompilation flag to false (defaults to true) to work 
around https://issues.apache.org/jira/browse/MCOMPILER-209


Diffs
-

  ambari-server/pom.xml 45d3ce4 

Diff: https://reviews.apache.org/r/51811/diff/


Testing
---

1. Verify Maven reports "Nothing to compile - all classes are up to date" if no 
class was changed.

```
mvn -pl ambari-server compile
mvn -pl ambari-server compile
```

2. Verify Maven only reports compilation of 3 source files after 3 source files 
updated.

```
mvn -pl ambari-server compile
find ambari-server/src/main/java -type f | head -3 | xargs touch
mvn -pl ambari-server compile
```

3. Verify Maven compiles all Java sources after removing class files.

```
mvn -pl ambari-server compile
find ambari-server -name '*.class' | xargs rm
mvn -pl ambari-server compile
```


Thanks,

Attila Doroszlai

Re: Review Request 51762: Declare URL uniqueness in repoinfo.xml

2016-09-12 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51762/#review148499
---


Ship it!




Ship It!

- Nate Cole


On Sept. 9, 2016, 1:07 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51762/
> ---
> 
> (Updated Sept. 9, 2016, 1:07 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18350
> https://issues.apache.org/jira/browse/AMBARI-18350
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Repository info for a stack can contain any number of repo definitions.  For 
> HDP, there are two repos used (HDP and HDP-UTILS).  When checking 
> repositories for duplicate URL, the code does this:
> {noformat}
> if (! repositoryEntity.getRepositoryId().startsWith("HDP-UTILS") &&  // 
> HDP-UTILS is shared between repo versions
> ! existingRepoVersion.getId().equals(repositoryVersion.getId())) { // Allow 
> modifying already defined repo version
> existingRepoUrls.add(repositoryEntity.getBaseUrl());
> }
> {noformat}
> 
> We should make the unique declaration directly for the repository so authors 
> don't have to guess.  This entails:
> * Change {{repoinfo.xml}} to support a {{}} element within {{}}
> * {{}} should apply to, say, versions of HDP that change with each 
> release.  Absence of said element implies {{false}} for backward 
> compatibility.
> * Change {{RepositoryVersionResourceProvider.validateRepositoryVersion(...)}} 
> to use this boolean for uniqueness check.
> * Update the {{version_definition.xsd}} to include this new element.
> 
> 
> Diffs
> -
> 
>   ambari-funtest/src/test/resources/stacks/HDP/0.1/repos/repoinfo.xml 1ea1809 
>   ambari-funtest/src/test/resources/stacks/HDP/0.2/repos/repoinfo.xml c0095b5 
>   ambari-funtest/src/test/resources/stacks/HDP/1.2.0/repos/repoinfo.xml 
> 04613ad 
>   ambari-funtest/src/test/resources/stacks/HDP/1.3.0/repos/repoinfo.xml 
> ec0be8f 
>   ambari-funtest/src/test/resources/stacks/HDP/1.3.1/repos/repoinfo.xml 
> ed1aed8 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.1/repos/repoinfo.xml 
> eb026a7 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.5/repos/repoinfo.xml 
> 91f0f02 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.6.1/repos/repoinfo.xml 
> 2739c5c 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.6/repos/repoinfo.xml 
> 2739c5c 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.7/repos/repoinfo.xml 
> 8ebea21 
>   
> ambari-funtest/src/test/resources/stacks/HDP/2.0.7/services/HBASE/metainfo.xml
>  e3ebea1 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.8/repos/repoinfo.xml 
> f8542d0 
>   ambari-funtest/src/test/resources/stacks/HDP/2.1.1/repos/repoinfo.xml 
> 9d8a232 
>   ambari-funtest/src/test/resources/stacks/HDP/2.2.0.ECS/repos/repoinfo.xml 
> 6c2f99a 
>   ambari-funtest/src/test/resources/stacks/HDP/2.2.0/repos/repoinfo.xml 
> 2a939f1 
>   ambari-funtest/src/test/resources/stacks/OTHER/1.0/repos/repoinfo.xml 
> 9d8a232 
>   ambari-funtest/src/test/resources/stacks/OTHER/2.0/repos/repoinfo.xml 
> 9d8a232 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3203539 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  e440460 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  8dc2ec9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  83a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  d26b834 
>   ambari-server/src/main/resources/stacks/BIGTOP/0.8/repos/repoinfo.xml 
> 6a385c4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6.GlusterFS/repos/repoinfo.xml
>  9b62568 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/repos/repoinfo.xml 
> 9855074 
>   ambari-server/src/main/resources/stacks/HDP/2.0/repos/repoinfo.xml a5d8a51 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/repos/repoinfo.xml 
> cfee6f5 
>   ambari-server/src/main/resources/stacks/HDP/2.1/repos/repoinfo.xml 938f627 
>   ambari-server/src/main/resources/stacks/HDP/2.2/repos/repoinfo.xml dbf8506 
>   ambari-server/src/main/resources/stacks/HDP/2.3.ECS/repos/repoinfo.xml 
> b44cca5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/repos/repoinfo.xml 
> 24b05b0 
>

Re: Review Request 51762: Declare URL uniqueness in repoinfo.xml

2016-09-12 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51762/#review148497
---


Ship it!




Ship It!

- Jonathan Hurley


On Sept. 9, 2016, 1:07 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51762/
> ---
> 
> (Updated Sept. 9, 2016, 1:07 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18350
> https://issues.apache.org/jira/browse/AMBARI-18350
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Repository info for a stack can contain any number of repo definitions.  For 
> HDP, there are two repos used (HDP and HDP-UTILS).  When checking 
> repositories for duplicate URL, the code does this:
> {noformat}
> if (! repositoryEntity.getRepositoryId().startsWith("HDP-UTILS") &&  // 
> HDP-UTILS is shared between repo versions
> ! existingRepoVersion.getId().equals(repositoryVersion.getId())) { // Allow 
> modifying already defined repo version
> existingRepoUrls.add(repositoryEntity.getBaseUrl());
> }
> {noformat}
> 
> We should make the unique declaration directly for the repository so authors 
> don't have to guess.  This entails:
> * Change {{repoinfo.xml}} to support a {{}} element within {{}}
> * {{}} should apply to, say, versions of HDP that change with each 
> release.  Absence of said element implies {{false}} for backward 
> compatibility.
> * Change {{RepositoryVersionResourceProvider.validateRepositoryVersion(...)}} 
> to use this boolean for uniqueness check.
> * Update the {{version_definition.xsd}} to include this new element.
> 
> 
> Diffs
> -
> 
>   ambari-funtest/src/test/resources/stacks/HDP/0.1/repos/repoinfo.xml 1ea1809 
>   ambari-funtest/src/test/resources/stacks/HDP/0.2/repos/repoinfo.xml c0095b5 
>   ambari-funtest/src/test/resources/stacks/HDP/1.2.0/repos/repoinfo.xml 
> 04613ad 
>   ambari-funtest/src/test/resources/stacks/HDP/1.3.0/repos/repoinfo.xml 
> ec0be8f 
>   ambari-funtest/src/test/resources/stacks/HDP/1.3.1/repos/repoinfo.xml 
> ed1aed8 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.1/repos/repoinfo.xml 
> eb026a7 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.5/repos/repoinfo.xml 
> 91f0f02 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.6.1/repos/repoinfo.xml 
> 2739c5c 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.6/repos/repoinfo.xml 
> 2739c5c 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.7/repos/repoinfo.xml 
> 8ebea21 
>   
> ambari-funtest/src/test/resources/stacks/HDP/2.0.7/services/HBASE/metainfo.xml
>  e3ebea1 
>   ambari-funtest/src/test/resources/stacks/HDP/2.0.8/repos/repoinfo.xml 
> f8542d0 
>   ambari-funtest/src/test/resources/stacks/HDP/2.1.1/repos/repoinfo.xml 
> 9d8a232 
>   ambari-funtest/src/test/resources/stacks/HDP/2.2.0.ECS/repos/repoinfo.xml 
> 6c2f99a 
>   ambari-funtest/src/test/resources/stacks/HDP/2.2.0/repos/repoinfo.xml 
> 2a939f1 
>   ambari-funtest/src/test/resources/stacks/OTHER/1.0/repos/repoinfo.xml 
> 9d8a232 
>   ambari-funtest/src/test/resources/stacks/OTHER/2.0/repos/repoinfo.xml 
> 9d8a232 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RepositoryResponse.java
>  30bd0db 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java
>  3203539 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  e440460 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryEntity.java
>  8dc2ec9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/RepositoryInfo.java
>  83a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/RepositoryXml.java
>  4a0ae3b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/RepositoryVersionHelper.java
>  d26b834 
>   ambari-server/src/main/resources/stacks/BIGTOP/0.8/repos/repoinfo.xml 
> 6a385c4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6.GlusterFS/repos/repoinfo.xml
>  9b62568 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/repos/repoinfo.xml 
> 9855074 
>   ambari-server/src/main/resources/stacks/HDP/2.0/repos/repoinfo.xml a5d8a51 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/repos/repoinfo.xml 
> cfee6f5 
>   ambari-server/src/main/resources/stacks/HDP/2.1/repos/repoinfo.xml 938f627 
>   ambari-server/src/main/resources/stacks/HDP/2.2/repos/repoinfo.xml dbf8506 
>   ambari-server/src/main/resources/stacks/HDP/2.3.ECS/repos/repoinfo.xml 
> b44cca5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/repos/repoinfo.xml 
> 24b05b0 
>

Re: Review Request 51761: AMBARI-18050 - Upgrade pre-req check code needs to be decoupled from CheckDescription class

2016-09-12 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51761/#review148496
---



Perhaps we should have a check interface instead of requiring extending the 
Abstract class?

- Nate Cole


On Sept. 9, 2016, 12:59 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51761/
> ---
> 
> (Updated Sept. 9, 2016, 12:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jayush Luniya, and 
> Nate Cole.
> 
> 
> Bugs: AMBARI-18050
> https://issues.apache.org/jira/browse/AMBARI-18050
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently each pre-req check must add an entry in the CheckDescription enum. 
> This limits the ability for third party stacks and extensions to provide 
> their own pre-req checks.
> 
> The CheckDescription enum should be rewritten as a class and each pre-req 
> check class should create an instance of it. This will allow stacks and 
> extensions to include their own pre-req checks in separate jar files without 
> requiring changes to ambari-server java code.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  aa8e20c 
> 
> Diff: https://reviews.apache.org/r/51761/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test 
> -Dtest=AbstractCheckDescriptorTest,UpgradeCheckOrderTest,CheckHelperTest
> Ran 267 tests in 8.183s
> Total run:1124
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 51704: Refactor logsearch portal code - part 2


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51704/#review148481
---


Ship it!




Ship It!

- Miklos Gergely


On Sept. 7, 2016, 8:52 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51704/
> ---
> 
> (Updated Sept. 7, 2016, 8:52 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, Sumit Mohanty, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18310
> https://issues.apache.org/jira/browse/AMBARI-18310
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - use fields instead of strings in Criteria objects
> - add rest API integration tests
> - remove xml based configurations (security config & web.xml)
> - fixed the name of external_auth properties
> - removed metrics code (solr)
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-it/pom.xml d918c8a 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/LogSearchApiSteps.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/steps/SolrSteps.java
>  e1e5b99 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/story/LogSearchApiQueryStory.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/java/org/apache/ambari/logsearch/story/LogSearchStory.java
>  f85eb08 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/resources/org/apache/ambari/logsearch/story/log_search_api_query_story.story
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/resources/org/apache/ambari/logsearch/story/logfeeder_parsing_story.story
>  a40d04a 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/resources/test-output/service-log-level-counts-values.json
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/resources/test-output/service-log-schema.json
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  47f139d 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml ebba82c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  27b943a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  edc78d1 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/ApplicationConfig.java
>  72ea942 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
>  3398a83 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SecurityConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  b325171 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java
>  a04dee4 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
>  7affc5a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/PublicManager.java
>  23f62aa 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
>  c4d14a9 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/model/response/AuditLogData.java
>  6df56a1 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/QueryGeneration.java
>  10224bc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/SearchCriteriaConstants.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/converter/AbstractCommonAuditLogRequestConverter.java
>  798bd47 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/converter/AbstractCommonSearchRequestConverter.java
>  3fd07e2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/converter/AbstractCommonServiceLogRequestConverter.java
>  a02d585 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/converter/AbstractConverterAware.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/converter/AnyGraphRequestConverter.java
>  0372168 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/query/converter/AuditBarGraphRequestConverter.java
>

Review Request 51806: ambari-agent check for unset variables (AMBARI-18317)

2016-09-12 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51806/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18360
https://issues.apache.org/jira/browse/AMBARI-18360


Repository: ambari


Description
---

Reported in community:




Diffs
-

  ambari-agent/conf/unix/ambari-agent 978e5c6 
  ambari-server/sbin/ambari-server 8064c45 

Diff: https://reviews.apache.org/r/51806/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 51801: Get LDAP tests to run in concurrent forked JVMs

2016-09-12 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51801/#review148480
---


Ship it!




Ship It!

- Sebastian Toader


On Sept. 12, 2016, 3:01 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51801/
> ---
> 
> (Updated Sept. 12, 2016, 3:01 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Sumit Mohanty, Sebastian Toader, and 
> Yusaku Sako.
> 
> 
> Bugs: AMBARI-18359
> https://issues.apache.org/jira/browse/AMBARI-18359
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
> increasing forkCount to speed up the unit tests.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java
>  1a18ff2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDuplicateUserTest.java
>  c0086d6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java
>  779b0a2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
>  1716c97 
> 
> Diff: https://reviews.apache.org/r/51801/diff/
> 
> 
> Testing
> ---
> 
> * Added forkCount with various values (1, 2, 4, 8, 16) to 
> `maven-surefire-plugin` configuration in `ambari-server/pom.xml`, eg.:
> ```
>   4
>   false
> ```
>  * Ran `mvn -f ambari-server/pom.xml test -DskipPythonTests -Drat.skip 
> -Dtest='*Ldap*Test'`
>  * Verified test failures/errors that occur without the patch are resolved
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 51801: Get LDAP tests to run in concurrent forked JVMs

2016-09-12 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51801/
---

(Updated Sept. 12, 2016, 1:01 p.m.)


Review request for Ambari, Oliver Szabo, Sumit Mohanty, Sebastian Toader, and 
Yusaku Sako.


Bugs: AMBARI-18359
https://issues.apache.org/jira/browse/AMBARI-18359


Repository: ambari


Description
---

Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
increasing forkCount to speed up the unit tests.


Diffs
-

  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java
 1a18ff2 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDuplicateUserTest.java
 c0086d6 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java
 779b0a2 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
 1716c97 

Diff: https://reviews.apache.org/r/51801/diff/


Testing
---

* Added forkCount with various values (1, 2, 4, 8, 16) to 
`maven-surefire-plugin` configuration in `ambari-server/pom.xml`, eg.:
```
  4
  false
```
 * Ran `mvn -f ambari-server/pom.xml test -DskipPythonTests -Drat.skip 
-Dtest='*Ldap*Test'`
 * Verified test failures/errors that occur without the patch are resolved


Thanks,

Attila Doroszlai

Review Request 51801: Get LDAP tests to run in concurrent forked JVMs

2016-09-12 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51801/
---

Review request for Ambari, Sumit Mohanty, Sebastian Toader, and Yusaku Sako.


Bugs: AMBARI-18359
https://issues.apache.org/jira/browse/AMBARI-18359


Repository: ambari


Description
---

Some LDAP tests fail to run with forkCount:>1, reuseForks:false. This blocks 
increasing forkCount to speed up the unit tests.


Diffs
-

  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDNWithSpaceTest.java
 1a18ff2 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderForDuplicateUserTest.java
 c0086d6 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapAuthenticationProviderTest.java
 779b0a2 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariLdapBindAuthenticatorTest.java
 1716c97 

Diff: https://reviews.apache.org/r/51801/diff/


Testing
---

* Added forkCount with various values (1, 2, 4, 8, 16) to 
`maven-surefire-plugin` configuration in `ambari-server/pom.xml`, eg.:
```
  4
  false
```
 * Ran `mvn -f ambari-server/pom.xml test -DskipPythonTests -Drat.skip 
-Dtest='*Ldap*Test'`
 * Verified test failures/errors that occur without the patch are resolved


Thanks,

Attila Doroszlai

Re: Review Request 51426: Journal node restart failing on RU from dergM10 to erie on Wire Encrypted cluster

2016-09-12 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51426/#review148478
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Aug. 25, 2016, 12:53 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51426/
> ---
> 
> (Updated Aug. 25, 2016, 12:53 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18260
> https://issues.apache.org/jira/browse/AMBARI-18260
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Type of upgrade : RU  
> Upgrade from HDP Derg M10 (2.4.2.0) to Erie (on secure, Wire encrypted
> cluster)
> 
> Journal node logs show :
> 
> 
> 
> 
> org.apache.hadoop.hdfs.qjournal.protocol.JournalOutOfSyncException: Can't 
> write, no segment open
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.checkSync(Journal.java:484)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.Journal.journal(Journal.java:353)
>   at 
> org.apache.hadoop.hdfs.qjournal.server.JournalNodeRpcServer.journal(JournalNodeRpcServer.java:152)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocolPB.QJournalProtocolServerSideTranslatorPB.journal(QJournalProtocolServerSideTranslatorPB.java:158)
>   at 
> org.apache.hadoop.hdfs.qjournal.protocol.QJournalProtocolProtos$QJournalProtocolService$2.callBlockingMethod(QJournalProtocolProtos.java:25421)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
>   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
> ç2016-08-25 07:19:47,651 INFO  namenode.FileJournalManager 
> (FileJournalManager.java:finalizeLogSegment(142)) - Finalizing edits file 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_inprogress_0073697
>  -> 
> /grid/0/hadoop/hdfs/journal/nameservice/current/edits_0073697-0073698
> 
> 
> Error at the exact RU task:
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 198, in 
> JournalNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 731, in restart
> self.post_upgrade_restart(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py",
>  line 75, in post_upgrade_restart
> journalnode_upgrade.post_upgrade_check()
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode_upgrade.py",
>  line 64, in post_upgrade_check
> namenode_ha.is_encrypted(), params.security_enabled)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py",
>  line 306, in get_jmx_data
> data = urllib2.urlopen(nn_address).read()
>   File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen
> return _opener.open(url, data, timeout)
>   File "/usr/lib64/python2.6/urllib2.py", line 391, in open
> response = self._open(req, data)
>   File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
> '_open', req)
>   File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
> result = func(*args)
>   File "/usr/lib64/python2.6/urllib2.py", line 1194, in https_open
> return self.do_open(httplib.HTTPSConnection, req)
>   File "/usr/lib64/python2.6/urllib2.py", line 1161, in do_open
> raise URLError(err)
> urllib2.URLError:  routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>
> 
> 
> Live cluster :  
> 
> 
> Artifacts:  dgm10toerienoha-s11/test-logs/ambariru-dgm10toerie-sec-noha/ambaritestartifact
> s/artifacts/screenshots/com.hw.ambari.ui.tests.monitoring.admin_page.TestQuick
> RollingUpgradeApi/test060_StartPerformUpgrade/_24_22_9_0_One_step_of_upgrade_f
>

Re: Review Request 51792: Refactor logsearch portal code - move solr field mappings to UI & remove unused params


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51792/
---

(Updated Sept. 12, 2016, 11:18 a.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
Toader.


Changes
---

removed unused code


Bugs: AMBARI-18310
https://issues.apache.org/jira/browse/AMBARI-18310


Repository: ambari


Description
---

- separate solr Dao into multiple classes based on responsibility
- move field - UI field mapping into the UI side (-> UI changes as well)
- rename property config objects
- remove unused query params from backend/UI
- remove more unused code
- format UI code (not all of it)


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-it/src/test/resources/test-output/service-log-schema.json
 9efa58b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ConfigHelper.java
 a3a71ea 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
 2590dfe 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
 36ecb81 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
 adb0699 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertyMapper.java
 90e2114 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
 46d07bd 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogConfig.java
 ae4dca9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrColumnConfig.java
 a12b2ce 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConfig.java
 cfbe097 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionConfig.java
 b3dceea 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogConfig.java
 e768402 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserConfig.java
 8ae630a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
 4f5e734 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
 e338b7c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrAliasDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 ac7f56f 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
 a9fb8d2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java
 4ca9df6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
 8e14452 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
 bc377e5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
 c64cf71 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogFileManager.java
 405eaef 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/PublicManager.java
 3ac2be6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
 02309fc

Re: Review Request 51796: Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

2016-09-12 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51796/#review148475
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Вер. 12, 2016, 9:42 до полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51796/
> ---
> 
> (Updated Вер. 12, 2016, 9:42 до полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18357
> https://issues.apache.org/jira/browse/AMBARI-18357
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py
>  3a4115c 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 
> ef79f10 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 06b976e 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 
> a0e65cd 
>   ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py 
> f56907c 
> 
> Diff: https://reviews.apache.org/r/51796/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 51792: Refactor logsearch portal code - move solr field mappings to UI & remove unused params


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51792/
---

(Updated Sept. 12, 2016, 9:54 a.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
Toader.


Changes
---

remove getter


Bugs: AMBARI-18310
https://issues.apache.org/jira/browse/AMBARI-18310


Repository: ambari


Description
---

- separate solr Dao into multiple classes based on responsibility
- move field - UI field mapping into the UI side (-> UI changes as well)
- rename property config objects
- remove unused query params from backend/UI
- remove more unused code
- format UI code (not all of it)


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-it/src/test/resources/test-output/service-log-schema.json
 9efa58b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ConfigHelper.java
 a3a71ea 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
 2590dfe 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
 36ecb81 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
 adb0699 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertyMapper.java
 90e2114 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
 46d07bd 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogConfig.java
 ae4dca9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrColumnConfig.java
 a12b2ce 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConfig.java
 cfbe097 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionConfig.java
 b3dceea 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogConfig.java
 e768402 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserConfig.java
 8ae630a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
 4f5e734 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
 e338b7c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrAliasDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 ac7f56f 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
 a9fb8d2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java
 4ca9df6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
 8e14452 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
 bc377e5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
 c64cf71 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogFileManager.java
 405eaef 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/PublicManager.java
 3ac2be6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
 02309fc

Review Request 51796: Ambari fails to start nodemanager due to unexpected return code from sudo su command on the pid file

2016-09-12 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51796/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18357
https://issues.apache.org/jira/browse/AMBARI-18357


Repository: ambari


Description
---


Diffs
-

  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/service.py
 3a4115c 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py ef79f10 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 06b976e 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 
a0e65cd 
  ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py 
f56907c 

Diff: https://reviews.apache.org/r/51796/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 51795: ZEPPELIN user won't get created in ZEPPELIN group

2016-09-12 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51795/#review148474
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Вер. 12, 2016, 9:37 до полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51795/
> ---
> 
> (Updated Вер. 12, 2016, 9:37 до полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18356
> https://issues.apache.org/jira/browse/AMBARI-18356
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Zeppelin user should also be created with group = ZEPPELIN. We should also 
> look at other
> services. Further we should look at how this user to group linkage can be
> config property driven instead of hardcoded logic in the before-ANY hook.
> 
>  service/blob/master/configuration/zeppelin-env.xml#L29-L41>  
>  server/src/main/resources/stacks/HDP/2.0.6/hooks/before-
> ANY/scripts/params.py#L209-L221>
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
>  5544085 
> 
> Diff: https://reviews.apache.org/r/51795/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 51792: Refactor logsearch portal code - move solr field mappings to UI & remove unused params


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51792/
---

(Updated Sept. 12, 2016, 9:39 a.m.)


Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
Toader.


Changes
---

injecting solrcollectionDao into alias solr dao


Bugs: AMBARI-18310
https://issues.apache.org/jira/browse/AMBARI-18310


Repository: ambari


Description
---

- separate solr Dao into multiple classes based on responsibility
- move field - UI field mapping into the UI side (-> UI changes as well)
- rename property config objects
- remove unused query params from backend/UI
- remove more unused code
- format UI code (not all of it)


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-it/src/test/resources/test-output/service-log-schema.json
 9efa58b 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ConfigHelper.java
 a3a71ea 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
 2590dfe 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
 36ecb81 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
 adb0699 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertyMapper.java
 90e2114 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
 46d07bd 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogConfig.java
 ae4dca9 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrColumnConfig.java
 a12b2ce 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConfig.java
 cfbe097 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionConfig.java
 b3dceea 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogConfig.java
 e768402 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserConfig.java
 8ae630a 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserPropsConfig.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
 4f5e734 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
 e338b7c 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrAliasDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 ac7f56f 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
 a9fb8d2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java
 4ca9df6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
 8e14452 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
 bc377e5 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
 c64cf71 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/LogFileManager.java
 405eaef 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/PublicManager.java
 3ac2be6 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
 02309fc

Review Request 51795: ZEPPELIN user won't get created in ZEPPELIN group

2016-09-12 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51795/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18356
https://issues.apache.org/jira/browse/AMBARI-18356


Repository: ambari


Description
---

Zeppelin user should also be created with group = ZEPPELIN. We should also look 
at other
services. Further we should look at how this user to group linkage can be
config property driven instead of hardcoded logic in the before-ANY hook.

  



Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
 5544085 

Diff: https://reviews.apache.org/r/51795/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 51792: Refactor logsearch portal code - move solr field mappings to UI & remove unused params



> On Sept. 12, 2016, 9:11 a.m., Miklos Gergely wrote:
> > Ship It!

Fix it, then ship it.


- Miklos


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51792/#review148470
---


On Sept. 11, 2016, 7:58 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51792/
> ---
> 
> (Updated Sept. 11, 2016, 7:58 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18310
> https://issues.apache.org/jira/browse/AMBARI-18310
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - separate solr Dao into multiple classes based on responsibility
> - move field - UI field mapping into the UI side (-> UI changes as well)
> - rename property config objects
> - remove unused query params from backend/UI
> - remove more unused code
> - format UI code (not all of it)
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/resources/test-output/service-log-schema.json
>  9efa58b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ConfigHelper.java
>  a3a71ea 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  2590dfe 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
>  36ecb81 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
>  adb0699 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertyMapper.java
>  90e2114 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
>  46d07bd 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogConfig.java
>  ae4dca9 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrColumnConfig.java
>  a12b2ce 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConfig.java
>  cfbe097 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionConfig.java
>  b3dceea 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogConfig.java
>  e768402 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserConfig.java
>  8ae630a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
>  4f5e734 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
>  e338b7c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrAliasDao.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  ac7f56f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  a9fb8d2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java
>  4ca9df6 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
>  8e14452 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
>  bc377e5 
>   
>

Re: Review Request 51792: Refactor logsearch portal code - move solr field mappings to UI & remove unused params


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51792/#review148470
---


Ship it!




Ship It!

- Miklos Gergely


On Sept. 11, 2016, 7:58 p.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51792/
> ---
> 
> (Updated Sept. 11, 2016, 7:58 p.m.)
> 
> 
> Review request for Ambari, Miklos Gergely, Robert Nettleton, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18310
> https://issues.apache.org/jira/browse/AMBARI-18310
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - separate solr Dao into multiple classes based on responsibility
> - move field - UI field mapping into the UI side (-> UI changes as well)
> - rename property config objects
> - remove unused query params from backend/UI
> - remove more unused code
> - format UI code (not all of it)
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-it/src/test/resources/test-output/service-log-schema.json
>  9efa58b 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ConfigHelper.java
>  a3a71ea 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/ExternalServerClient.java
>  2590dfe 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/LogSearchConstants.java
>  36ecb81 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertiesHelper.java
>  adb0699 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/common/PropertyMapper.java
>  90e2114 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthConfig.java
>  46d07bd 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/AuthPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogConfig.java
>  ae4dca9 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrAuditLogPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrColumnConfig.java
>  a12b2ce 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConfig.java
>  cfbe097 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionConfig.java
>  b3dceea 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrConnectionPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogConfig.java
>  e768402 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrServiceLogPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserConfig.java
>  8ae630a 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/conf/SolrUserPropsConfig.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
>  4f5e734 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
>  e338b7c 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrAliasDao.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrCollectionDao.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  ac7f56f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrSchemaFieldDao.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  a9fb8d2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserDao.java
>  4ca9df6 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/doc/DocConstants.java
>  8e14452 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/graph/GraphDataGenerator.java
>  bc377e5 
>   
>

Re: Review Request 51792: Refactor logsearch portal code - move solr field mappings to UI & remove unused params