Re: Review Request 48147: Fix metric sink + rename misnamed zk-connect-string variables.

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48147/#review136591
---


Ship it!




AbstractTimelineMetricsSink changes look good with no additional side-effect.

- Sid Wagle


On June 1, 2016, 7 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48147/
> ---
> 
> (Updated June 1, 2016, 7 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, Sumit Mohanty, and 
> Sid Wagle.
> 
> 
> Bugs: AMBARI-16991
> https://issues.apache.org/jira/browse/AMBARI-16991
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> After a recent change the AbstractTimelineMetricsSink class has an abstract 
> function called getZookeeperQuorum(). If the extending class is returning 
> null, then the sink is not working (neither does it if it returns a valid 
> zookeeper connection string). Fixed the first issue, now it is working 
> without a zookeeper connect string.
> 
> Also fixed the name of the misnamed variables called zk_hosts which actually 
> contained a zookeper connect string.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/FetchConfigFromSolr.java
>  1f86dd0 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputSolr.java
>  6fb0b0e 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SolrUtil.java
>  200a603 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/config.json.j2 
> b6301ca 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  076c09c 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/OutputSolrTest.java
>  afbccca 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
>  03ff0ff 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/ServiceLogsSolrDao.java
>  14125bc 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  147e148 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/UserConfigSolrDao.java
>  007c357 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/solr/metrics/SolrMetricsLoader.java
>  21c010f 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties.j2
>  0a94186 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudCLI.java
>  a2da737 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClient.java
>  2805b0b 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientBuilder.java
>  0813221 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/main/java/org/apache/ambari/logsearch/solr/commands/GetSolrHostsCommand.java
>  f814678 
>   
> ambari-logsearch/ambari-logsearch-solr-client/src/test/java/org/apache/ambari/logsearch/solr/AmbariSolrCloudClientTest.java
>  c382c14 
>   
> ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
>  ca7ccea 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logfeeder.properties.j2
>  6a52708 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/logsearch.properties.j2
>  7e649ea 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/output.config.json.j2
>  b31f39b 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json e01bc26 
> 
> Diff: https://reviews.apache.org/r/48147/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> ambari-logsearch-solr-client:
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
> 
> ambari-logsearch-logfeeder:
> Tests run: 35, Failures: 0, Errors: 0, Skipped: 0
> 
> ambari-server:
> Total run:1052
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 47941: [AMBARI-16920] Spark2 thrift server can not started due to miss of spark-thrift-fairscheduler.xml

[Jeff Zhang]

> On June 8, 2016, 1:08 a.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py,
> >  line 63
> > 
> >
> > Why replace_existing_files=True?
> > This will make it slower on Start

You have one comment about "This tarball should be deleted and recreated every 
time the service is restarted.", does this mean to upload new tarball when 
service is restarted ?


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review136568
---


On June 7, 2016, 6:16 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 7, 2016, 6:16 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  3316f78 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  2eae3e7 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 1b33345 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 47941: [AMBARI-16920] Spark2 thrift server can not started due to miss of spark-thrift-fairscheduler.xml

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/#review136568
---




ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 (line 61)


This is calling what's meant to be a private function and getting the 
element at index 1 without checking the length.

Rename the function by removing the leading "_" and check the length before 
accessing [1]



ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 (line 63)


Why replace_existing_files=True?
This will make it slower on Start


- Alejandro Fernandez


On June 7, 2016, 6:16 a.m., Jeff Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47941/
> ---
> 
> (Updated June 7, 2016, 6:16 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16920
> https://issues.apache.org/jira/browse/AMBARI-16920
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This a followup ticket for add spark2 stack definition. There's serveral 
> issues:
> 1.  Spark2 thrift server can not started due to miss of 
> spark-thrift-fairscheduler.xml
> 2.  Miss of add spark2 cache file in copy_barball.py
> 3.  Miss the role_commnad_order of spark2
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
>  286df8d 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  3316f78 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  2eae3e7 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 1b33345 
> 
> Diff: https://reviews.apache.org/r/47941/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>

Re: Review Request 48030: AMBARI-16946 Storm Metrics Sink has high chance to discard some datapoints

[Jungtaek Lim]

> On 6 7, 2016, 6:07 오전, Sid Wagle wrote:
> > Changes look good, only thing to consider is the changes to the metric 
> > name. Cluster Aggregation will not occur at topology level since appId = 
> > topologyName for metrics with the same metric name. Is the metric name to 
> > fine grained? Only thing to consider is whether we need task metrics to be 
> > aggregated across topology? If yes, taskId cannot be part of the metric 
> > name. 
> > 
> > Could you also add Aravindan Vijayan to the reviewers? Thanks.
> 
> Sid Wagle wrote:
> Rephrase: Cluster Aggregation will *now* occur at topology level
> 
> Jungtaek Lim wrote:
> > Only thing to consider is whether we need task metrics to be aggregated 
> across topology? If yes, taskId cannot be part of the metric name. 
> 
> It depends on users, but most cases I don't think user will aggregate 
> metrics across topologies.
> 
> And like what I was saying, technically there're no way to aggregate 
> metrics on sink side since parallelism of sink can be set to more than 1 (I 
> mean, we could have multiple aggregation points which breaks aggregation.)
> So we need to push task level metrics into AMS, and task id should be 
> included as metric name for making it unique.
> 
> Based on that, we need `series aggregation` to aggregate task level 
> metrics by higher level. (I'm trying to address this via AMBARI-17027)
> 
> The only thing which affects aggregation is appId.
> 
> - When we set appId to 'component name' (current), same component (Spout, 
> Bolt) name across topologies can be queried together.
> - When we set appId to 'topology name', same metric name (for Storm's 
> view) from different components in topology can be queried together.
> - When we set appId to 'Storm', all metrics can be queried together. 
> (metric name should also include topology name as well)
> 
> I'm not familiar with structure of metrics so I'm not sure how they 
> affects performance while storing / querying. So I'd like to hear opinions on 
> reviewers.
> 
> For reference, below is how storm-graphite composes prefix of metric 
> name. It uses topology name, component name, worker host, worker port, task 
> id.
> 
> https://github.com/verisign/storm-graphite/blob/master/src/main/java/com/verisign/storm/metrics/GraphiteMetricsConsumer.java#L278
> 
> When querying metrics from Graphite users can query with wildcards & 
> series function to aggregate metrics into one and Grafana can show that. 
> That's what I want to address to AMS.
> 
> Aravindan Vijayan wrote:
> Jungtaek, there is also another level of classification called 
> "instancedId" Every appId can have multiple instanceIds. You should find that 
> in the TimelineMetric object. Perhaps, we can use that here.
> 
> Sriharsha Chintalapani wrote:
> @Jungtaek what Sid was saying is we won't be able to see the aggregated 
> metrics for a single topology if we use taskId. Lets say if I want to see how 
> many tuples are processed in last hour for that topology that aggregation 
> won't be possible using taskId.

@Aravindan
Yeah, actually I was trying to use instanceId for the first time, but 
TimelineMetricCache (TimelineMetricHolder) only checks metric name as unique 
key while putting so that's what I described. Furthermore, I would want to 
aggregate task level metrics into component level metrics but there's no 
wildcard support on instanceId. So there seems to much effort to use instanceId 
for taskId, and having taskId as metric name is easier way to achieve 
aggregation for now.
But I'm open to use instanceId, and support wildcard & aggregation on this. 
Please let me know we would want to use instanceId. @Sid @Sriharsha

@Sriharsha
I guess what Sid was saying is opposite to what you're saying. Let's pretend 
cluster aggregation is occurred.

A. current

appId is component name and metric name is just metric name so there's no 
distinction between topologies. (topology name is not placed anywhere now) So 
cluster aggregation will aggregate metrics across topologies, and we can't 
query last 1 hours of Topology T1, Bolt B1, Metric M1. There're no way to query 
within Topology T1 regardless of aggregation.

B. after patch

appId is topology name and metric name is component name + task id + metric 
name so metrics are classified as topology name. So cluster aggregation will 
aggregate metrics for each topology, and we can't query last 1 hours of all 
topologies, Bolt B1, Metric M1.
There're no way to query across topologies regardless of aggregation.

So if we want to have full flexible queries, topology name also should be 
placed to metric name as same as component name and task id.

@Sid Please correct me if I'm wrong.

Metrics aggregation at query time is not possible for now but will be possible 
with AMBARI-17027.
https://issues.apache.org/jira/browse/AMBARI-17027

>From what I'm understanding, AMS aggregates metrics by 

Re: Review Request 48379: EU - HDP 2.4 to 2.5 fails restarting DRPC server on a kerberized cluster, need to use org.apache.storm.security.auth.KerberosPrincipalToLocal

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48379/
---

(Updated June 7, 2016, 11:58 p.m.)


Review request for Ambari and Sriharsha Chintalapani.


Bugs: AMBARI-17100
https://issues.apache.org/jira/browse/AMBARI-17100


Repository: ambari


Description
---

STR:
* Install Ambari 2.4
* Install HDP 2.4
* Kerberize the cluster
* Perform EU to HDP 2.5

The config packs have


However, storm.yaml still has storm.principal.tolocal : 
'backtype.storm.security.auth.KerberosPrincipalToLocal'

I replaced that property as well and it started working.
Storm 1.0.1 already has its kerberos.json file using "storm.principal.tolocal": 
"org.apache.storm.security.auth.KerberosPrincipalToLocal", and stack HDP 2.5 
uses that version of Storm, so EU/RU should set the property if it exists, 
meaning the cluster is kerberized.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
9df0663 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
f559031 

Diff: https://reviews.apache.org/r/48379/diff/


Testing (updated)
---

Verified during EU from HDP 2.4 to 2.5 on a Kerberized cluster with Storm that 
the property was changed.


Thanks,

Alejandro Fernandez

Re: Review Request 48065: AMBARI-16949 Metrics Collector API shows NPE if we use wildcard (%25 for '%') for metric name

[Jungtaek Lim]

> On 6 7, 2016, 3:07 오후, Aravindan Vijayan wrote:
> > Has this been checked in? If you need help doing that, I can do that for 
> > you.

No it's not checked in yet. Please check this in. Thanks!


- Jungtaek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48065/#review136468
---


On 6 3, 2016, 12:43 오전, Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48065/
> ---
> 
> (Updated 6 3, 2016, 12:43 오전)
> 
> 
> Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.
> 
> 
> Bugs: AMBARI-16949
> https://issues.apache.org/jira/browse/AMBARI-16949
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When we request '/ws/v1/timeline/metrics' with metric name which contains %25 
> (escaped '%'), response for the API is json describing there's NPE.
> And NPE is logged for ambari-metrics-collector log file.
> 
> ```
> 2016-05-30 09:15:05,061 WARN 
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.appendAggregateMetricFromResultSet(PhoenixHBaseAccessor.java:810)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.getAggregateMetricRecords(PhoenixHBaseAccessor.java:772)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.getTimelineMetrics(HBaseTimelineMetricStore.java:178)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TimelineWebServices.getTimelineMetrics(TimelineWebServices.java:372)
>   at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
> ```
> 
> Reason of NPE: 
> 
> Metrics are properly fetched with wildcard. But when applying functions to 
> result set, actual metric name is not exist from map of metric name to list 
> of function.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  47962cb 
> 
> Diff: https://reviews.apache.org/r/48065/diff/
> 
> 
> Testing
> ---
> 
> Test failed from local but it's occurred from another modules, which is not 
> related to current modification.
> 
> ```
> Results :
> 
> Failed tests:
>   PrivilegeEventCreatorTest.putTest:107 expected:<...), Roles(
> Permission[2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...), Roles(
> Permission[1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   RepositoryVersionEventCreatorTest.postTest:70 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   RepositoryVersionEventCreatorTest.putTest:100 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   ViewPrivilegeEventCreatorTest.putTest:85 expected:<...tatus(200 OK), 
> Type([MyView), Version(MyView), Name(MyView), Permissions(
> Permission2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...tatus(200 OK), Type([null), Version(null), 
> Name(null), Permissions(
> Permission1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   
> ComponentResourceProviderTest.testGetResourcesAsAdministrator:190->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsClusterAdministrator:195->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsServiceAdministrator:200->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> AmbariLdapDataPopulatorTest.testSynchronizeExistingLdapGroups_removeDuringIteration:333
>   Expectation failure on verify:
> 

Review Request 48382: AMBARI-17101 Select Stack Page : "Next" button disabled for blank repo fields

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48382/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Richard Zang, 
Srimanth Gunturi, Xi Wang, and Yusaku Sako.


Bugs: AMBARI-17101
https://issues.apache.org/jira/browse/AMBARI-17101


Repository: ambari


Description
---

1. Provide better regex and checking. Make it more clear to handle blanks and 
bad URLs (missing protocol, etc)
2. Treat blank entries as removed, when the user hits Next, as long as they 
complete ONE OS (or choose Satellite).


Diffs
-

  ambari-web/app/controllers/installer.js 251daad 
  ambari-web/app/messages.js 2ea1560 
  ambari-web/app/models/operating_system.js 31f8922 
  ambari-web/app/models/repository.js 6fc5731 
  ambari-web/app/templates/wizard/step1.hbs 4eaef8e 
  ambari-web/app/utils/validator.js d29a6bb 
  ambari-web/app/views/wizard/step1_view.js efdae76 
  ambari-web/test/controllers/installer_test.js e799467 
  ambari-web/test/utils/validator_test.js f70d1ba 
  ambari-web/test/views/wizard/step1_view_test.js 3621aa9 

Diff: https://reviews.apache.org/r/48382/diff/


Testing
---

Modified unit test.
Local ambari-web test passed.
28665 tests complete (24 seconds)
154 tests pending
Manual testing done.


Thanks,

Zhe (Joe) Wang

Re: Review Request 48379: EU - HDP 2.4 to 2.5 fails restarting DRPC server on a kerberized cluster, need to use org.apache.storm.security.auth.KerberosPrincipalToLocal

[Sriharsha Chintalapani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48379/#review136560
---


Ship it!




Ship It!

- Sriharsha Chintalapani


On June 7, 2016, 10:55 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48379/
> ---
> 
> (Updated June 7, 2016, 10:55 p.m.)
> 
> 
> Review request for Ambari and Sriharsha Chintalapani.
> 
> 
> Bugs: AMBARI-17100
> https://issues.apache.org/jira/browse/AMBARI-17100
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> * Install Ambari 2.4
> * Install HDP 2.4
> * Kerberize the cluster
> * Perform EU to HDP 2.5
> 
> The config packs have
>  find="backtype.storm.security.auth.KerberosPrincipalToLocal"
>  
> replace-with="org.apache.storm.security.auth.KerberosPrincipalToLocal" />
> 
> However, storm.yaml still has storm.principal.tolocal : 
> 'backtype.storm.security.auth.KerberosPrincipalToLocal'
> 
> I replaced that property as well and it started working.
> Storm 1.0.1 already has its kerberos.json file using 
> "storm.principal.tolocal": 
> "org.apache.storm.security.auth.KerberosPrincipalToLocal", and stack HDP 2.5 
> uses that version of Storm, so EU/RU should set the property if it exists, 
> meaning the cluster is kerberized.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> 9df0663 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> f559031 
> 
> Diff: https://reviews.apache.org/r/48379/diff/
> 
> 
> Testing
> ---
> 
> Verified during EU from HDP 2.4 to 2.5 on a Kerberized cluster with Storm.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 48379: EU - HDP 2.4 to 2.5 fails restarting DRPC server on a kerberized cluster, need to use org.apache.storm.security.auth.KerberosPrincipalToLocal

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48379/
---

Review request for Ambari and Sriharsha Chintalapani.


Bugs: AMBARI-17100
https://issues.apache.org/jira/browse/AMBARI-17100


Repository: ambari


Description
---

STR:
* Install Ambari 2.4
* Install HDP 2.4
* Kerberize the cluster
* Perform EU to HDP 2.5

The config packs have


However, storm.yaml still has storm.principal.tolocal : 
'backtype.storm.security.auth.KerberosPrincipalToLocal'

I replaced that property as well and it started working.
Storm 1.0.1 already has its kerberos.json file using "storm.principal.tolocal": 
"org.apache.storm.security.auth.KerberosPrincipalToLocal", and stack HDP 2.5 
uses that version of Storm, so EU/RU should set the property if it exists, 
meaning the cluster is kerberized.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
9df0663 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
f559031 

Diff: https://reviews.apache.org/r/48379/diff/


Testing
---

Verified during EU from HDP 2.4 to 2.5 on a Kerberized cluster with Storm.


Thanks,

Alejandro Fernandez

Re: Review Request 48355: AMBARI-17051: Falcon startup properties changes for 2.5

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48355/#review136554
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 7, 2016, 9:43 p.m., Venkat Ranganathan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48355/
> ---
> 
> (Updated June 7, 2016, 9:43 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-17051
> https://issues.apache.org/jira/browse/AMBARI-17051
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added the startup.properties changes required for Lifecycle policies and 
> extension support
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  22fb691 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> c72070b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  0a1bb40 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> a3a3c7d 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> 60cac05 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  0f3bff4 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> cadb3c7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/configuration/falcon-startup.properties.xml
>  0f542cc 
> 
> Diff: https://reviews.apache.org/r/48355/diff/
> 
> 
> Testing
> ---
> 
> Made sure the startup properties changes are valid manually
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>

Re: Review Request 48355: AMBARI-17051: Falcon startup properties changes for 2.5

[Venkat Ranganathan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48355/
---

(Updated June 7, 2016, 2:13 p.m.)


Review request for Ambari and Alejandro Fernandez.


Changes
---

Fixed typo in property name for services config


Bugs: AMBARI-17051
https://issues.apache.org/jira/browse/AMBARI-17051


Repository: ambari


Description
---

Added the startup.properties changes required for Lifecycle policies and 
extension support


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
c72070b 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
 111b432 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
 9365646 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
712241b 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
4187d64 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
60cac05 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/configuration/falcon-startup.properties.xml
 0f542cc 

Diff: https://reviews.apache.org/r/48355/diff/


Testing
---

Made sure the startup properties changes are valid manually


Thanks,

Venkat Ranganathan

Re: Review Request 48361: VDF: install wizard "Select Version" UI issues

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48361/#review136541
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On June 7, 2016, 9:08 p.m., Xi Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48361/
> ---
> 
> (Updated June 7, 2016, 9:08 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17022
> https://issues.apache.org/jira/browse/AMBARI-17022
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fixed several issues in Installer wizard step 1 when selecting version.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/stack.js df217f2 
>   ambari-web/app/styles/application.less 5ee09cc 
>   ambari-web/app/styles/common.less 738edce 
> 
> Diff: https://reviews.apache.org/r/48361/diff/
> 
> 
> Testing
> ---
> 
> 28646 tests complete (27 seconds)
> 154 tests pending
> 
> 
> Thanks,
> 
> Xi Wang
> 
>

Review Request 48361: VDF: install wizard "Select Version" UI issues

[Xi Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48361/
---

Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.


Bugs: AMBARI-17022
https://issues.apache.org/jira/browse/AMBARI-17022


Repository: ambari


Description
---

Fixed several issues in Installer wizard step 1 when selecting version.


Diffs
-

  ambari-web/app/models/stack.js df217f2 
  ambari-web/app/styles/application.less 5ee09cc 
  ambari-web/app/styles/common.less 738edce 

Diff: https://reviews.apache.org/r/48361/diff/


Testing
---

28646 tests complete (27 seconds)
154 tests pending


Thanks,

Xi Wang

Re: Review Request 48355: AMBARI-17051: Falcon startup properties changes for 2.5

[Venkat Ranganathan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48355/
---

(Updated June 7, 2016, 2 p.m.)


Review request for Ambari and Alejandro Fernandez.


Changes
---

Addressed review comments


Bugs: AMBARI-17051
https://issues.apache.org/jira/browse/AMBARI-17051


Repository: ambari


Description
---

Added the startup.properties changes required for Lifecycle policies and 
extension support


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/.config-upgrade.xml.swp
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
c72070b 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
 111b432 
  
ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
 9365646 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml 
712241b 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
4187d64 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
60cac05 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/configuration/falcon-startup.properties.xml
 0f542cc 

Diff: https://reviews.apache.org/r/48355/diff/


Testing
---

Made sure the startup properties changes are valid manually


Thanks,

Venkat Ranganathan

Review Request 48355: AMBARI-17051: Falcon startup properties changes for 2.5

[Venkat Ranganathan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48355/
---

Review request for Ambari and Alejandro Fernandez.


Bugs: AMBARI-17051
https://issues.apache.org/jira/browse/AMBARI-17051


Repository: ambari


Description
---

Added the startup.properties changes required for Lifecycle policies and 
extension support


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/FALCON/configuration/falcon-startup.properties.xml
 0f542cc 

Diff: https://reviews.apache.org/r/48355/diff/


Testing
---


Thanks,

Venkat Ranganathan

Re: Review Request 48350: Atlas Integration : Ambari overwrites users-credentials.properties and policy-store.txt

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48350/#review136519
---


Ship it!




Ship It!

- Robert Levas


On June 7, 2016, 12:33 p.m., Tom Beerbower wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48350/
> ---
> 
> (Updated June 7, 2016, 12:33 p.m.)
> 
> 
> Review request for Ambari, John Speidel and Robert Levas.
> 
> 
> Bugs: AMBARI-17098
> https://issues.apache.org/jira/browse/AMBARI-17098
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The Atlas users-credentials.properties and policy-store.txt are managed by 
> Ambari but can not be changed.  Ambari overwrites any changes when the Atlas 
> server is restarted.
> 
> Make the files un-managed by Ambari so that they can be edited and will pick 
> up the latest changes from the Atlas distro on installation.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/files/policy-store.txt
>  4b3b126 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/files/users-credentials.properties
>  33b326f 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  47cedef 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  1d40ae1 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  98cbc30 
>   ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
> c9a8de0 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py fd75bf7 
> 
> Diff: https://reviews.apache.org/r/48350/diff/
> 
> 
> Testing
> ---
> 
> Updated unit tests.
> 
> manual test Atlas install
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Tom Beerbower
> 
>

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review136517
---




ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 (line 128)


Please consult with Jayush Luniya.
The stack already has a config for the stack root, so hdp_select (or distro 
select as it should be called) shouldn't be hardcoding paths.



ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 (line 129)


Change this to hdp-select status to make it evident what it is doing.


- Alejandro Fernandez


On June 7, 2016, 12:56 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 7, 2016, 12:56 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Daniel Gergely, Oliver Szabo, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installation related information.
> 
> The patch only applies for 2.2-next versions.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 48184: clean up import * for SPARK2 service scripts in common-services

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48184/#review136516
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 7, 2016, 4:09 p.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48184/
> ---
> 
> (Updated June 7, 2016, 4:09 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Matt.
> 
> 
> Bugs: AMBARI-16916
> https://issues.apache.org/jira/browse/AMBARI-16916
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Python code at at common-services level used generic imports form 
> resource_management (from resource_management import *)
> Ideally, for easier code tracking and performance, these import should be 
> more specific, such as: 
> from resource_management.libraries.script.script import Script
> from resource_management.core.resources.system import Directory
> This patch cleans up import * from resource_management for SPARK2 service 
> scripts in common-services
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/job_history_server.py
>  b3720f0 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  3316f78 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py
>  9316ba9 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py
>  434b4b1 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  2eae3e7 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_thrift_server.py
>  be65986 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/status_params.py
>  4196638 
> 
> Diff: https://reviews.apache.org/r/48184/diff/
> 
> 
> Testing
> ---
> 
> added a comment - 7 minutes ago
> -1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12807806/AMBARI-16916.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> -1 tests included. The patch doesn't appear to include any new or modified 
> tests.
> Please justify why no new tests are needed for this patch.
> Also please list what manual steps were performed to verify this patch.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in ambari-server.
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7156//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7156//console
> This message is automatically generated.
> 
> 
> Thanks,
> 
> Juanjo  Marron
> 
>

Re: Review Request 48289: AMBARI-10908 Usability: ability to perform bulk delete host

[Richard Zang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48289/#review136512
---


Ship it!




Ship It!

- Richard Zang


On June 6, 2016, 6:22 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48289/
> ---
> 
> (Updated June 6, 2016, 6:22 p.m.)
> 
> 
> Review request for Ambari, Ajit Kumar, Jaimin Jetly, Richard Zang, Xi Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-10908
> https://issues.apache.org/jira/browse/AMBARI-10908
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On Hosts page, provide ability to delete hosts in bulk.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/host/bulk_operations_controller.js 09d6a52 
>   ambari-web/app/messages.js e8ca9ae 
>   ambari-web/app/styles/application.less 5ee09cc 
>   ambari-web/app/templates/main/host/delete_hosts_dry_run_popup.hbs 
> PRE-CREATION 
>   ambari-web/app/templates/main/host/delete_hosts_popup.hbs PRE-CREATION 
>   ambari-web/app/templates/main/host/delete_hosts_result_popup.hbs 
> PRE-CREATION 
>   ambari-web/app/utils/ajax/ajax.js 195dab4 
>   ambari-web/app/views/main/host/hosts_table_menu_view.js 5e4b0e0 
> 
> Diff: https://reviews.apache.org/r/48289/diff/
> 
> 
> Testing
> ---
> 
> Local ambari-web test passed.
> 28648 tests complete (24 seconds)
> 154 tests pending
> Manual testing done.
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Re: Review Request 47746: Update Moment.js to latest stable version 2.13.0

[Sangeeta Ravindran]

> On June 7, 2016, 5:44 p.m., Di Li wrote:
> > Ship It!

Thank you Di!


- Sangeeta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47746/#review136501
---


On June 7, 2016, 5:23 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47746/
> ---
> 
> (Updated June 7, 2016, 5:23 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jaimin Jetly, and Yusaku Sako.
> 
> 
> Bugs: ambari-16017
> https://issues.apache.org/jira/browse/ambari-16017
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The latest stable version of Moment.js is 2.13.0.
> This task is for updating the version of Moment.js used in
> 
> contrib/views/slider/src/main/resources/ui/vendor/scripts/common/
> ambari-web/vendor/scripts
> 
> 
> Diffs
> -
> 
>   ambari-web/config.coffee 6ed0915 
>   ambari-web/karma.conf.js b11cf76 
>   ambari-web/vendor/scripts/moment.js fe0e19c 
>   ambari-web/vendor/scripts/moment.min.js PRE-CREATION 
>   contrib/views/slider/src/main/resources/ui/config.js 9527393 
>   contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.js 
> b40374b 
>   
> contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.min.js
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47746/diff/
> 
> 
> Testing
> ---
> 
> Manual testing - setting timezone in User Settings.
> Slider view - application start time.
> 
> Tests ran clean for ambari-web 
> 
> 27834 tests complete (42 seconds)
>   154 tests pending
> 
> Tests ran clean for contrib/views/slider
> Took 9506ms to run 345 tests. 345 passed, 0 failed.
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 48273: AMBARI-17054 : Configure Atlas Ranger Plugin

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48273/#review136503
---


Ship it!




Ship It!

- Jonathan Hurley


On June 6, 2016, 11:26 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48273/
> ---
> 
> (Updated June 6, 2016, 11:26 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
> Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17054
> https://issues.apache.org/jira/browse/AMBARI-17054
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains changes to support Ranger Atlas Plugin from Ambari.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  1e9e7a7 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  a79a456 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  4c54214 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  4db7f45 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/themes/theme_version_3.json
>  0f7b0c0 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  98cbc30 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-policymgr-ssl.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-security.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 379c268 
> 
> Diff: https://reviews.apache.org/r/48273/diff/
> 
> 
> Testing
> ---
> 
> Verified : 
> 
> 1) Enable / disable actions for Ranger Atlas Plugin
> 2) Checked Ranger plugin comminication. 
> 3) Checked policy enforecements on Atlas based on Ranger's policies.
> 4) Verified stack advisor recommendations to chnange following properties : 
> 
> atlas.authorizer.impl=org.apache.ranger.authorization.atlas.authorizer.RangerAtlasAuthorizer
> ranger-atlas-plugin-enabled=yes
> 5) Verified Ranger service creation (while enabling Ranger Atlas Plugin)
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 47746: Update Moment.js to latest stable version 2.13.0

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47746/#review136501
---


Ship it!




Ship It!

- Di Li


On June 7, 2016, 5:23 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47746/
> ---
> 
> (Updated June 7, 2016, 5:23 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jaimin Jetly, and Yusaku Sako.
> 
> 
> Bugs: ambari-16017
> https://issues.apache.org/jira/browse/ambari-16017
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The latest stable version of Moment.js is 2.13.0.
> This task is for updating the version of Moment.js used in
> 
> contrib/views/slider/src/main/resources/ui/vendor/scripts/common/
> ambari-web/vendor/scripts
> 
> 
> Diffs
> -
> 
>   ambari-web/config.coffee 6ed0915 
>   ambari-web/karma.conf.js b11cf76 
>   ambari-web/vendor/scripts/moment.js fe0e19c 
>   ambari-web/vendor/scripts/moment.min.js PRE-CREATION 
>   contrib/views/slider/src/main/resources/ui/config.js 9527393 
>   contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.js 
> b40374b 
>   
> contrib/views/slider/src/main/resources/ui/vendor/scripts/common/moment.min.js
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47746/diff/
> 
> 
> Testing
> ---
> 
> Manual testing - setting timezone in User Settings.
> Slider view - application start time.
> 
> Tests ran clean for ambari-web 
> 
> 27834 tests complete (42 seconds)
>   154 tests pending
> 
> Tests ran clean for contrib/views/slider
> Took 9506ms to run 345 tests. 345 passed, 0 failed.
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 48279: Visual explain unit tests

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48279/#review136500
---


Ship it!




Ship It!

- Nitiraj Rathore


On June 6, 2016, 2:24 p.m., Pallav Kulshreshtha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48279/
> ---
> 
> (Updated June 6, 2016, 2:24 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, and Nitiraj 
> Rathore.
> 
> 
> Bugs: AMBARI-17060
> https://issues.apache.org/jira/browse/AMBARI-17060
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - test cases written for visual explain
> 
> 
> Diffs
> -
> 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/app/templates/databases.hbs 
> 290cdac 
>   contrib/views/hive/src/main/resources/ui/hive-web/tests/helpers/api-mock.js 
> ed4822d 
>   contrib/views/hive/src/main/resources/ui/hive-web/tests/helpers/dbclick.js 
> PRE-CREATION 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/database-test.js
>  52cda77 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/query-editor-test.js
>  b409e12 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/saved-queries-test.js
>  c444523 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/tests/integration/udfs-test.js
>  95a0043 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/tests/unit/components/udf-tr-view-test.js
>  PRE-CREATION 
>   
> contrib/views/hive/src/main/resources/ui/hive-web/tests/unit/controllers/udfs-test.js
>  5bd369e 
> 
> Diff: https://reviews.apache.org/r/48279/diff/
> 
> 
> Testing
> ---
> 
> ember test --server
> 
> 
> Thanks,
> 
> Pallav Kulshreshtha
> 
>

Re: Review Request 48332: Log search capability for Nifi

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48332/#review136499
---


Ship it!




Ship It!

- Jayush Luniya


On June 7, 2016, 10:05 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48332/
> ---
> 
> (Updated June 7, 2016, 10:05 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Oliver Szabo, Robert Nettleton, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17086
> https://issues.apache.org/jira/browse/AMBARI-17086
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Giving users the ability to deploy a service to their cluster to collect, 
> index, and explore those log files is needed to improve the ease of 
> administration for the operators.
> Add Log search capability for NiFi
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  34583ba 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-nifi.json.j2
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48332/diff/
> 
> 
> Testing
> ---
> 
> Log Seaerch was installed and logs were loaded fine on local cluster with NiFi
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 48306: AMBARI-17077 - Unable to change user role in list view

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48306/#review136498
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On June 7, 2016, 2:29 a.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48306/
> ---
> 
> (Updated June 7, 2016, 2:29 a.m.)
> 
> 
> Review request for Ambari and Zhe (Joe) Wang.
> 
> 
> Bugs: AMBARI-17077
> https://issues.apache.org/jira/browse/AMBARI-17077
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix role update logic
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/clusters/UserAccessListCtrl.js
>  cd72ca6 
> 
> Diff: https://reviews.apache.org/r/48306/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> All unit tests passed.
> PhantomJS 1.9.7 (Mac OS X): Executed 71 of 71 SUCCESS (0.151 secs / 0.399 
> secs)
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Re: Review Request 48184: clean up import * for SPARK2 service scripts in common-services

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48184/#review136492
---


Ship it!




Ship It!

- Matt


On June 7, 2016, 9:09 a.m., Juanjo  Marron wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48184/
> ---
> 
> (Updated June 7, 2016, 9:09 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Matt.
> 
> 
> Bugs: AMBARI-16916
> https://issues.apache.org/jira/browse/AMBARI-16916
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Python code at at common-services level used generic imports form 
> resource_management (from resource_management import *)
> Ideally, for easier code tracking and performance, these import should be 
> more specific, such as: 
> from resource_management.libraries.script.script import Script
> from resource_management.core.resources.system import Directory
> This patch cleans up import * from resource_management for SPARK2 service 
> scripts in common-services
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/job_history_server.py
>  b3720f0 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  3316f78 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/service_check.py
>  694f046 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py
>  9316ba9 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py
>  434b4b1 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  2eae3e7 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_thrift_server.py
>  be65986 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/status_params.py
>  4196638 
> 
> Diff: https://reviews.apache.org/r/48184/diff/
> 
> 
> Testing
> ---
> 
> added a comment - 7 minutes ago
> -1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12807806/AMBARI-16916.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> -1 tests included. The patch doesn't appear to include any new or modified 
> tests.
> Please justify why no new tests are needed for this patch.
> Also please list what manual steps were performed to verify this patch.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in ambari-server.
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7156//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7156//console
> This message is automatically generated.
> 
> 
> Thanks,
> 
> Juanjo  Marron
> 
>

Re: Review Request 48030: AMBARI-16946 Storm Metrics Sink has high chance to discard some datapoints

[Sriharsha Chintalapani]

> On June 7, 2016, 6:07 a.m., Sid Wagle wrote:
> > Changes look good, only thing to consider is the changes to the metric 
> > name. Cluster Aggregation will not occur at topology level since appId = 
> > topologyName for metrics with the same metric name. Is the metric name to 
> > fine grained? Only thing to consider is whether we need task metrics to be 
> > aggregated across topology? If yes, taskId cannot be part of the metric 
> > name. 
> > 
> > Could you also add Aravindan Vijayan to the reviewers? Thanks.
> 
> Sid Wagle wrote:
> Rephrase: Cluster Aggregation will *now* occur at topology level
> 
> Jungtaek Lim wrote:
> > Only thing to consider is whether we need task metrics to be aggregated 
> across topology? If yes, taskId cannot be part of the metric name. 
> 
> It depends on users, but most cases I don't think user will aggregate 
> metrics across topologies.
> 
> And like what I was saying, technically there're no way to aggregate 
> metrics on sink side since parallelism of sink can be set to more than 1 (I 
> mean, we could have multiple aggregation points which breaks aggregation.)
> So we need to push task level metrics into AMS, and task id should be 
> included as metric name for making it unique.
> 
> Based on that, we need `series aggregation` to aggregate task level 
> metrics by higher level. (I'm trying to address this via AMBARI-17027)
> 
> The only thing which affects aggregation is appId.
> 
> - When we set appId to 'component name' (current), same component (Spout, 
> Bolt) name across topologies can be queried together.
> - When we set appId to 'topology name', same metric name (for Storm's 
> view) from different components in topology can be queried together.
> - When we set appId to 'Storm', all metrics can be queried together. 
> (metric name should also include topology name as well)
> 
> I'm not familiar with structure of metrics so I'm not sure how they 
> affects performance while storing / querying. So I'd like to hear opinions on 
> reviewers.
> 
> For reference, below is how storm-graphite composes prefix of metric 
> name. It uses topology name, component name, worker host, worker port, task 
> id.
> 
> https://github.com/verisign/storm-graphite/blob/master/src/main/java/com/verisign/storm/metrics/GraphiteMetricsConsumer.java#L278
> 
> When querying metrics from Graphite users can query with wildcards & 
> series function to aggregate metrics into one and Grafana can show that. 
> That's what I want to address to AMS.
> 
> Aravindan Vijayan wrote:
> Jungtaek, there is also another level of classification called 
> "instancedId" Every appId can have multiple instanceIds. You should find that 
> in the TimelineMetric object. Perhaps, we can use that here.

@Jungtaek what Sid was saying is we won't be able to see the aggregated metrics 
for a single topology if we use taskId. Lets say if I want to see how many 
tuples are processed in last hour for that topology that aggregation won't be 
possible using taskId.


- Sriharsha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48030/#review136411
---


On June 7, 2016, 6:14 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48030/
> ---
> 
> (Updated June 7, 2016, 6:14 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sriharsha Chintalapani, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-16946
> https://issues.apache.org/jira/browse/AMBARI-16946
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There's a mismatch between TimelineMetricsCache and Storm metrics unit, while 
> TimelineMetricsCache considers "metric name + timestamp" to be unique but 
> Storm is not.
> 
> For example, assume that bolt B has task T1, T2 and B has registered metrics 
> M1. It's possible for metrics sink to receive (T1, M1) and (T2, M1) with same 
> timestamp TS1 (in TaskInfo, not current time), and received later will be 
> discarded from TimelineMetricsCache.
> 
> If we want to have unique metric point of Storm, we should use "topology name 
> + component name + task id + metric name" to metric name so that "metric name 
> + timestamp" will be unique.
> 
> There're other issues I would like to address, too.
> 
> - Currently, hostname is written to hostname of the machine which runs 
> metrics sink. Since TaskInfo has hostname of the machine which runs task, 
> we're better to use this.
> - Unit of timestamp of TaskInfo is second, while Storm Metrics Sink uses this 
> as millisecond, resulting in timestamp flaw, and malfunction of cache 
> eviction. It should be multiplied by 1000.
> - 

Review Request 48348: AMBARI-17089: HDFS logs not picked by log feeder on a newly installed cluster with log search

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48348/
---

Review request for Ambari, Don Bosco Durai, Miklos Gergely, Robert Nettleton, 
Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-17089
https://issues.apache.org/jira/browse/AMBARI-17089


Repository: ambari


Description
---

- Change logfeeder process/files to use sudo user instead of 
logfeeder/logfeeder user/group (to make sure logfeeder can read any kind of the 
logs).
- solr and logsearch user both moved to hadoop group


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/solr_cloud_util.py
 b099a1e 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 47cedef 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 1d40ae1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 46ac4c2 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 7943cd0 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-properties.xml
 65dc378 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 73fecb6 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
 c0689f3 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch.py
 2b5fdf7 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_common.py
 d0ac389 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logsearch_solr.py
 b55f3d6 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 34583ba 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logfeeder.py
 5ca2bd5 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
 58239c7 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch_solr.py
 6e71334 
  ambari-server/src/test/python/stacks/2.3/ATLAS/test_metadata_server.py 
c9a8de0 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py 54e08e4 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py bfe6921 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_solr.py 0590dca 
  ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py fd75bf7 
  ambari-server/src/test/python/stacks/2.5/configs/default.json 1015593 
  ambari-web/app/data/HDP2/site_properties.js 794da25 

Diff: https://reviews.apache.org/r/48348/diff/


Testing
---

lot of ambari-server python errors with global name 'is_empty' is nod defined 
error, but it is not related with my change. logsearch related tests passed.

FT: tested locally with 4 node cluster.


Thanks,

Oliver Szabo

Re: Review Request 48096: AMBARI-16935: Retry and recover from component install failures

[Nahappan Somasundaram]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48096/
---

(Updated June 7, 2016, 8:51 a.m.)


Review request for Ambari, Ajit Kumar, Jonathan Hurley, Sumit Mohanty, and Sid 
Wagle.


Changes
---

Fixed code review comments.


Bugs: AMBARI-16935
https://issues.apache.org/jira/browse/AMBARI-16935


Repository: ambari


Description
---

AMBARI-16935: Retry and recover from component install failures

** Issue **

There are multiple instances where components end up in INSTALL_FAILED state 
during cluster setup. Ambari does not retry or recover from INSTALL_FAILED 
state. 

Ambari should retry and recover from installation failures.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py 
fdda63251b3ddf96cec794037029e786a9c0bc91 
  ambari-agent/src/main/python/ambari_agent/RecoveryManager.py 
87d9483c634026897629396bb48ec0cbabfcfae6 
  ambari-agent/src/test/python/ambari_agent/TestRecoveryManager.py 
ed0fd2fd3cfd37f535fa14f52835ddefd376038b 

Diff: https://reviews.apache.org/r/48096/diff/


Testing
---

** 1. mvn clean install **

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [8.346s]
[INFO] Apache Ambari Project POM . SUCCESS [0.036s]
[INFO] Ambari Web  SUCCESS [24.196s]
[INFO] Ambari Views .. SUCCESS [1.370s]
[INFO] Ambari Admin View . SUCCESS [7.555s]
[INFO] ambari-metrics  SUCCESS [0.388s]
[INFO] Ambari Metrics Common . SUCCESS [14.289s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.879s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [0.951s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [1.085s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [2.354s]
[INFO] Ambari Metrics Collector .. SUCCESS [6.883s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.126s]
[INFO] Ambari Metrics Grafana  SUCCESS [0.886s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:15.977s]
[INFO] Ambari Server . SUCCESS [3:06.681s]
[INFO] Ambari Functional Tests ... SUCCESS [1.430s]
[INFO] Ambari Agent .. SUCCESS [30.176s]
[INFO] Ambari Client . SUCCESS [0.052s]
[INFO] Ambari Python Client .. SUCCESS [1.129s]
[INFO] Ambari Groovy Client .. SUCCESS [2.394s]
[INFO] Ambari Shell .. SUCCESS [0.078s]
[INFO] Ambari Python Shell ... SUCCESS [0.858s]
[INFO] Ambari Groovy Shell ... SUCCESS [4.609s]
[INFO] ambari-logsearch .. SUCCESS [0.264s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.231s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [4.324s]
[INFO] Ambari Logsearch Portal ... SUCCESS [6.150s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [2.309s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.101s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 6:29.831s
[INFO] Finished at: Tue May 31 15:21:18 PDT 2016
[INFO] Final Memory: 294M/1039M
[INFO] 

** 2. mvn test -DskipSurefireTests **

--
Ran 261 tests in 6.695s

OK
--
Total run:1052
Total errors:0
Total failures:0
OK
INFO: AMBARI_SERVER_LIB is not set, using default /usr/lib/ambari-server
INFO: Return code from stack upgrade command, retcode = 0
StackAdvisor implementation for stack HDP1, version 2.0.6 was not found
Returning DefaultStackAdvisor implementation
StackAdvisor implementation for stack XYZ, version 1.0.0 was loaded
StackAdvisor implementation for stack XYZ, version 1.0.1 was loaded
Returning XYZ101StackAdvisor implementation
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 55.370s
[INFO] Finished at: Tue May 31 15:09:42 PDT 2016
[INFO] Final Memory: 57M/1010M

Re: Review Request 48321: Ambari server failed to start METRICS_COLLECTOR via BP

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48321/#review136481
---


Ship it!




LGTM

- Aravindan Vijayan


On June 7, 2016, 2:25 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48321/
> ---
> 
> (Updated June 7, 2016, 2:25 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, Robert 
> Nettleton, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17082
> https://issues.apache.org/jira/browse/AMBARI-17082
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If blueprint contains 2 AMS collectors, Ambari server failed to start 
> METRICS_COLLECTOR with
> 
> ERROR [pool-16-thread-1] TopologyManager:782 - 
> TopologyManager.ConfigureClusterTask: An exception occurred while attempting 
> to process cluster configs and set on cluster:
> java.lang.IllegalArgumentException: Unable to update configuration property 
> 'timeline.metrics.service.webapp.address' with topology information. 
> Component 'METRICS_COLLECTOR' is mapped to an invalid number of hosts '2'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  a0af813 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  8b1a9a6 
> 
> Diff: https://reviews.apache.org/r/48321/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 48065: AMBARI-16949 Metrics Collector API shows NPE if we use wildcard (%25 for '%') for metric name

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48065/#review136468
---


Ship it!




Has this been checked in? If you need help doing that, I can do that for you.

- Aravindan Vijayan


On June 3, 2016, 12:43 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48065/
> ---
> 
> (Updated June 3, 2016, 12:43 a.m.)
> 
> 
> Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.
> 
> 
> Bugs: AMBARI-16949
> https://issues.apache.org/jira/browse/AMBARI-16949
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When we request '/ws/v1/timeline/metrics' with metric name which contains %25 
> (escaped '%'), response for the API is json describing there's NPE.
> And NPE is logged for ambari-metrics-collector log file.
> 
> ```
> 2016-05-30 09:15:05,061 WARN 
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.appendAggregateMetricFromResultSet(PhoenixHBaseAccessor.java:810)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.getAggregateMetricRecords(PhoenixHBaseAccessor.java:772)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.getTimelineMetrics(HBaseTimelineMetricStore.java:178)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TimelineWebServices.getTimelineMetrics(TimelineWebServices.java:372)
>   at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
> ```
> 
> Reason of NPE: 
> 
> Metrics are properly fetched with wildcard. But when applying functions to 
> result set, actual metric name is not exist from map of metric name to list 
> of function.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  47962cb 
> 
> Diff: https://reviews.apache.org/r/48065/diff/
> 
> 
> Testing
> ---
> 
> Test failed from local but it's occurred from another modules, which is not 
> related to current modification.
> 
> ```
> Results :
> 
> Failed tests:
>   PrivilegeEventCreatorTest.putTest:107 expected:<...), Roles(
> Permission[2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...), Roles(
> Permission[1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   RepositoryVersionEventCreatorTest.postTest:70 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   RepositoryVersionEventCreatorTest.putTest:100 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   ViewPrivilegeEventCreatorTest.putTest:85 expected:<...tatus(200 OK), 
> Type([MyView), Version(MyView), Name(MyView), Permissions(
> Permission2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...tatus(200 OK), Type([null), Version(null), 
> Name(null), Permissions(
> Permission1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   
> ComponentResourceProviderTest.testGetResourcesAsAdministrator:190->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsClusterAdministrator:195->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsServiceAdministrator:200->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> AmbariLdapDataPopulatorTest.testSynchronizeExistingLdapGroups_removeDuringIteration:333
>   Expectation failure on verify:
> AmbariLdapDataPopulatorTestInstance.getLdapGroupByMemberAttr("group2"): 
> expected: 1, actual: 0
>   

Re: Review Request 48284: Retrieve specific metrics when Ambari queries NameNode HA states

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48284/#review136466
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
 (line 88)


Let's put some JavaDoc on this so people know what it's used for.


- Jonathan Hurley


On June 7, 2016, 7:47 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48284/
> ---
> 
> (Updated June 7, 2016, 7:47 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17063
> https://issues.apache.org/jira/browse/AMBARI-17063
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This is a follow-on jira of BUG-53459: when Ambari queries NameNode's HA
> states, currently it retrieves a group of metrics including some unnecessary
> ones that may compete for NameNode lock. It would be better for Ambari to only
> retrieve HA state metrics from NameNode.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
>  a315e5c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/jmx/TestStreamProvider.java
>  8819991 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
>  4adea20 
>   ambari-server/src/test/resources/hdfs_namenode_jmx_ha_only.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48284/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48292: VDF: exception when trying to register -> add versions

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48292/#review136465
---


Ship it!




Ship It!

- Jonathan Hurley


On June 6, 2016, 5:34 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48292/
> ---
> 
> (Updated June 6, 2016, 5:34 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17071
> https://issues.apache.org/jira/browse/AMBARI-17071
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Only verify duplicate repo URLs if Ambari is managing the repos for the 
> desired repo_version.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProvider.java
>  57fb115 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryVersionResourceProviderTest.java
>  a61af95 
> 
> Diff: https://reviews.apache.org/r/48292/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> 
> Tests run: 4454, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 36:27.253s
> [INFO] Finished at: Mon Jun 06 17:26:34 EDT 2016
> [INFO] Final Memory: 35M/569M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 48338: LogSearch https support.

[Robert Nettleton]

> On June 7, 2016, 1:15 p.m., Robert Nettleton wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java,
> >  line 38
> > 
> >
> > It looks like all these properties should be defined in the "LOGSEARCH" 
> > stack definitions in Ambari, so I'd recommend adding them in this patch.
> 
> Oliver Szabo wrote:
> I can add that part after the patch is merged. (in case if everything 
> works fine if these properties are not used)
> 
> Dharmesh Makwana wrote:
> Oliver, logsearch will work fine in http protocol without these 
> properties. You can go ahead and merge it.

Ok, thanks for the clarification, I'll drop this issue, since it looks like the 
Ambari side of things will be handled in a separate patch.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48338/#review136454
---


On June 7, 2016, 1:25 p.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48338/
> ---
> 
> (Updated June 7, 2016, 1:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
> Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
> Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17092
> https://issues.apache.org/jira/browse/AMBARI-17092
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> LogSearch portal should support HTTPS to ensure secure communication
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  2ebf981 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/PropertiesUtil.java
>  f31e8f8 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  f477c5a 
> 
> Diff: https://reviews.apache.org/r/48338/diff/
> 
> 
> Testing
> ---
> 
> Setup Logsearch on 3 node cluster and tested the above feature.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>

Re: Review Request 48338: LogSearch https support.

[Dharmesh Makwana]

> On June 7, 2016, 1:15 p.m., Robert Nettleton wrote:
> > ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java,
> >  line 38
> > 
> >
> > It looks like all these properties should be defined in the "LOGSEARCH" 
> > stack definitions in Ambari, so I'd recommend adding them in this patch.
> 
> Oliver Szabo wrote:
> I can add that part after the patch is merged. (in case if everything 
> works fine if these properties are not used)

Oliver, logsearch will work fine in http protocol without these properties. You 
can go ahead and merge it.


- Dharmesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48338/#review136454
---


On June 7, 2016, 1:25 p.m., Dharmesh Makwana wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48338/
> ---
> 
> (Updated June 7, 2016, 1:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
> Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
> Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17092
> https://issues.apache.org/jira/browse/AMBARI-17092
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> LogSearch portal should support HTTPS to ensure secure communication
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
>  2ebf981 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/PropertiesUtil.java
>  f31e8f8 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
>  f477c5a 
> 
> Diff: https://reviews.apache.org/r/48338/diff/
> 
> 
> Testing
> ---
> 
> Setup Logsearch on 3 node cluster and tested the above feature.
> 
> 
> Thanks,
> 
> Dharmesh Makwana
> 
>

Re: Review Request 48321: Ambari server failed to start METRICS_COLLECTOR via BP

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48321/#review136457
---


Fix it, then Ship it!




Thanks for providing this patch.

I do have one question listed as an issue below that I think should be resolved 
prior to merging this.

I'd recommend asking Aravindan to review the code and my question, just to 
clarify what this property's value can be in a multi-collector environment. 

Thanks again!


ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 (line 2623)


I'm a little unsure about this change.

On the surface, this appears to be correct in terms of resolving the 
Exception, but my concern here is that we're not exactly sure what the value of 
this property should be when more than 1 Metrics Collector is deployed.

I would recommend asking a Metrics expert about this. 

Basically, can the

"timeline.metrics.service.webapp.address" property be used in a 
multi-collector environment?  If so, is the value expected to be a 
comma-separated list of collectors?


- Robert Nettleton


On June 7, 2016, 8:12 a.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48321/
> ---
> 
> (Updated June 7, 2016, 8:12 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, Robert 
> Nettleton, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17082
> https://issues.apache.org/jira/browse/AMBARI-17082
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If blueprint contains 2 AMS collectors, Ambari server failed to start 
> METRICS_COLLECTOR with
> 
> ERROR [pool-16-thread-1] TopologyManager:782 - 
> TopologyManager.ConfigureClusterTask: An exception occurred while attempting 
> to process cluster configs and set on cluster:
> java.lang.IllegalArgumentException: Unable to update configuration property 
> 'timeline.metrics.service.webapp.address' with topology information. 
> Component 'METRICS_COLLECTOR' is mapped to an invalid number of hosts '2'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  a0af813 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  8b1a9a6 
> 
> Diff: https://reviews.apache.org/r/48321/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 48338: LogSearch https support.

[Dharmesh Makwana]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48338/
---

(Updated June 7, 2016, 1:25 p.m.)


Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Don Bosco 
Durai, Jaimin Jetly, Oliver Szabo, Robert Nettleton, Sandor Magyari, Sumit 
Mohanty, and Sebastian Toader.


Changes
---

Fixed typo.


Bugs: AMBARI-17092
https://issues.apache.org/jira/browse/AMBARI-17092


Repository: ambari


Description
---

LogSearch portal should support HTTPS to ensure secure communication


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/LogSearch.java
 2ebf981 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/util/PropertiesUtil.java
 f31e8f8 
  
ambari-logsearch/ambari-logsearch-portal/src/main/resources/logsearch.properties
 f477c5a 

Diff: https://reviews.apache.org/r/48338/diff/


Testing
---

Setup Logsearch on 3 node cluster and tested the above feature.


Thanks,

Dharmesh Makwana

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 7, 2016, 12:15 p.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.

The patch only applies for 2.2-next versions.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing (updated)
---

Unit tests passed.


Thanks,

Laszlo Puskas

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 7, 2016, 12:14 p.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Removed sudo from ls command executions.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.

The patch only applies for 2.2-next versions.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing
---

Unit tests passed.
Manual testing underway.
// need to verify if ls is configured with sudo on agents.


Thanks,

Laszlo Puskas

Re: Review Request 48284: Retrieve specific metrics when Ambari queries NameNode HA states

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48284/#review136452
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Червень 7, 2016, 11:47 до полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48284/
> ---
> 
> (Updated Червень 7, 2016, 11:47 до полудня)
> 
> 
> Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17063
> https://issues.apache.org/jira/browse/AMBARI-17063
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This is a follow-on jira of BUG-53459: when Ambari queries NameNode's HA
> states, currently it retrieves a group of metrics including some unnecessary
> ones that may compete for NameNode lock. It would be better for Ambari to only
> retrieve HA state metrics from NameNode.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
>  a315e5c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/jmx/TestStreamProvider.java
>  8819991 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
>  4adea20 
>   ambari-server/src/test/resources/hdfs_namenode_jmx_ha_only.json 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48284/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48284: Retrieve specific metrics when Ambari queries NameNode HA states

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48284/
---

(Updated June 7, 2016, 11:47 a.m.)


Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.


Bugs: AMBARI-17063
https://issues.apache.org/jira/browse/AMBARI-17063


Repository: ambari


Description
---

This is a follow-on jira of BUG-53459: when Ambari queries NameNode's HA
states, currently it retrieves a group of metrics including some unnecessary
ones that may compete for NameNode lock. It would be better for Ambari to only
retrieve HA state metrics from NameNode.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java
 a315e5c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/jmx/TestStreamProvider.java
 8819991 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java
 4adea20 
  ambari-server/src/test/resources/hdfs_namenode_jmx_ha_only.json PRE-CREATION 

Diff: https://reviews.apache.org/r/48284/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 48335: Zeppelin service: Update default zeppelin_pid_dir to /var/run/zeppelin

[Renjith Kamath]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48335/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Jayush 
Luniya, Pallav Kulshreshtha, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17088
https://issues.apache.org/jira/browse/AMBARI-17088


Repository: ambari


Description
---

update dir name from zeppelin-notebook to zeppelin


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
 efd31f1 

Diff: https://reviews.apache.org/r/48335/diff/


Testing
---

manually tested on CentOS


Thanks,

Renjith Kamath

Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/#review136446
---



Wouldn't better to fire these events only on component state changes instead of 
each hearbeat. HertbeatProcessor is fireing events like 
ServiceComponentHostEventType.HOST_SVCCOMP_STARTED, HOST_SVCCOMP_STOPPED, 
HOST_SVCCOMP_OP_FAILED.

- Sandor Magyari


On June 7, 2016, 7:47 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48266/
> ---
> 
> (Updated June 7, 2016, 7:47 a.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
> Sandor Magyari, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17053
> https://issues.apache.org/jira/browse/AMBARI-17053
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add a log message to see when the cluster is ready to use (cluster creation 
> finishes).
> 
> An event after each heartbeat. At that time cluster provision request is 
> checked if it is finished or not. In case of an ambari server restart, the 
> replayed requests are used to determine which one was the provision request. 
> I could not find a nice way to do it, but I could make a workaround.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java
>  3feac55 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
> 1079806 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  e3f5b49 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
>  fd8653c 
> 
> Diff: https://reviews.apache.org/r/48266/diff/
> 
> 
> Testing
> ---
> 
> Still running locally...
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 48325: Unit tests failing because of the order of hash sets/maps

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48325/#review136445
---




ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RequestAuditEventCreatorHelper.java
 (lines 88 - 89)


Is it a valid scenario that ```getPropertyList``` is invoked with an 
invalid ```propertyName```? If not than that's an error case an we should throw 
an exception.



ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/PrivilegeEventCreatorTest.java
 (lines 99 - 110)


You could extract the common part into a separate string such as to reduce 
the copy-pasting.



ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/RepositoryVersionEventCreatorTest.java
 (lines 62 - 75)


You could extract the common part into a separate string such as to reduce 
the copy-pasting.



ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/RepositoryVersionEventCreatorTest.java
 (lines 100 - 113)


You could extract the common part into a separate string such as to reduce 
the copy-pasting.



ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/ViewPrivilegeEventCreatorTest.java
 (lines 77 - 90)


You could extract the common part into a separate string such as to reduce 
the copy-pasting.



ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ComponentResourceProviderTest.java
 (lines 229 - 231)


Can you comment how this change relates to the hash/sets ordering?


- Sebastian Toader


On June 7, 2016, 11:53 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48325/
> ---
> 
> (Updated June 7, 2016, 11:53 a.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Oliver Szabo, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17004
> https://issues.apache.org/jira/browse/AMBARI-17004
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Some unit tests are failing because they depend on the order elements are 
> returned by the iterator of a HashSet/HashMap. Usually they are small (2-3 
> entries), so they may easily produce good results. Still in other 
> environments they fail as the order the elements are returned from a 
> HashSet/HashMap may depend on the JVM used.
> 
> The issues are elaborated in the comments of the bug.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/RequestAuditEventCreatorHelper.java
>  f7a6e36 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/PrivilegeEventCreatorTest.java
>  ff4abd6 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/RepositoryVersionEventCreatorTest.java
>  18e2d3f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/creator/ViewPrivilegeEventCreatorTest.java
>  ab2b068 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ComponentResourceProviderTest.java
>  d299fad 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/ldap/AmbariLdapDataPopulatorTest.java
>  fbe233d 
> 
> Diff: https://reviews.apache.org/r/48325/diff/
> 
> 
> Testing
> ---
> 
> Tests are running fine now even on my computer which wasn't true before. They 
> failed some times on Jenkins too.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Review Request 48334: takeover_config_merge.py should provide XML, yaml, properties-diff capability

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48334/
---

Review request for Ambari and Sid Wagle.


Bugs: AMBARI-17087
https://issues.apache.org/jira/browse/AMBARI-17087


Repository: ambari


Description
---

`takeover_config_merge.py` should provide XML-diff capability, where the
script is provided 2 folders containing Hadoop configs, and the utility prints
the keys which are different in the left and right folders.

This will be very useful in comparing the pre-takeover config folders with the
post-takeover config folders, to verify that Ambari did not make any unwanted
changes.


Diffs
-

  ambari-server/src/main/resources/scripts/takeover_config_merge.py 075f99f 

Diff: https://reviews.apache.org/r/48334/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 48332: Log search capability for Nifi

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48332/
---

Review request for Ambari, Jayush Luniya, Oliver Szabo, Robert Nettleton, and 
Sumit Mohanty.


Bugs: AMBARI-17086
https://issues.apache.org/jira/browse/AMBARI-17086


Repository: ambari


Description
---

Giving users the ability to deploy a service to their cluster to collect, 
index, and explore those log files is needed to improve the ease of 
administration for the operators.
Add Log search capability for NiFi


Diffs
-

  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
 34583ba 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-nifi.json.j2
 PRE-CREATION 

Diff: https://reviews.apache.org/r/48332/diff/


Testing
---

Log Seaerch was installed and logs were loaded fine on local cluster with NiFi


Thanks,

Miklos Gergely

Re: Review Request 48321: Ambari server failed to start METRICS_COLLECTOR via BP

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48321/#review136443
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Червень 7, 2016, 8:12 до полудня, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48321/
> ---
> 
> (Updated Червень 7, 2016, 8:12 до полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, Robert 
> Nettleton, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17082
> https://issues.apache.org/jira/browse/AMBARI-17082
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If blueprint contains 2 AMS collectors, Ambari server failed to start 
> METRICS_COLLECTOR with
> 
> ERROR [pool-16-thread-1] TopologyManager:782 - 
> TopologyManager.ConfigureClusterTask: An exception occurred while attempting 
> to process cluster configs and set on cluster:
> java.lang.IllegalArgumentException: Unable to update configuration property 
> 'timeline.metrics.service.webapp.address' with topology information. 
> Component 'METRICS_COLLECTOR' is mapped to an invalid number of hosts '2'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  a0af813 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  8b1a9a6 
> 
> Diff: https://reviews.apache.org/r/48321/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 48330: Hive View : Error in persisting JobImpl while submitting Hive Job

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48330/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, 
Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-17085
https://issues.apache.org/jira/browse/AMBARI-17085


Repository: ambari


Description
---

Earlier : The error occurs because of incorrect critical section implementation 
in DataStoreImpl.java. initialized= true was set even before actual 
initialization.

In this patch : 
The location of initialized variable is set after actual initialization.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java
 039fd6f 

Diff: https://reviews.apache.org/r/48330/diff/


Testing
---

Manual testing done.


Thanks,

Nitiraj Rathore

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 7, 2016, 9:18 a.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Applied review notes.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.

The patch only applies for 2.2-next versions.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing (updated)
---

Unit tests passed.
Manual testing underway.
// need to verify if ls is configured with sudo on agents.


Thanks,

Laszlo Puskas

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review136439
---




ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 (line 128)


Instead of using {sudo} param. Can we use something like this:

Execute(('ls','-la','/usr/hdp', sudo=True)

This will ensure that we add specific flags for older sudo packages.


- Andrew Onischuk


On June 7, 2016, 8:17 a.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 7, 2016, 8:17 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installationrelated information.
> 
> The patch only applies for 2.2-next versions.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> Manual testing underway.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Andrew Onischuk]

> On June 7, 2016, 8:53 a.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py,
> >  line 130
> > 
> >
> > I don't think we have ts in sudo permissions. Can we use test here?
> 
> Andrew Onischuk wrote:
> ls not ts.

For the reference 
https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_Ambari_Security_Guide/content/_commands.html


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review136436
---


On June 7, 2016, 8:17 a.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 7, 2016, 8:17 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installationrelated information.
> 
> The patch only applies for 2.2-next versions.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> Manual testing underway.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Andrew Onischuk]

> On June 7, 2016, 8:53 a.m., Andrew Onischuk wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py,
> >  line 130
> > 
> >
> > I don't think we have ts in sudo permissions. Can we use test here?

ls not ts.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/#review136436
---


On June 7, 2016, 8:17 a.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48044/
> ---
> 
> (Updated June 7, 2016, 8:17 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-16952
> https://issues.apache.org/jira/browse/AMBARI-16952
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In case the hdp-select command fails during a service / component 
> installation there's no contextual information about the cause of the failure.
> This issue is for logging information about the machine on which the 
> hdp-select command fails.
> This solution wraps hdp-select command calls in a try/catch block and logs 
> failure / hdp installationrelated information.
> 
> The patch only applies for 2.2-next versions.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
>  9a3201e 
> 
> Diff: https://reviews.apache.org/r/48044/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> Manual testing underway.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 48273: AMBARI-17054 : Configure Atlas Ranger Plugin

[Gautam Borad]

> On June 6, 2016, 5:09 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py,
> >  lines 74-78
> > 
> >
> > You're uploading things to HDFS on Ranger start; but this will fail on 
> > any stack upgrade as HDFS is started after Ranger.

@Jonathan -- are you referring to HDFS directory creation logic (for Audit to 
HDFS purpose) on restart of Atlas Metadata Server?

Are you suggesting to skip  HDFS directory creation during upgrade?


> On June 6, 2016, 5:09 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py,
> >  lines 31-33
> > 
> >
> > Doesn't Ranger start before HDFS?

Can you please elaborate the issue here? 
This condition is for : if Audit to HDFS is enabled then it checks if Namenode 
is present on cluster or not and then it tries to create a HDFS directory on 
Restart of Atlas Service only.


- Gautam


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48273/#review136305
---


On June 6, 2016, 3:26 p.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48273/
> ---
> 
> (Updated June 6, 2016, 3:26 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Srimanth 
> Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17054
> https://issues.apache.org/jira/browse/AMBARI-17054
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Patch contains changes to support Ranger Atlas Plugin from Ambari.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  34d1a1a 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  1e9e7a7 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  a79a456 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/setup_ranger_atlas.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py
>  4c54214 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  4db7f45 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/themes/theme_version_3.json
>  0f7b0c0 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  8b669e8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  98cbc30 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-plugin-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-policymgr-ssl.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-security.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 379c268 
> 
> Diff: https://reviews.apache.org/r/48273/diff/
> 
> 
> Testing
> ---
> 
> Verified : 
> 
> 1) Enable / disable actions for Ranger Atlas Plugin
> 2) Checked Ranger plugin comminication. 
> 3) Checked policy enforecements on Atlas based on Ranger's policies.
> 4) Verified stack advisor recommendations to chnange following properties : 
> 
> atlas.authorizer.impl=org.apache.ranger.authorization.atlas.authorizer.RangerAtlasAuthorizer
> ranger-atlas-plugin-enabled=yes
> 5) Verified Ranger service creation (while enabling Ranger Atlas Plugin)
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 7, 2016, 8:17 a.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Changes
---

Reverted the last version of the patch.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description (updated)
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.

The patch only applies for 2.2-next versions.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing
---

Unit tests passed.
Manual testing underway.


Thanks,

Laszlo Puskas

Review Request 48321: Ambari server failed to start METRICS_COLLECTOR via BP

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48321/
---

Review request for Ambari, Andrew Onischuk, Robert Nettleton, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-17082
https://issues.apache.org/jira/browse/AMBARI-17082


Repository: ambari


Description
---

If blueprint contains 2 AMS collectors, Ambari server failed to start 
METRICS_COLLECTOR with

ERROR [pool-16-thread-1] TopologyManager:782 - 
TopologyManager.ConfigureClusterTask: An exception occurred while attempting to 
process cluster configs and set on cluster:
java.lang.IllegalArgumentException: Unable to update configuration property 
'timeline.metrics.service.webapp.address' with topology information. Component 
'METRICS_COLLECTOR' is mapped to an invalid number of hosts '2'.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 a0af813 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 8b1a9a6 

Diff: https://reviews.apache.org/r/48321/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 7, 2016, 8:03 a.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing
---

Unit tests passed.
Manual testing underway.


Thanks,

Laszlo Puskas

Re: Review Request 48044: Provide context for hdp-select failures during ambari component install

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48044/
---

(Updated June 7, 2016, 8:02 a.m.)


Review request for Ambari, Andrew Onischuk, Oliver Szabo, Sumit Mohanty, and 
Sebastian Toader.


Bugs: AMBARI-16952
https://issues.apache.org/jira/browse/AMBARI-16952


Repository: ambari


Description
---

In case the hdp-select command fails during a service / component installation 
there's no contextual information about the cause of the failure.
This issue is for logging information about the machine on which the hdp-select 
command fails.
This solution wraps hdp-select command calls in a try/catch block and logs 
failure / hdp installationrelated information.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
 9a3201e 

Diff: https://reviews.apache.org/r/48044/diff/


Testing (updated)
---

Unit tests passed.
Manual testing underway.


Thanks,

Laszlo Puskas

Re: Review Request 48266: Add explicit ambari-server log line indicating cluster creation complete

[Daniel Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48266/
---

(Updated jún. 7, 2016, 7:47 de)


Review request for Ambari, Laszlo Puskas, Oliver Szabo, Robert Nettleton, 
Sandor Magyari, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-17053
https://issues.apache.org/jira/browse/AMBARI-17053


Repository: ambari


Description
---

Add a log message to see when the cluster is ready to use (cluster creation 
finishes).

An event after each heartbeat. At that time cluster provision request is 
checked if it is finished or not. In case of an ambari server restart, the 
replayed requests are used to determine which one was the provision request. I 
could not find a nice way to do it, but I could make a workaround.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
 c6036c2 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java
 3feac55 
  ambari-server/src/main/java/org/apache/ambari/server/events/AmbariEvent.java 
1079806 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 e3f5b49 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
 fd8653c 

Diff: https://reviews.apache.org/r/48266/diff/


Testing
---

Still running locally...


Thanks,

Daniel Gergely

Review Request 48319: Hive View : Upload Table : Support for Date and Timestamp type detection according to hive specs

[Nitiraj Rathore]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48319/
---

Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, 
Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-17081
https://issues.apache.org/jira/browse/AMBARI-17081


Repository: ambari


Description
---

added regex check and casting to hive supported format for detection of date 
and timestamp type.


Diffs
-

  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/parsers/ParseUtils.java
 3261bfa 
  
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/parsers/Parser.java
 450d896 
  
contrib/views/hive/src/test/java/org/apache/ambari/view/hive/resources/upload/ParseUtilsTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/48319/diff/


Testing
---

Unit test case added for date detection.


Thanks,

Nitiraj Rathore

Re: Review Request 47941: [AMBARI-16920] Spark2 thrift server can not started due to miss of spark-thrift-fairscheduler.xml

[Jeff Zhang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47941/
---

(Updated June 7, 2016, 6:16 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Changes
---

Address the comments


Bugs: AMBARI-16920
https://issues.apache.org/jira/browse/AMBARI-16920


Repository: ambari


Description
---

This a followup ticket for add spark2 stack definition. There's serveral issues:
1.  Spark2 thrift server can not started due to miss of 
spark-thrift-fairscheduler.xml
2.  Miss of add spark2 cache file in copy_barball.py
3.  Miss the role_commnad_order of spark2


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 286df8d 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
 3316f78 
  
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
 2eae3e7 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
1b33345 

Diff: https://reviews.apache.org/r/47941/diff/


Testing
---

Manually verified.


Thanks,

Jeff Zhang

Re: Review Request 48065: AMBARI-16949 Metrics Collector API shows NPE if we use wildcard (%25 for '%') for metric name

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48065/#review136413
---


Ship it!




- Sid Wagle


On June 3, 2016, 12:43 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48065/
> ---
> 
> (Updated June 3, 2016, 12:43 a.m.)
> 
> 
> Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.
> 
> 
> Bugs: AMBARI-16949
> https://issues.apache.org/jira/browse/AMBARI-16949
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When we request '/ws/v1/timeline/metrics' with metric name which contains %25 
> (escaped '%'), response for the API is json describing there's NPE.
> And NPE is logged for ambari-metrics-collector log file.
> 
> ```
> 2016-05-30 09:15:05,061 WARN 
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.appendAggregateMetricFromResultSet(PhoenixHBaseAccessor.java:810)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.PhoenixHBaseAccessor.getAggregateMetricRecords(PhoenixHBaseAccessor.java:772)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.HBaseTimelineMetricStore.getTimelineMetrics(HBaseTimelineMetricStore.java:178)
>   at 
> org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TimelineWebServices.getTimelineMetrics(TimelineWebServices.java:372)
>   at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
> ```
> 
> Reason of NPE: 
> 
> Metrics are properly fetched with wildcard. But when applying functions to 
> result set, actual metric name is not exist from map of metric name to list 
> of function.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  47962cb 
> 
> Diff: https://reviews.apache.org/r/48065/diff/
> 
> 
> Testing
> ---
> 
> Test failed from local but it's occurred from another modules, which is not 
> related to current modification.
> 
> ```
> Results :
> 
> Failed tests:
>   PrivilegeEventCreatorTest.putTest:107 expected:<...), Roles(
> Permission[2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...), Roles(
> Permission[1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   RepositoryVersionEventCreatorTest.postTest:70 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   RepositoryVersionEventCreatorTest.putTest:100 expected:<...ating system: 
> redhat[6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6.com)
> Operating system: redhat7
> Repository ID(1), Repository name(MyRepo), Base url(http://example].com)
> )> but was:<...ating system: redhat[7
> Repository ID(1), Repository name(MyRepo), Base url(http://example.com)
> Operating system: redhat6
> Repository ID(2), Repository name(MyRepo6), Base url(http://example6].com)
> )>
>   ViewPrivilegeEventCreatorTest.putTest:85 expected:<...tatus(200 OK), 
> Type([MyView), Version(MyView), Name(MyView), Permissions(
> Permission2:
>   Users: testuser2
> Permission1:
>   Users: testuser
>   Groups: testgroup])> but was:<...tatus(200 OK), Type([null), Version(null), 
> Name(null), Permissions(
> Permission1:
>   Users: testuser
>   Groups: testgroup
> Permission2:
>   Users: testuser2])>
>   
> ComponentResourceProviderTest.testGetResourcesAsAdministrator:190->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsClusterAdministrator:195->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> ComponentResourceProviderTest.testGetResourcesAsServiceAdministrator:200->testGetResources:296
>  expected:<[tru]e> but was:<[fals]e>
>   
> AmbariLdapDataPopulatorTest.testSynchronizeExistingLdapGroups_removeDuringIteration:333
>   Expectation failure on verify:
> AmbariLdapDataPopulatorTestInstance.getLdapGroupByMemberAttr("group2"): 
> expected: 1, actual: 0
>   UpgradeCatalog222Test.testInitializeStromAndKafkaWidgets:1107
>   Unexpected method call 
> 

Re: Review Request 48030: AMBARI-16946 Storm Metrics Sink has high chance to discard some datapoints

[Sid Wagle]

> On June 7, 2016, 6:07 a.m., Sid Wagle wrote:
> > Changes look good, only thing to consider is the changes to the metric 
> > name. Cluster Aggregation will not occur at topology level since appId = 
> > topologyName for metrics with the same metric name. Is the metric name to 
> > fine grained? Only thing to consider is whether we need task metrics to be 
> > aggregated across topology? If yes, taskId cannot be part of the metric 
> > name. 
> > 
> > Could you also add Aravindan Vijayan to the reviewers? Thanks.

Rephrase: Cluster Aggregation will *now* occur at topology level


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48030/#review136411
---


On May 30, 2016, 8:21 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48030/
> ---
> 
> (Updated May 30, 2016, 8:21 a.m.)
> 
> 
> Review request for Ambari, Sriharsha Chintalapani and Sid Wagle.
> 
> 
> Bugs: AMBARI-16946
> https://issues.apache.org/jira/browse/AMBARI-16946
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There's a mismatch between TimelineMetricsCache and Storm metrics unit, while 
> TimelineMetricsCache considers "metric name + timestamp" to be unique but 
> Storm is not.
> 
> For example, assume that bolt B has task T1, T2 and B has registered metrics 
> M1. It's possible for metrics sink to receive (T1, M1) and (T2, M1) with same 
> timestamp TS1 (in TaskInfo, not current time), and received later will be 
> discarded from TimelineMetricsCache.
> 
> If we want to have unique metric point of Storm, we should use "topology name 
> + component name + task id + metric name" to metric name so that "metric name 
> + timestamp" will be unique.
> 
> There're other issues I would like to address, too.
> 
> - Currently, hostname is written to hostname of the machine which runs 
> metrics sink. Since TaskInfo has hostname of the machine which runs task, 
> we're better to use this.
> - Unit of timestamp of TaskInfo is second, while Storm Metrics Sink uses this 
> as millisecond, resulting in timestamp flaw, and malfunction of cache 
> eviction. It should be multiplied by 1000.
> - 'component name' is not unique across the cluster, so it's not fit for app 
> id. 'topology name' is unique so proper value of app id is topology name.
> 
> Consideration: Hostname for determining 'write shard' is set to hostname of 
> the machine which runs metrics sink. Since I don't know read also be sharded, 
> I'm not sure it's safe to use TaskInfo's hostname to hostname of 
> TimelineMetric. Please review carefully regarding this.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  02f5598 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  8171a4d 
> 
> Diff: https://reviews.apache.org/r/48030/diff/
> 
> 
> Testing
> ---
> 
> I tested this only ambari-metrics module since changeset is not related on 
> other modules.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] ambari-metrics . SUCCESS [  0.896 
> s]
> [INFO] Ambari Metrics Common .. SUCCESS [ 13.386 
> s]
> [INFO] Ambari Metrics Hadoop Sink . SUCCESS [  5.085 
> s]
> [INFO] Ambari Metrics Flume Sink .. SUCCESS [  6.827 
> s]
> [INFO] Ambari Metrics Kafka Sink .. SUCCESS [  4.190 
> s]
> [INFO] Ambari Metrics Storm Sink .. SUCCESS [  1.384 
> s]
> [INFO] Ambari Metrics Collector ... SUCCESS [04:06 
> min]
> [INFO] Ambari Metrics Monitor . SUCCESS [  3.556 
> s]
> [INFO] Ambari Metrics Grafana . SUCCESS [01:03 
> min]
> [INFO] Ambari Metrics Assembly  SUCCESS [  3.567 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 05:48 min
> [INFO] Finished at: 2016-05-30T16:46:07+09:00
> [INFO] Final Memory: 78M/1038M
> [INFO] 
> 
> 
> In fact, I tried to run `mvn test` from ambari root directory but build is 
> failing from ambari-web.
> 
> ```
> > fsevents@0.2.1 install 
> > /Users/jlim/WorkArea/JavaProjects/ambari/ambari-web/node_modules/chokidar/node_modules/fsevents
> >