Re: Review Request 50059: ArrayIndexOutOfBounds exception thrown on shard calculation

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50059/
---

(Updated July 15, 2016, 5:37 a.m.)


Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-17726
https://issues.apache.org/jira/browse/AMBARI-17726


Repository: ambari


Description
---

java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.elementData(ArrayList.java:418)
at java.util.ArrayList.get(ArrayList.java:431)
at 
org.apache.hadoop.metrics2.sink.timeline.availability.MetricSinkWriteShardHostnameHashingStrategy.findCollectorShard(MetricSinkWriteShardHostnameHashingStrategy.java:40)
at 
org.apache.hadoop.metrics2.sink.timeline.AbstractTimelineMetricsSink$1.get(AbstractTimelineMetricsSink.java:371)
at


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 f0cea7a 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricSinkWriteShardHostnameHashingStrategy.java
 1c89884 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/ShardingStrategyTest.java
 c6041db 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
 84e4153 

Diff: https://reviews.apache.org/r/50059/diff/


Testing
---

Unit test passed.


Thanks,

Sid Wagle

Review Request 50063: AMBARI-17725: AMS Storm Sink: Storm topology level metrics should have prefix to distinguish cluster level metrics

[Jungtaek Lim]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50063/
---

Review request for Ambari, Aravindan Vijayan, Sriharsha Chintalapani, and Sid 
Wagle.


Bugs: AMBARI-17725
https://issues.apache.org/jira/browse/AMBARI-17725


Repository: ambari


Description
---

Since Ambari 2.4.0 we have two kinds of Storm metrics into AMS which one is 
cluster level and another one is topology level.
Currently we don't add prefix on them which makes them hard to distinguish, 
which should be needed to create topology level dashboard in Grafana.

To resolve this we should add the prefix to Storm topology level metrics.

This patch adds 'topology.' as prefix of topology level metrics so that it can 
be distinguished.

Also shows warning message to user (in Storm worker) to notice period ('.') in 
topology name is not recommended for AMS.


Diffs
-

  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 9e6cc98 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
 271c11f 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 dcc3192 
  
ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
 c76197b 

Diff: https://reviews.apache.org/r/50063/diff/


Testing
---

Build OK via `mvn clean install` for both of storm-sink and storm-sink-legacy.
Tested manually.


Thanks,

Jungtaek Lim

Review Request 50059: ArrayIndexOutOfBounds exception thrown on shard calculation

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50059/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-17726
https://issues.apache.org/jira/browse/AMBARI-17726


Repository: ambari


Description
---

java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.elementData(ArrayList.java:418)
at java.util.ArrayList.get(ArrayList.java:431)
at 
org.apache.hadoop.metrics2.sink.timeline.availability.MetricSinkWriteShardHostnameHashingStrategy.findCollectorShard(MetricSinkWriteShardHostnameHashingStrategy.java:40)
at 
org.apache.hadoop.metrics2.sink.timeline.AbstractTimelineMetricsSink$1.get(AbstractTimelineMetricsSink.java:371)
at


Diffs
-

  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/AbstractTimelineMetricsSink.java
 f0cea7a 
  
ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/availability/MetricSinkWriteShardHostnameHashingStrategy.java
 1c89884 
  
ambari-metrics/ambari-metrics-common/src/test/java/org/apache/hadoop/metrics2/sink/timeline/availability/ShardingStrategyTest.java
 c6041db 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
 84e4153 

Diff: https://reviews.apache.org/r/50059/diff/


Testing
---

Unit test passed.


Thanks,

Sid Wagle

Re: Review Request 49838: AMBARI-17636 Service Configs page: can't see all config versions in dropdown

[Vivek Ratnavel Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49838/
---

(Updated July 15, 2016, 1:55 a.m.)


Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.


Changes
---

* Fixed a bug (Top bar buttons not working) 
* Handled click event in dropdown row view
* Ambari web tests pass
  29021 tests complete (25 seconds)
  154 tests pending
* Manual testing done


Bugs: AMBARI-17636
https://issues.apache.org/jira/browse/AMBARI-17636


Repository: ambari


Description
---

* Go to Service->Configs page (that have more than ~ 30 versions)
* Open versions dropdown

*Result:*
Number of visible versions limited by window height.


Diffs (updated)
-

  ambari-web/app/styles/config_history_flow.less fff3151 
  ambari-web/app/templates/common/configs/config_history_dropdown_row.hbs 
PRE-CREATION 
  ambari-web/app/templates/common/configs/config_history_flow.hbs 73d6913 
  ambari-web/app/views/common/configs/config_history_flow.js ad2b1ab 

Diff: https://reviews.apache.org/r/49838/diff/


Testing
---

Local Ambari web test passed.
28944 tests complete (25 seconds)
154 tests pending
Manual testing done.


File Attachments


Gif of latest changes
  
https://reviews.apache.org/media/uploaded/files/2016/07/14/225a1346-1632-4471-bbf2-99bf05c5d275__bugfix.gif


Thanks,

Vivek Ratnavel Subramanian

Re: Review Request 49935: AMBARI-17633 yarn.nodemanager.remote-app-log-dir should be added stickybit.

[Akira Ajisaka]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49935/#review142307
---


Ship it!




Built Hadoop cluster with Apache Ambari with editing the yarn.py. The 
permission of the directory is correct after the patch. +1 (non-binding)

- Akira Ajisaka


On 7月 12, 2016, 9:55 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49935/
> ---
> 
> (Updated 7月 12, 2016, 9:55 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, jun aoki, and Myroslav 
> Papirkovskyy.
> 
> 
> Bugs: AMBARI-17633
> https://issues.apache.org/jira/browse/AMBARI-17633
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When installing YARN+MapReduce2, I got a WARN message in nodemanager log(in 
> Log Search View) like below
> ```
> 2016-07-09 06:30:21,865 WARN logaggregation.LogAggregationService 
> LogAggregationService.java:230 - Remote Root Log Dir [/app-logs] already 
> exist, but with incorrect permissions. Expected: [rwxrwxrwt], Found: 
> [rwxrwxrwx]. The cluster may have problems with multiple users. 
> ```
> I think the cause of this WARN is 
> [this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py#L115).
> We should add stickybit to yarn.nodemanager.remote-app-log-dir.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  cf5736e 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 
> 4600063 
> 
> Diff: https://reviews.apache.org/r/49935/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test && manual test
> 
> I think Hadoop QA result is unrelated to this patch
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49838: AMBARI-17636 Service Configs page: can't see all config versions in dropdown

[Vivek Ratnavel Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49838/
---

(Updated July 14, 2016, 10:49 p.m.)


Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.


Changes
---

Video showing the change in UI


Bugs: AMBARI-17636
https://issues.apache.org/jira/browse/AMBARI-17636


Repository: ambari


Description
---

* Go to Service->Configs page (that have more than ~ 30 versions)
* Open versions dropdown

*Result:*
Number of visible versions limited by window height.


Diffs
-

  ambari-web/app/styles/config_history_flow.less fff3151 
  ambari-web/app/templates/common/configs/config_history_dropdown_row.hbs 
PRE-CREATION 
  ambari-web/app/templates/common/configs/config_history_flow.hbs 73d6913 
  ambari-web/app/views/common/configs/config_history_flow.js ad2b1ab 

Diff: https://reviews.apache.org/r/49838/diff/


Testing
---

Local Ambari web test passed.
28944 tests complete (25 seconds)
154 tests pending
Manual testing done.


File Attachments (updated)


Gif of latest changes
  
https://reviews.apache.org/media/uploaded/files/2016/07/14/225a1346-1632-4471-bbf2-99bf05c5d275__bugfix.gif


Thanks,

Vivek Ratnavel Subramanian

Re: Review Request 49838: AMBARI-17636 Service Configs page: can't see all config versions in dropdown

[Zhe (Joe) Wang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49838/#review142302
---




ambari-web/app/views/common/configs/config_history_flow.js (line 305)


this.get('hoveredServiceVersion.version')


- Zhe (Joe) Wang


On July 14, 2016, 10:21 p.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49838/
> ---
> 
> (Updated July 14, 2016, 10:21 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17636
> https://issues.apache.org/jira/browse/AMBARI-17636
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Go to Service->Configs page (that have more than ~ 30 versions)
> * Open versions dropdown
> 
> *Result:*
> Number of visible versions limited by window height.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/styles/config_history_flow.less fff3151 
>   ambari-web/app/templates/common/configs/config_history_dropdown_row.hbs 
> PRE-CREATION 
>   ambari-web/app/templates/common/configs/config_history_flow.hbs 73d6913 
>   ambari-web/app/views/common/configs/config_history_flow.js ad2b1ab 
> 
> Diff: https://reviews.apache.org/r/49838/diff/
> 
> 
> Testing
> ---
> 
> Local Ambari web test passed.
> 28944 tests complete (25 seconds)
> 154 tests pending
> Manual testing done.
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Re: Review Request 50035: AMBARI-17714: Add Service Wizard/Security wizard : Configure Identities page does not fully load after upgrade from 2.1.1 to 2.4.0

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50035/#review142301
---


Ship it!




Ship It!

- Sumit Mohanty


On July 14, 2016, 5:59 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50035/
> ---
> 
> (Updated July 14, 2016, 5:59 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17714
> https://issues.apache.org/jira/browse/AMBARI-17714
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-17714: Add Service Wizard/Security wizard : Configure Identities page 
> does not fully load after upgrade from 2.1.1 to 2.4.0
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  10310d1c4326ce2c7868afa1390a240a04b7e78a 
> 
> Diff: https://reviews.apache.org/r/50035/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [10.946s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.036s]
> [INFO] Ambari Web  SUCCESS [25.301s]
> [INFO] Ambari Views .. SUCCESS [1.337s]
> [INFO] Ambari Admin View . SUCCESS [8.551s]
> [INFO] ambari-metrics  SUCCESS [0.677s]
> [INFO] Ambari Metrics Common . SUCCESS [3.345s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.068s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.228s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.232s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [4.569s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.432s]
> [INFO] Ambari Metrics Collector .. SUCCESS [9.999s]
> [INFO] Ambari Metrics Monitor  SUCCESS [2.193s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.196s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:26.727s]
> [INFO] Ambari Server . SUCCESS [3:10.861s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.118s]
> [INFO] Ambari Agent .. SUCCESS [25.833s]
> [INFO] Ambari Client . SUCCESS [0.084s]
> [INFO] Ambari Python Client .. SUCCESS [0.976s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.105s]
> [INFO] Ambari Shell .. SUCCESS [0.070s]
> [INFO] Ambari Python Shell ... SUCCESS [0.691s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.839s]
> [INFO] ambari-logsearch .. SUCCESS [0.231s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.346s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.236s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [5.884s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.772s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.100s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:38.416s
> [INFO] Finished at: Thu Jul 14 09:58:11 PDT 2016
> [INFO] Final Memory: 340M/1156M
> [INFO] 
> 
> 
> ** 2. mvn test -DskipPythonTests -Dtest=*Upgrade* **
> 
> Results :
> 
> Tests run: 252, Failures: 0, Errors: 0, Skipped: 9
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 10:00.595s
> [INFO] Finished at: Thu Jul 14 10:13:52 PDT 2016
> [INFO] Final Memory: 55M/881M
> [INFO] 
> 
> 
> ** 3. Manual tests **
> 
> Installed Ambari 2.1.1 with Atlas, Hive, etc. and then upgraded to ambari 
> 2.4.0 with the modified
> application-properties.xml and verified that the Configure Identities page 
> loaded successfully.
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 50048: System "boottime" metric is not being collected by AMS

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50048/#review142299
---


Ship it!




Ship It!

- Sumit Mohanty


On July 14, 2016, 9:18 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50048/
> ---
> 
> (Updated July 14, 2016, 9:18 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17600
> https://issues.apache.org/jira/browse/AMBARI-17600
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The System boottime metric used to be collected by AMS. It seems like it is 
> no longer collected.
> 
> {code}
>   "metrics/boottime":{
> "metric":"boottime",
> "pointInTime":true,
> "temporal":true
>   }
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py
>  9df2366 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/metrics.json
>  2f15888 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/metrics.json
>  158f3d7 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metrics.json 
> e71af2c 
>   ambari-server/src/main/resources/common-services/STORM/0.10.0/metrics.json 
> a040f8a 
>   ambari-server/src/main/resources/common-services/STORM/0.9.1/metrics.json 
> 08c2b50 
>   ambari-server/src/main/resources/common-services/STORM/0.9.3/metrics.json 
> e739e3a 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metrics.json 
> 604e545 
> 
> Diff: https://reviews.apache.org/r/50048/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> --
> Ran 12 tests in 0.023s
> 
> OK
> --
> Total run:12
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 49838: AMBARI-17636 Service Configs page: can't see all config versions in dropdown

[Vivek Ratnavel Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49838/
---

(Updated July 14, 2016, 10:21 p.m.)


Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.


Changes
---

* Applied a workaround for a bug 
(http://stackoverflow.com/questions/21369752/safari-positionfixed-doesnt-breaks-element-from-scope/21440899#21440899)
 in Safari. 
* Removed top and bottom scroll buttons to enable auto-scrolling.
* Displaying sub-menu popup with respect to the window height.
* Added Firefox specific styles to be consistent across all browsers.
* Handled sub-menu display manually instead of using the twitter bootstrap 
framework.
* Refactored code.
* Ambari web test passed
  29021 tests complete (25 seconds)
  154 tests pending
* Manual testing done in the latest versions of Google Chrome, Mozilla Firefox 
and Safari.


Bugs: AMBARI-17636
https://issues.apache.org/jira/browse/AMBARI-17636


Repository: ambari


Description
---

* Go to Service->Configs page (that have more than ~ 30 versions)
* Open versions dropdown

*Result:*
Number of visible versions limited by window height.


Diffs (updated)
-

  ambari-web/app/styles/config_history_flow.less fff3151 
  ambari-web/app/templates/common/configs/config_history_dropdown_row.hbs 
PRE-CREATION 
  ambari-web/app/templates/common/configs/config_history_flow.hbs 73d6913 
  ambari-web/app/views/common/configs/config_history_flow.js ad2b1ab 

Diff: https://reviews.apache.org/r/49838/diff/


Testing
---

Local Ambari web test passed.
28944 tests complete (25 seconds)
154 tests pending
Manual testing done.


Thanks,

Vivek Ratnavel Subramanian

Re: Review Request 50048: System "boottime" metric is not being collected by AMS

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50048/#review142295
---


Ship it!




Ship It!

- Aravindan Vijayan


On July 14, 2016, 9:18 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50048/
> ---
> 
> (Updated July 14, 2016, 9:18 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17600
> https://issues.apache.org/jira/browse/AMBARI-17600
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The System boottime metric used to be collected by AMS. It seems like it is 
> no longer collected.
> 
> {code}
>   "metrics/boottime":{
> "metric":"boottime",
> "pointInTime":true,
> "temporal":true
>   }
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py
>  9df2366 
>   
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/metrics.json
>  2f15888 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/metrics.json
>  158f3d7 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metrics.json 
> e71af2c 
>   ambari-server/src/main/resources/common-services/STORM/0.10.0/metrics.json 
> a040f8a 
>   ambari-server/src/main/resources/common-services/STORM/0.9.1/metrics.json 
> 08c2b50 
>   ambari-server/src/main/resources/common-services/STORM/0.9.3/metrics.json 
> e739e3a 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metrics.json 
> 604e545 
> 
> Diff: https://reviews.apache.org/r/50048/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> 
> --
> Ran 12 tests in 0.023s
> 
> OK
> --
> Total run:12
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Review Request 50048: System "boottime" metric is not being collected by AMS

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50048/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-17600
https://issues.apache.org/jira/browse/AMBARI-17600


Repository: ambari


Description
---

The System boottime metric used to be collected by AMS. It seems like it is no 
longer collected.

{code}
  "metrics/boottime":{
"metric":"boottime",
"pointInTime":true,
"temporal":true
  }
{code}


Diffs
-

  
ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py 
9df2366 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/metrics.json
 2f15888 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/metrics.json 
158f3d7 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metrics.json 
e71af2c 
  ambari-server/src/main/resources/common-services/STORM/0.10.0/metrics.json 
a040f8a 
  ambari-server/src/main/resources/common-services/STORM/0.9.1/metrics.json 
08c2b50 
  ambari-server/src/main/resources/common-services/STORM/0.9.3/metrics.json 
e739e3a 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/metrics.json 
604e545 

Diff: https://reviews.apache.org/r/50048/diff/


Testing
---

Manually verified.

--
Ran 12 tests in 0.023s

OK
--
Total run:12
Total errors:0
Total failures:0
OK


Thanks,

Sid Wagle

Review Request 50047: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50047/
---

Review request for Ambari, Di Li and Robert Levas.


Bugs: AMBARI-17694
https://issues.apache.org/jira/browse/AMBARI-17694


Repository: ambari


Description
---

When kerberos is enabled, the protocol for listeners in 
/etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
even though the Ambari UI shows otherwise


Diffs
-

  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 33275f9 
  ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
2b1c01b 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
2a2a3a3 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py a6baeea 

Diff: https://reviews.apache.org/r/50047/diff/


Testing
---

Added 1 new test case,
 Ran mvn test


Thanks,

Anita Jebaraj

Re: Review Request 50045: AMS HA not working in secure cluster

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50045/#review142283
---


Ship it!




Ship It!

- Aravindan Vijayan


On July 14, 2016, 8:39 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50045/
> ---
> 
> (Updated July 14, 2016, 8:39 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17716
> https://issues.apache.org/jira/browse/AMBARI-17716
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Helix not able to connect to Zk while starting up.
> 
> {code}
> 2016-07-13 17:56:41,263 INFO org.apache.helix.manager.zk.ZKHelixManager: 
> KeeperState: SyncConnected, zookeeper:State:CONNECTED Timeout:3 
> sessionid:0x355e190123e001d local:/10.240.0.32:54434 
> remoteserver:ambari-sid-3.c.pramod-thangali.internal/10.240.0.30:2181 
> lastZxid:8589
> 935298 xid:9 sent:9 recv:11 queuedpkts:0 pendingresp:0 queuedevents:0
> 
> 2016-07-13 17:57:41,264 ERROR org.apache.helix.manager.zk.ZKHelixManager: 
> fail to connect zkserver: 
> ambari-sid-1.c.pramod-thangali.internal:2181,ambari-sid-2.c.pramod-thangali.internal:2181,ambari-sid-3.c.pramod-thangali.internal:2181
>  in 6ms. expiredSessionId: null, clusterName: ambari-metrics-cluster
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-timelineservice/pom.xml 7ae6155 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  6ac5f45 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/AggregationTaskRunner.java
>  cad2a2b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
>  18b9059 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/OnlineOfflineStateModelFactory.java
>  7d3350b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAControllerTest.java
>  91ec305 
> 
> Diff: https://reviews.apache.org/r/50045/diff/
> 
> 
> Testing
> ---
> 
> Manually verified.
> Unit test pass.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 49995: AMBARI-17695 : AMS Split point calculation not optimal for large clusters.

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49995/
---

(Updated July 14, 2016, 8:26 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Changes
---

Disabling hbase normalizer feature for AMS


Bugs: AMBARI-17695
https://issues.apache.org/jira/browse/AMBARI-17695


Repository: ambari


Description
---

Use hbase.regionserver.global.memstore.upperLimit instead of lowerLimit for 
available memory for split points calculation. 

Change default region counts for METRIC_RECORD and METRIC_AGGREGATE to 2.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 b86f97a 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 4ddcfa1 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-site.xml
 a7d8228 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
 fa4deaf 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 819616b 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 86bf14d 

Diff: https://reviews.apache.org/r/49995/diff/


Testing
---

Manually tested.


Thanks,

Aravindan Vijayan

Review Request 50045: AMS HA not working in secure cluster

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50045/
---

Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-17716
https://issues.apache.org/jira/browse/AMBARI-17716


Repository: ambari


Description
---

Helix not able to connect to Zk while starting up.

{code}
2016-07-13 17:56:41,263 INFO org.apache.helix.manager.zk.ZKHelixManager: 
KeeperState: SyncConnected, zookeeper:State:CONNECTED Timeout:3 
sessionid:0x355e190123e001d local:/10.240.0.32:54434 
remoteserver:ambari-sid-3.c.pramod-thangali.internal/10.240.0.30:2181 
lastZxid:8589
935298 xid:9 sent:9 recv:11 queuedpkts:0 pendingresp:0 queuedevents:0

2016-07-13 17:57:41,264 ERROR org.apache.helix.manager.zk.ZKHelixManager: fail 
to connect zkserver: 
ambari-sid-1.c.pramod-thangali.internal:2181,ambari-sid-2.c.pramod-thangali.internal:2181,ambari-sid-3.c.pramod-thangali.internal:2181
 in 6ms. expiredSessionId: null, clusterName: ambari-metrics-cluster
{code}


Diffs
-

  ambari-metrics/ambari-metrics-timelineservice/pom.xml 7ae6155 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
 6ac5f45 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/AggregationTaskRunner.java
 cad2a2b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAController.java
 18b9059 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/OnlineOfflineStateModelFactory.java
 7d3350b 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/MetricCollectorHAControllerTest.java
 91ec305 

Diff: https://reviews.apache.org/r/50045/diff/


Testing
---

Manually verified.
Unit test pass.


Thanks,

Sid Wagle

Re: Review Request 49826: AUTH_TO_LOCAL rules are not updated when adding services to a Blueprint-installed cluster

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49826/
---

(Updated July 14, 2016, 3:15 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sebastian Toader.


Bugs: AMBARI-17629
https://issues.apache.org/jira/browse/AMBARI-17629


Repository: ambari


Description
---

When adding new services and components to a cluster that was initially created 
via Blueprints (rather than via the Ambari UI), auth-to-local rules that are 
expected to be created as indicated by the Kerberos descriptor are not.

It occurs because the components being installed are in the `INIT` state where 
the logic to determine whether to include the new auth-to-local rules or not 
expects the components to be in either the `INSTALLED` or `STARTED` states. 
This is due to logic added when resolving AMBARI-14232.

#Solution:
Update logic building auth-to-local rules to occur when new services and 
components are added to the cluster.  Logic assumes caller passes around 
explict lists of currently installed services and component (including added 
components), previously installed services and components, and a filter to 
indicate which services and components are newly added. 

This data will help avoid generating regressions for AMBARI-14232 and 
AMBARI-14506


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 872ec2d 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
 8170f17 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 cc48240 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
 f70c546 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java
 88f0e3c 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 66ed68d 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java
 58919b9 

Diff: https://reviews.apache.org/r/49826/diff/


Testing
---

Manually tested using various Blueprint and UI-based scenarios.

# Local unit tests: 

Tests run: 4536, Failures: 0, Errors: 0, Skipped: 34

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:12:50.002s
[INFO] Finished at: Wed Jul 13 13:33:27 EDT 2016
[INFO] Final Memory: 61M/1918M
[INFO] 

# Jenkins unit tests: PENDING


Thanks,

Robert Levas

Re: Review Request 50012: AMBARI-17702. Fix the stack_advisor so that LLAP can start when a nested queue is selected.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50012/#review142269
---


Ship it!




Ship It!

- Sumit Mohanty


On July 14, 2016, 5:46 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50012/
> ---
> 
> (Updated July 14, 2016, 5:46 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17702
> https://issues.apache.org/jira/browse/AMBARI-17702
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Earlier, we were putting the paths for the queues for selection. But YARN 
> needs the terminal queue name.
> 
> FIX: Updated the code to put only the terminal leaf queue names, instead of 
> complete path.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> c2dfd88 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 0e455e9 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> 40f3367 
> 
> Diff: https://reviews.apache.org/r/50012/diff/
> 
> 
> Testing
> ---
> 
> Modified Python UT.
> - Python UT passes.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/#review142266
---



Can we add any unit tests to cover these cases?

- Jayush Luniya


On July 14, 2016, 2:23 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49987/
> ---
> 
> (Updated July 14, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17687
> https://issues.apache.org/jira/browse/AMBARI-17687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Scenario:
> 1. install ranger with stack 2.4 and ambari-2.4.0
> 2. register the version of stack 2.5
> 3. perform the upgrade
> at the step where it where zookeepers are upgraded , just downgraded.
> 4. downgrade was successful 
> 5. check ranger service status , it was down
> try to restart the ranger but ranger startup is failing
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  0df8270 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  145c216 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  f24bd83 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  63a1100 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  f47eee4 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7e0477f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  fad4b9b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dfcad32 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  5941ca1 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  44a0446 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 4c122e9 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> bbd0fe4 
> 
> Diff: https://reviews.apache.org/r/49987/diff/
> 
> 
> Testing
> ---
> 
> Test Ranger Downgrade from stack 2.5 to 2.4
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 50030: AMBARI-17688 Ranger stack changes in Ambari to support secure Solr

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50030/#review142261
---




ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 (line 191)


`master_keytab_path` is not be set/created if `securiry_enabled` is 
`false`.  This may cause an issue down the road if some other variable needs 
it... For Example:

```
ranger_hbase_keytab = master_keytab_path
```

Note: the above example will only exeucte if `security_enabled` is `true`; 
but this is still a possible issue.



ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 (line 630)


Whynot use `hive_server2_keytab` from line 336?

```
hive_server2_keytab = 
config['configurations']['hive-site']['hive.server2.authentication.kerberos.keytab']
```



ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 (line 239)


please verify the indent on this line, it appears to be a character off.



ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py (line 
1504)


The existance of KERBEROS in the services list does not indicate whether 
Kerberos is enabled or not.  Use  `cluster-env/security_enabled`, which used by 
the `isSecurityEnabled` - see 
`stacks.stack_advisor.DefaultStackAdvisor#isSecurityEnabled`



ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
(lines 1629 - 1632)


The existance or non-existance of the KERBEROS service should not be used 
to determine if Kerberos is enabled.  Use 
`stacks.stack_advisor.DefaultStackAdvisor#isSecurityEnabled` instead.


- Robert Levas


On July 14, 2016, 9:08 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50030/
> ---
> 
> (Updated July 14, 2016, 9:08 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Robert Levas, Srimanth Gunturi, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-17688
> https://issues.apache.org/jira/browse/AMBARI-17688
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Below properties should be introduced to Ranger admin and plugins to support 
> secure Solr. Also recommend for plugin in any of the below property chnages 
> on Ranger Admin:
> 
> xasecure.audit.jaas.Client.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
> xasecure.audit.jaas.Client.loginModuleControlFlag=required
> xasecure.audit.jaas.Client.option.useKeyTab=true
> xasecure.audit.jaas.Client.option.storeKey=false
> xasecure.audit.jaas.Client.option.serviceName=solr
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  145c216 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  63a1100 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  fad4b9b 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py
>  529ac8c 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
>  1b2b5e0 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  341cff7 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/themes/theme_version_3.json
>  3f50774 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dfcad32 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
>  d3f9143 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> e570a5b7 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 2a2a3a3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
>  efeea5f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/configuration/ranger-hbase-audit.xml
>  d3f9143 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/ranger-hdfs-audit.xml
>  019602a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/ranger-hive-audit.xml
>  d3f9143 
>   
> 

Re: Review Request 49905: AMBARI-17645 Update JDK version of Dockerfile in dev-support

[Masahiro Tanaka]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49905/
---

(Updated July 15, 2016, 3:24 a.m.)


Review request for Ambari, Jaimin Jetly, jun aoki, and Sumit Mohanty.


Changes
---

Update a patch


Bugs: AMBARI-17645
https://issues.apache.org/jira/browse/AMBARI-17645


Repository: ambari


Description
---

The Dockerfile in dev-support uses Java 7 while its support is finished. It 
would be nice it is Java 8 for developing purpose.


Diffs (updated)
-

  dev-support/docker/docker/Dockerfile 802d20a 

Diff: https://reviews.apache.org/r/49905/diff/


Testing
---

Hadooop QAed & manual tests done

```
-1 overall. Here are the results of testing the latest attachment 
http://issues.apache.org/jira/secure/attachment/12817064/AMBARI-17645.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac 
compiler warnings.
+1 release audit. The applied patch does not increase the total number of 
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7768//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7768//console
This message is automatically generated.
```


Thanks,

Masahiro Tanaka

Re: Review Request 49995: AMBARI-17695 : AMS Split point calculation not optimal for large clusters.

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49995/
---

(Updated July 14, 2016, 6:23 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-17695
https://issues.apache.org/jira/browse/AMBARI-17695


Repository: ambari


Description
---

Use hbase.regionserver.global.memstore.upperLimit instead of lowerLimit for 
available memory for split points calculation. 

Change default region counts for METRIC_RECORD and METRIC_AGGREGATE to 2.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 b86f97a 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
 fa4deaf 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 86bf14d 

Diff: https://reviews.apache.org/r/49995/diff/


Testing
---

Manually tested.


Thanks,

Aravindan Vijayan

Re: Review Request 49995: AMBARI-17695 : AMS Split point calculation not optimal for large clusters.

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49995/#review142260
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
 (line 108)


Can you leave a comment of 15% implicity give to MRM, not clear if you look 
at the percentages.


- Sid Wagle


On July 14, 2016, 5:59 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49995/
> ---
> 
> (Updated July 14, 2016, 5:59 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17695
> https://issues.apache.org/jira/browse/AMBARI-17695
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Use hbase.regionserver.global.memstore.upperLimit instead of lowerLimit for 
> available memory for split points calculation. 
> 
> Change default region counts for METRIC_RECORD and METRIC_AGGREGATE to 2.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  b86f97a 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
>  fa4deaf 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 86bf14d 
> 
> Diff: https://reviews.apache.org/r/49995/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 50035: AMBARI-17714: Add Service Wizard/Security wizard : Configure Identities page does not fully load after upgrade from 2.1.1 to 2.4.0

[Nahappan Somasundaram]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50035/
---

(Updated July 14, 2016, 10:59 a.m.)


Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Changes
---

Revert properties to their original true value which were set to false in 
AMBARI-17616.


Bugs: AMBARI-17714
https://issues.apache.org/jira/browse/AMBARI-17714


Repository: ambari


Description
---

AMBARI-17714: Add Service Wizard/Security wizard : Configure Identities page 
does not fully load after upgrade from 2.1.1 to 2.4.0


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 10310d1c4326ce2c7868afa1390a240a04b7e78a 

Diff: https://reviews.apache.org/r/50035/diff/


Testing
---

** 1. mvn clean install **

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [10.946s]
[INFO] Apache Ambari Project POM . SUCCESS [0.036s]
[INFO] Ambari Web  SUCCESS [25.301s]
[INFO] Ambari Views .. SUCCESS [1.337s]
[INFO] Ambari Admin View . SUCCESS [8.551s]
[INFO] ambari-metrics  SUCCESS [0.677s]
[INFO] Ambari Metrics Common . SUCCESS [3.345s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.068s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [1.228s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [1.232s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [4.569s]
[INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.432s]
[INFO] Ambari Metrics Collector .. SUCCESS [9.999s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.193s]
[INFO] Ambari Metrics Grafana  SUCCESS [1.196s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:26.727s]
[INFO] Ambari Server . SUCCESS [3:10.861s]
[INFO] Ambari Functional Tests ... SUCCESS [2.118s]
[INFO] Ambari Agent .. SUCCESS [25.833s]
[INFO] Ambari Client . SUCCESS [0.084s]
[INFO] Ambari Python Client .. SUCCESS [0.976s]
[INFO] Ambari Groovy Client .. SUCCESS [2.105s]
[INFO] Ambari Shell .. SUCCESS [0.070s]
[INFO] Ambari Python Shell ... SUCCESS [0.691s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.839s]
[INFO] ambari-logsearch .. SUCCESS [0.231s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.346s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.236s]
[INFO] Ambari Logsearch Portal ... SUCCESS [5.884s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.772s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.100s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 6:38.416s
[INFO] Finished at: Thu Jul 14 09:58:11 PDT 2016
[INFO] Final Memory: 340M/1156M
[INFO] 

** 2. mvn test -DskipPythonTests -Dtest=*Upgrade* **

Results :

Tests run: 252, Failures: 0, Errors: 0, Skipped: 9

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 10:00.595s
[INFO] Finished at: Thu Jul 14 10:13:52 PDT 2016
[INFO] Final Memory: 55M/881M
[INFO] 

** 3. Manual tests **

Installed Ambari 2.1.1 with Atlas, Hive, etc. and then upgraded to ambari 2.4.0 
with the modified
application-properties.xml and verified that the Configure Identities page 
loaded successfully.


Thanks,

Nahappan Somasundaram

Re: Review Request 49995: AMBARI-17695 : AMS Split point calculation not optimal for large clusters.

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49995/
---

(Updated July 14, 2016, 5:59 p.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-17695
https://issues.apache.org/jira/browse/AMBARI-17695


Repository: ambari


Description
---

Use hbase.regionserver.global.memstore.upperLimit instead of lowerLimit for 
available memory for split points calculation. 

Change default region counts for METRIC_RECORD and METRIC_AGGREGATE to 2.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 b86f97a 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/split_points.py
 fa4deaf 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 86bf14d 

Diff: https://reviews.apache.org/r/49995/diff/


Testing
---

Manually tested.


Thanks,

Aravindan Vijayan

Re: Review Request 50012: AMBARI-17702. Fix the stack_advisor so that LLAP can start when a nested queue is selected.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50012/
---

(Updated July 14, 2016, 5:46 p.m.)


Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Changes
---

Changed the check 'from a subtring in a given string' to 'endswith' a given 
substring while looking for state and capacity keys.


Bugs: AMBARI-17702
https://issues.apache.org/jira/browse/AMBARI-17702


Repository: ambari


Description
---

Earlier, we were putting the paths for the queues for selection. But YARN needs 
the terminal queue name.

FIX: Updated the code to put only the terminal leaf queue names, instead of 
complete path.


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
c2dfd88 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
0e455e9 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 40f3367 

Diff: https://reviews.apache.org/r/50012/diff/


Testing
---

Modified Python UT.
- Python UT passes.


Thanks,

Swapan Shridhar

Review Request 50035: AMBARI-17714: Add Service Wizard/Security wizard : Configure Identities page does not fully load after upgrade from 2.1.1 to 2.4.0

[Nahappan Somasundaram]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50035/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-17714
https://issues.apache.org/jira/browse/AMBARI-17714


Repository: ambari


Description
---

AMBARI-17714: Add Service Wizard/Security wizard : Configure Identities page 
does not fully load after upgrade from 2.1.1 to 2.4.0


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 10310d1c4326ce2c7868afa1390a240a04b7e78a 

Diff: https://reviews.apache.org/r/50035/diff/


Testing
---

** 1. mvn clean install **

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [10.946s]
[INFO] Apache Ambari Project POM . SUCCESS [0.036s]
[INFO] Ambari Web  SUCCESS [25.301s]
[INFO] Ambari Views .. SUCCESS [1.337s]
[INFO] Ambari Admin View . SUCCESS [8.551s]
[INFO] ambari-metrics  SUCCESS [0.677s]
[INFO] Ambari Metrics Common . SUCCESS [3.345s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [2.068s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [1.228s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [1.232s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [4.569s]
[INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.432s]
[INFO] Ambari Metrics Collector .. SUCCESS [9.999s]
[INFO] Ambari Metrics Monitor  SUCCESS [2.193s]
[INFO] Ambari Metrics Grafana  SUCCESS [1.196s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:26.727s]
[INFO] Ambari Server . SUCCESS [3:10.861s]
[INFO] Ambari Functional Tests ... SUCCESS [2.118s]
[INFO] Ambari Agent .. SUCCESS [25.833s]
[INFO] Ambari Client . SUCCESS [0.084s]
[INFO] Ambari Python Client .. SUCCESS [0.976s]
[INFO] Ambari Groovy Client .. SUCCESS [2.105s]
[INFO] Ambari Shell .. SUCCESS [0.070s]
[INFO] Ambari Python Shell ... SUCCESS [0.691s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.839s]
[INFO] ambari-logsearch .. SUCCESS [0.231s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.346s]
[INFO] Ambari Logsearch Solr Client .. SUCCESS [1.236s]
[INFO] Ambari Logsearch Portal ... SUCCESS [5.884s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [3.772s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.100s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 6:38.416s
[INFO] Finished at: Thu Jul 14 09:58:11 PDT 2016
[INFO] Final Memory: 340M/1156M
[INFO] 

** 2. mvn test -DskipPythonTests -Dtest=*Upgrade* **

Results :

Tests run: 252, Failures: 0, Errors: 0, Skipped: 9

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 10:00.595s
[INFO] Finished at: Thu Jul 14 10:13:52 PDT 2016
[INFO] Final Memory: 55M/881M
[INFO] 

** 3. Manual tests **

Installed Ambari 2.1.1 with Atlas, Hive, etc. and then upgraded to ambari 2.4.0 
with the modified
application-properties.xml and verified that the Configure Identities page 
loaded successfully.


Thanks,

Nahappan Somasundaram

Re: Review Request 49930: AMBARI-17614 Clean up import * for AMBARI_METRICS services

[Masahiro Tanaka]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49930/#review142255
---



Thanks Juanjo Marron and Aravindan Vijayan!

- Masahiro Tanaka


On July 12, 2016, 7:59 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49930/
> ---
> 
> (Updated July 12, 2016, 7:59 a.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jayush Luniya, and Juanjo  
> Marron.
> 
> 
> Bugs: AMBARI-17614
> https://issues.apache.org/jira/browse/AMBARI-17614
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py
>  and 
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py
> uses `from resource_management import *`. It increases code tracking 
> difficulty.
> I think this is related to AMBARI-16101
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py
>  7dc66b2 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status.py
>  fa4e8a4 
> 
> Diff: https://reviews.apache.org/r/49930/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 50031: Error for Pig view after upgrade to 2.4.0.0 [PersistenceException: Can't initialize data store for view PIG]

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50031/#review142245
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On July 14, 2016, 1:09 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50031/
> ---
> 
> (Updated July 14, 2016, 1:09 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17710
> https://issues.apache.org/jira/browse/AMBARI-17710
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update SmokeTestEntity class name in viewentity table.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  67495b5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  54f0e57 
> 
> Diff: https://reviews.apache.org/r/50031/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/#review142242
---


Ship it!




Ship It!

- Jonathan Hurley


On July 14, 2016, 10:23 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49987/
> ---
> 
> (Updated July 14, 2016, 10:23 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17687
> https://issues.apache.org/jira/browse/AMBARI-17687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Scenario:
> 1. install ranger with stack 2.4 and ambari-2.4.0
> 2. register the version of stack 2.5
> 3. perform the upgrade
> at the step where it where zookeepers are upgraded , just downgraded.
> 4. downgrade was successful 
> 5. check ranger service status , it was down
> try to restart the ranger but ranger startup is failing
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  0df8270 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  145c216 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  f24bd83 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  63a1100 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  f47eee4 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7e0477f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  fad4b9b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dfcad32 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  5941ca1 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  44a0446 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 4c122e9 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> bbd0fe4 
> 
> Diff: https://reviews.apache.org/r/49987/diff/
> 
> 
> Testing
> ---
> 
> Test Ranger Downgrade from stack 2.5 to 2.4
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Mugdha Varadkar]

> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > I'm curious if you can make this simply by only using the 
> > `commandParams/version` property when calculating what to do. Here's an 
> > example of the various values during normal commands, upgrades, and 
> > downgrades. `version` seems to always be what you want, no?
> > 
> > ```
> > NORMAL COMMANDS
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.3
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > version = 2.3.4.0-1234
> > 
> > 
> > UPGRADE
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.3
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > original_stack = 2.3
> > target_stack = 2.5  
> > upgrade_direction = UPGRADE
> > version = 2.5.0.0-
> > 
> > 
> > DOWNGRADE
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.5
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > original_stack = 2.3
> > target_stack = 2.5  
> > upgrade_direction = DOWNGRADE
> > version = 2.3.4.0-1234
> > downgrade_from_version = 2.5.0.0-
> > ```
> 
> Mugdha Varadkar wrote:
> Thanks Jonathan Hurley. Actually this was done by taking reference from 
> similar code from 
> Ambari.(https://github.com/apache/ambari/commit/63f8d74f806aa55fcbb4e51b2c84a3fd0146c062#diff-a9a4cbc8111690ebc76b1e83491d2dba)
> 
> @Jayush Luniya, can we use version in NORMAL COMMANDS/UPGRADE/DOWNGRADE 
> scenarios?
> 
> Jonathan Hurley wrote:
> Actually, an even better option is this:
> 
> ```
> # determine which version to use for checking stack features; version is 
> the most accurate but
> # may not always be available (especially on first install of the cluster)
> version_for_stack_feature_checks = version if not None else 
> stack_version_unformatted
> ```
> 
> It's possible that on initial install, `version` isn't defined since we 
> haven't calculated it yet. So, the above defaults to the stack version in 
> that case. Now, version covers all 3 of your scenarios.

Updated in latest patch


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py,
> >  line 60
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.
> 
> Mugdha Varadkar wrote:
> @Jonathan Hurley, am directly using check_stack_feature here because even 
> if we pass version as ---> 2.3.4.0-1234, the code has format flag as true, 
> which calls format_stack_version() function which can be used for comparision.
> "compare_versions(stack_version, min_version, format = True) < 0"
> 
> Also I have added the check for "version" in this way "if 
> upgrade_direction == Direction.DOWNGRADE and version"
> 
> Jonathan Hurley wrote:
> My point was that the code was originally:
> `foo and check_stack_feature(feature, foo) `
> 
> You don't need the "foo and" part since the check_stack_feature is 
> checking it for None anyway.

Updated in latest patch


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py,
> >  line 61
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.
> 
> Mugdha Varadkar wrote:
> Same as above

Updated in latest patch


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py,
> >  line 58
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.
> 
> Mugdha Varadkar wrote:
> Same as above

Updated in latest patch


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py,
> >  line 57
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.
> 
> Mugdha Varadkar wrote:
> Same as above

Updated in latest patch


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py,
> >  line 47
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.
> 
> Mugdha Varadkar wrote:
> Same as above

Updated in latest patch


- Mugdha

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/
---

(Updated July 14, 2016, 2:23 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan Hurley, 
Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.


Changes
---

Handle Jonathan Hurley comments


Bugs: AMBARI-17687
https://issues.apache.org/jira/browse/AMBARI-17687


Repository: ambari


Description
---

Scenario:
1. install ranger with stack 2.4 and ambari-2.4.0
2. register the version of stack 2.5
3. perform the upgrade
at the step where it where zookeepers are upgraded , just downgraded.
4. downgrade was successful 
5. check ranger service status , it was down
try to restart the ranger but ranger startup is failing


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 0df8270 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 145c216 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 f24bd83 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 63a1100 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
 f47eee4 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
 7e0477f 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 fad4b9b 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 dfcad32 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 5941ca1 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 44a0446 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
4c122e9 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
bbd0fe4 

Diff: https://reviews.apache.org/r/49987/diff/


Testing
---

Test Ranger Downgrade from stack 2.5 to 2.4


Thanks,

Mugdha Varadkar

Re: Review Request 50031: Error for Pig view after upgrade to 2.4.0.0 [PersistenceException: Can't initialize data store for view PIG]

[Ashwin Rajeev]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50031/#review142237
---


Ship it!




Ship It!

- Ashwin Rajeev


On July 14, 2016, 1:09 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50031/
> ---
> 
> (Updated July 14, 2016, 1:09 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17710
> https://issues.apache.org/jira/browse/AMBARI-17710
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update SmokeTestEntity class name in viewentity table.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  67495b5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  54f0e57 
> 
> Diff: https://reviews.apache.org/r/50031/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 50031: Error for Pig view after upgrade to 2.4.0.0 [PersistenceException: Can't initialize data store for view PIG]

[Rohit Choudhary]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50031/#review142236
---


Ship it!




Ship It!

- Rohit Choudhary


On July 14, 2016, 1:09 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50031/
> ---
> 
> (Updated July 14, 2016, 1:09 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17710
> https://issues.apache.org/jira/browse/AMBARI-17710
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update SmokeTestEntity class name in viewentity table.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  67495b5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  54f0e57 
> 
> Diff: https://reviews.apache.org/r/50031/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 49826: AUTH_TO_LOCAL rules are not updated when adding services to a Blueprint-installed cluster

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49826/#review142235
---


Ship it!




Ship It!

- Nate Cole


On July 13, 2016, 4:33 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49826/
> ---
> 
> (Updated July 13, 2016, 4:33 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17629
> https://issues.apache.org/jira/browse/AMBARI-17629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding new services and components to a cluster that was initially 
> created via Blueprints (rather than via the Ambari UI), auth-to-local rules 
> that are expected to be created as indicated by the Kerberos descriptor are 
> not.
> 
> It occurs because the components being installed are in the `INIT` state 
> where the logic to determine whether to include the new auth-to-local rules 
> or not expects the components to be in either the `INSTALLED` or `STARTED` 
> states. This is due to logic added when resolving AMBARI-14232.
> 
> #Solution:
> Update logic building auth-to-local rules to occur when new services and 
> components are added to the cluster.  Logic assumes caller passes around 
> explict lists of currently installed services and component (including added 
> components), previously installed services and components, and a filter to 
> indicate which services and components are newly added. 
> 
> This data will help avoid generating regressions for AMBARI-14232 and 
> AMBARI-14506
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  872ec2d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  8170f17 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  cc48240 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
>  f70c546 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java
>  88f0e3c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  66ed68d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java
>  58919b9 
> 
> Diff: https://reviews.apache.org/r/49826/diff/
> 
> 
> Testing
> ---
> 
> Manually tested using various Blueprint and UI-based scenarios.
> 
> # Local unit tests: 
> 
> Tests run: 4536, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:12:50.002s
> [INFO] Finished at: Wed Jul 13 13:33:27 EDT 2016
> [INFO] Final Memory: 61M/1918M
> [INFO] 
> 
> 
> # Jenkins unit tests: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 50031: Error for Pig view after upgrade to 2.4.0.0 [PersistenceException: Can't initialize data store for view PIG]

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50031/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.


Bugs: AMBARI-17710
https://issues.apache.org/jira/browse/AMBARI-17710


Repository: ambari


Description
---

Update SmokeTestEntity class name in viewentity table.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 67495b5 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 54f0e57 

Diff: https://reviews.apache.org/r/50031/diff/


Testing
---

Manually tested.


Thanks,

Gaurav Nagar

Review Request 50030: AMBARI-17688 Ranger stack changes in Ambari to support secure Solr

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50030/
---

Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan Hurley, 
Jayush Luniya, Robert Levas, Srimanth Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17688
https://issues.apache.org/jira/browse/AMBARI-17688


Repository: ambari


Description
---

Below properties should be introduced to Ranger admin and plugins to support 
secure Solr. Also recommend for plugin in any of the below property chnages on 
Ranger Admin:

xasecure.audit.jaas.Client.loginModuleName=com.sun.security.auth.module.Krb5LoginModule
xasecure.audit.jaas.Client.loginModuleControlFlag=required
xasecure.audit.jaas.Client.option.useKeyTab=true
xasecure.audit.jaas.Client.option.storeKey=false
xasecure.audit.jaas.Client.option.serviceName=solr


Diffs
-

  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 145c216 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 63a1100 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 fad4b9b 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py
 529ac8c 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-admin-site.xml
 1b2b5e0 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
 341cff7 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/themes/theme_version_3.json
 3f50774 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 dfcad32 
  
ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
 d3f9143 
  ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
e570a5b7 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
2a2a3a3 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/ranger-atlas-audit.xml
 efeea5f 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/configuration/ranger-hbase-audit.xml
 d3f9143 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/ranger-hdfs-audit.xml
 019602a 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/ranger-hive-audit.xml
 d3f9143 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/KAFKA/configuration/ranger-kafka-audit.xml
 d3f9143 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/configuration/ranger-knox-audit.xml
 d3f9143 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/configuration/ranger-kms-audit.xml
 02b7565 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/ranger-yarn-audit.xml
 d3f9143 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
0e455e9 
  ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 86bf14d 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
67b00a1 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
9911e10 
  ambari-web/app/data/HDP2/site_properties.js 23fbf5e 
  ambari-web/app/models/stack_service.js c63df3d 

Diff: https://reviews.apache.org/r/50030/diff/


Testing
---

Tested Ranger and Ranger Plugins Installation with Logsearch solr on secure 
cluster.


Thanks,

Mugdha Varadkar

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Jonathan Hurley]

> On July 13, 2016, 1:29 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml, 
> > line 529
> > 
> >
> > Why is this needed exactly? Does Ranger not support a downgrade from 
> > HDP 2.5 to an earlier stack with the earlier configs?
> 
> Mugdha Varadkar wrote:
> upgrade to 2.5 needs config changes which are added inside 
> . During downgrade all the tasks under 
> 
>id="hdp_2_5_0_0_remove_audit_db_flag" />
>id="hdp_2_5_0_0_remove_audit_db_admin_properties" />
>id="hdp_2_5_0_0_remove_audit_db_ranger_admin_site" />
>/>
>  
> 
> gets executed. This leads to below failure:
> 
> Fail("Configuration parameter '" + self.name + "' was not found in 
> configurations dictionary!")
> resource_management.core.exceptions.Fail: Configuration parameter 
> 'audit_db_name' was not found in configurations dictionary!
> 
> To avoid that  tag is used. 
> Refer: 
> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml#L606

Yes, it's something that is needed. It's not very intuitive, but  
runs on downgrade as well unless you override it with a blank  
element.

When we define out XSD for upgrades in the next version, we'll need to enforce 
that  exists even if it's the same work as  to 
avoid this weird side effects.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/#review142084
---


On July 13, 2016, 8:25 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49987/
> ---
> 
> (Updated July 13, 2016, 8:25 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17687
> https://issues.apache.org/jira/browse/AMBARI-17687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Scenario:
> 1. install ranger with stack 2.4 and ambari-2.4.0
> 2. register the version of stack 2.5
> 3. perform the upgrade
> at the step where it where zookeepers are upgraded , just downgraded.
> 4. downgrade was successful 
> 5. check ranger service status , it was down
> try to restart the ranger but ranger startup is failing
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  8445214 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  6d304d4 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  f24bd83 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  571eebd 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  f47eee4 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7e0477f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  fad4b9b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dfcad32 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  2afb187 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  44a0446 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 4c122e9 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> bbd0fe4 
> 
> Diff: https://reviews.apache.org/r/49987/diff/
> 
> 
> Testing
> ---
> 
> Test Ranger Downgrade from stack 2.5 to 2.4
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Jonathan Hurley]

> On July 13, 2016, 10:33 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py,
> >  line 60
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.
> 
> Mugdha Varadkar wrote:
> @Jonathan Hurley, am directly using check_stack_feature here because even 
> if we pass version as ---> 2.3.4.0-1234, the code has format flag as true, 
> which calls format_stack_version() function which can be used for comparision.
> "compare_versions(stack_version, min_version, format = True) < 0"
> 
> Also I have added the check for "version" in this way "if 
> upgrade_direction == Direction.DOWNGRADE and version"

My point was that the code was originally:
`foo and check_stack_feature(feature, foo) `

You don't need the "foo and" part since the check_stack_feature is checking it 
for None anyway.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/#review142063
---


On July 13, 2016, 8:25 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49987/
> ---
> 
> (Updated July 13, 2016, 8:25 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17687
> https://issues.apache.org/jira/browse/AMBARI-17687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Scenario:
> 1. install ranger with stack 2.4 and ambari-2.4.0
> 2. register the version of stack 2.5
> 3. perform the upgrade
> at the step where it where zookeepers are upgraded , just downgraded.
> 4. downgrade was successful 
> 5. check ranger service status , it was down
> try to restart the ranger but ranger startup is failing
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  8445214 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  6d304d4 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  f24bd83 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  571eebd 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  f47eee4 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7e0477f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  fad4b9b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dfcad32 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  2afb187 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  44a0446 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 4c122e9 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> bbd0fe4 
> 
> Diff: https://reviews.apache.org/r/49987/diff/
> 
> 
> Testing
> ---
> 
> Test Ranger Downgrade from stack 2.5 to 2.4
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Jonathan Hurley]

> On July 13, 2016, 10:33 a.m., Jonathan Hurley wrote:
> > I'm curious if you can make this simply by only using the 
> > `commandParams/version` property when calculating what to do. Here's an 
> > example of the various values during normal commands, upgrades, and 
> > downgrades. `version` seems to always be what you want, no?
> > 
> > ```
> > NORMAL COMMANDS
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.3
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > version = 2.3.4.0-1234
> > 
> > 
> > UPGRADE
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.3
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > original_stack = 2.3
> > target_stack = 2.5  
> > upgrade_direction = UPGRADE
> > version = 2.5.0.0-
> > 
> > 
> > DOWNGRADE
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.5
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > original_stack = 2.3
> > target_stack = 2.5  
> > upgrade_direction = DOWNGRADE
> > version = 2.3.4.0-1234
> > downgrade_from_version = 2.5.0.0-
> > ```
> 
> Mugdha Varadkar wrote:
> Thanks Jonathan Hurley. Actually this was done by taking reference from 
> similar code from 
> Ambari.(https://github.com/apache/ambari/commit/63f8d74f806aa55fcbb4e51b2c84a3fd0146c062#diff-a9a4cbc8111690ebc76b1e83491d2dba)
> 
> @Jayush Luniya, can we use version in NORMAL COMMANDS/UPGRADE/DOWNGRADE 
> scenarios?

Actually, an even better option is this:

```
# determine which version to use for checking stack features; version is the 
most accurate but
# may not always be available (especially on first install of the cluster)
version_for_stack_feature_checks = version if not None else 
stack_version_unformatted
```

It's possible that on initial install, `version` isn't defined since we haven't 
calculated it yet. So, the above defaults to the stack version in that case. 
Now, version covers all 3 of your scenarios.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/#review142063
---


On July 13, 2016, 8:25 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49987/
> ---
> 
> (Updated July 13, 2016, 8:25 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17687
> https://issues.apache.org/jira/browse/AMBARI-17687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Scenario:
> 1. install ranger with stack 2.4 and ambari-2.4.0
> 2. register the version of stack 2.5
> 3. perform the upgrade
> at the step where it where zookeepers are upgraded , just downgraded.
> 4. downgrade was successful 
> 5. check ranger service status , it was down
> try to restart the ranger but ranger startup is failing
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  8445214 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  6d304d4 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  f24bd83 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  571eebd 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  f47eee4 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7e0477f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  fad4b9b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dfcad32 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  2afb187 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  44a0446 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 4c122e9 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> bbd0fe4 
> 
> Diff: https://reviews.apache.org/r/49987/diff/
> 
> 
> Testing
> ---
> 
> Test Ranger Downgrade from stack 2.5 to 2.4
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 49991: Rerunning View data migration commands throwing exception

[Ashwin Rajeev]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49991/#review142228
---


Ship it!




Ship It!

- Ashwin Rajeev


On July 13, 2016, 4:37 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49991/
> ---
> 
> (Updated July 13, 2016, 4:37 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17691
> https://issues.apache.org/jira/browse/AMBARI-17691
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added persistence-unit for ambari-view-migration
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationContextImpl.java
>  909b3a1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationUtility.java
>  15318c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreModule.java
>  a995868 
>   ambari-server/src/main/resources/META-INF/persistence.xml d44f484 
> 
> Diff: https://reviews.apache.org/r/49991/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 50007: Hive Restart Failed During RU Due To Missing SQL JAR

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50007/#review142223
---


Ship it!




Ship It!

- Nate Cole


On July 13, 2016, 6:05 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50007/
> ---
> 
> (Updated July 13, 2016, 6:05 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-17697
> https://issues.apache.org/jira/browse/AMBARI-17697
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> # Deploy HDP 2.3 with Hive
> # Register and install HDP 2.5.0.0-945
> # Perform RU to 2.5.0.0-945 version
> # After Hive Upgrades, Downgrade
> 
> {code}
> resource_management.core.exceptions.Fail: Execution of 'cp 
> --remove-destination /var/lib/ambari-agent/tmp/mysql-connector-java.jar 
> /usr/hdp/2.3.0.0-2557/hive2/lib/mysql-connector-java.jar' returned 1. cp: 
> cannot create regular file 
> '/usr/hdp/2.3.0.0-2557/hive2/lib/mysql-connector-java.jar': No such file or 
> directory
> {code}
> 
> I believe the problem is this line of code:
> {code}
> if check_stack_feature(StackFeature.HIVE_SERVER_INTERACTIVE, 
> stack_version_unformatted):
> {code}
> 
> On a downgrade from 2.5 to 2.3 {{version}} and {{current_version}} are 
> pointing to 2.3, but stack_version_unformatted is still 2.5. Because of this, 
> we initialize the hive2 variables which leads us to think that hive2 is the 
> target of copying the JDBC:
> 
> {code}
>   if name == 'metastore' or name == 'hiveserver2':
> if params.hive_jdbc_target is not None and not 
> os.path.exists(params.hive_jdbc_target):
>   jdbc_connector(params.hive_jdbc_target, params.hive_previous_jdbc_jar)
> if params.hive2_jdbc_target is not None and not 
> os.path.exists(params.hive2_jdbc_target):
>   jdbc_connector(params.hive2_jdbc_target, params.hive2_previous_jdbc_jar)
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  571eebd 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py
>  b148849 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hcat_client.py 54d3e84 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py 77d6bdb 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py 
> 4909c77 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 6c46267 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 
> a8b5421 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py 
> f71bba9 
>   ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 
> 044598d 
> 
> Diff: https://reviews.apache.org/r/50007/diff/
> 
> 
> Testing
> ---
> 
> OK
> --
> Total run:1018
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 49145: Ability to add "javax.jdo.option.ConnectionPassword" to hive clients from ambari

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49145/
---

(Updated July 14, 2016, 10:57 a.m.)


Review request for Ambari, Robert Levas and Vitalyi Brodetskyi.


Bugs: AMBARI-17223
https://issues.apache.org/jira/browse/AMBARI-17223


Repository: ambari


Description
---

Problem is that they need to access the "javax.jdo.option.ConnectionPassword"
password but Ambari is pushing only to /etc/hive/conf.server/hive-site.xml. I
have instructed and recommended various workarounds and even suggest modifying
their app. Even having a static hive-site.xm with the information they did not
accept and asked to open an enhancement request. The enhancement is to allow
adding "javax.jdo.option.ConnectionPassword" to hive clients from ambari and
have a separate hive-site.xml path for clients.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 bf2ff38 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 872ec2d 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClientConfigResourceProvider.java
 9a86827 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
46a3f3e 
  ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 
c30f28f 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ValueAttributesInfo.java
 7cccedf 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
 61437d5 
  ambari-server/src/main/resources/configuration-schema.xsd 0589d7e 
  
ambari-server/src/main/resources/stacks/HDP/2.1/services/HIVE/configuration/hive-site.xml
 229f7e1 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
 27d9e4b 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
 0ff143c 
  
ambari-server/src/test/resources/stacks/HDP/2.0.5/services/HIVE/configuration/hive-site.xml
 5a3cb6c 

Diff: https://reviews.apache.org/r/49145/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49991: Rerunning View data migration commands throwing exception

[DIPAYAN BHOWMICK]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49991/#review142221
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On July 13, 2016, 4:37 p.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49991/
> ---
> 
> (Updated July 13, 2016, 4:37 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, Rohit Choudhary, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-17691
> https://issues.apache.org/jira/browse/AMBARI-17691
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added persistence-unit for ambari-view-migration
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationContextImpl.java
>  909b3a1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationUtility.java
>  15318c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreModule.java
>  a995868 
>   ambari-server/src/main/resources/META-INF/persistence.xml d44f484 
> 
> Diff: https://reviews.apache.org/r/49991/diff/
> 
> 
> Testing
> ---
> 
> Manually Tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Re: Review Request 49755: Capacity Scheduler View - Fetching current RM configuration of queues and preemption implementation

[Akhil PB]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49755/
---

(Updated July 14, 2016, 10:28 a.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and Sreenath 
Somarajapuram.


Changes
---

Updated with review comments


Bugs: AMBARI-17602
https://issues.apache.org/jira/browse/AMBARI-17602


Repository: ambari


Description
---

Fetching current rm configuration of queues.
Updating queue states based on states in configuration.
Preemption implementation.
Bug fixes.


Diffs (updated)
-

  
contrib/views/capacity-scheduler/src/main/java/org/apache/ambari/view/capacityscheduler/ConfigurationService.java
 03520fe 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/adapters.js 
068c241 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/capacityInput.js
 bf74af6 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/displayNodeLabels.js
 9b31e57 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/editLabelCapacity.js
 7bb3f82 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueSummary.js
 bfcd4b6 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/capsched.js
 545d348 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
 f42d2e4 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
 683bc6f 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/models/queue.js 
e91c90f 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/router.js 49363d4 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/serializers.js 
c3976b4 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/store.js 4a9c6b0 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
 e849ad4 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/templates.js 
6e310ad 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched.hbs
 b75237f 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/labelCapacity.hbs
 e659f5c 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/preemption.hbs
 PRE-CREATION 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueCapacity.hbs
 b4891cd 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/queueResources.hbs
 bb82025 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf.hbs
 b1a8708 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/queuesconf/editqueue.hbs
 2848fed 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/trace.hbs
 PRE-CREATION 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/editLabelCapacity.hbs
 9e25cc6 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/editQueueCapacity.hbs
 c430a44 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/labelCapacityBar.hbs
 96cbe99 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueMapping.hbs
 bd49c50 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueSummary.hbs
 ef5663d 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/views/editqueue.js 
a5d0ef6 

Diff: https://reviews.apache.org/r/49755/diff/


Testing
---

Manual Unit Testing


Thanks,

Akhil PB

Review Request 50025: Atlas metadata server fails to install in non-root

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50025/
---

Review request for Ambari and Myroslav Papirkovskyy.


Bugs: AMBARI-17707
https://issues.apache.org/jira/browse/AMBARI-17707


Repository: ambari


Description
---

Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
 line 173, in 
MetadataServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
 line 46, in install
shutil.rmtree(expanded_atlas)
  File "/usr/lib64/python2.6/shutil.py", line 204, in rmtree
onerror(os.listdir, path, sys.exc_info())
  File "/usr/lib64/python2.6/shutil.py", line 202, in rmtree
names = os.listdir(path)
OSError: [Errno 13] Permission denied: 
'/usr/hdp/current/atlas-server/server/webapp/atlas'


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
 062e8ef 

Diff: https://reviews.apache.org/r/50025/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Mugdha Varadkar]

> On July 13, 2016, 5:29 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml, 
> > line 529
> > 
> >
> > Why is this needed exactly? Does Ranger not support a downgrade from 
> > HDP 2.5 to an earlier stack with the earlier configs?

upgrade to 2.5 needs config changes which are added inside 
. During downgrade all the tasks under 

  
  
  
  
 

gets executed. This leads to below failure:

Fail("Configuration parameter '" + self.name + "' was not found in 
configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 
'audit_db_name' was not found in configurations dictionary!

To avoid that  tag is used. 
Refer: 
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml#L606


> On July 13, 2016, 5:29 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py,
> >  line 214
> > 
> >
> > This is not needed since setting it to the same value under more 
> > conditions.

stack_supports_ranger_kerberos feature have min support version as 2.5

In normal case with 2.5 below code logic will be
stack_version_unformatted = config['hostLevelParams']['stack_version']  > 
2.5
stack_version_formatted = format_stack_version(stack_version_unformatted) > 
2.5.0.0
stack_supports_ranger_kerberos = stack_version_formatted and 
check_stack_feature(StackFeature.RANGER_KERBEROS_SUPPORT, 
stack_version_formatted)  ---> True

In Downgrade case from 2.5 to lower version
stack_version still points to 2.5, because of which 
stack_supports_ranger_kerberos flag becomes true which should be false.
That is the reason "if upgrade_direction == Direction.DOWNGRADE and version:" 
check was added. Using version = default("/commandParams/version", None) which 
gives downgrade version.


> On July 13, 2016, 5:29 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py,
> >  line 76
> > 
> >
> > No need to change stack_supports_ranger_kerberos

It is needed to be changed. Please see above comment


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/#review142084
---


On July 13, 2016, 12:25 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49987/
> ---
> 
> (Updated July 13, 2016, 12:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17687
> https://issues.apache.org/jira/browse/AMBARI-17687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Scenario:
> 1. install ranger with stack 2.4 and ambari-2.4.0
> 2. register the version of stack 2.5
> 3. perform the upgrade
> at the step where it where zookeepers are upgraded , just downgraded.
> 4. downgrade was successful 
> 5. check ranger service status , it was down
> try to restart the ranger but ranger startup is failing
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  8445214 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  6d304d4 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  f24bd83 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  571eebd 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py
>  f47eee4 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
>  7e0477f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  fad4b9b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dfcad32 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  2afb187 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
>  44a0446 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 4c122e9 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> bbd0fe4 
> 

Re: Review Request 49987: AMBARI-17687 [RU/Downgrade] ranger service goes down in between of RU downgrade

[Mugdha Varadkar]

> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > I'm curious if you can make this simply by only using the 
> > `commandParams/version` property when calculating what to do. Here's an 
> > example of the various values during normal commands, upgrades, and 
> > downgrades. `version` seems to always be what you want, no?
> > 
> > ```
> > NORMAL COMMANDS
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.3
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > version = 2.3.4.0-1234
> > 
> > 
> > UPGRADE
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.3
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > original_stack = 2.3
> > target_stack = 2.5  
> > upgrade_direction = UPGRADE
> > version = 2.5.0.0-
> > 
> > 
> > DOWNGRADE
> >   hostLevelParams
> > stack_name = HDP
> > stack_version = 2.5
> > current_version = 2.3.4.0-1234
> > 
> >   commandParams
> > original_stack = 2.3
> > target_stack = 2.5  
> > upgrade_direction = DOWNGRADE
> > version = 2.3.4.0-1234
> > downgrade_from_version = 2.5.0.0-
> > ```

Thanks Jonathan Hurley. Actually this was done by taking reference from similar 
code from 
Ambari.(https://github.com/apache/ambari/commit/63f8d74f806aa55fcbb4e51b2c84a3fd0146c062#diff-a9a4cbc8111690ebc76b1e83491d2dba)

@Jayush Luniya, can we use version in NORMAL COMMANDS/UPGRADE/DOWNGRADE 
scenarios?


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py,
> >  line 60
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.

@Jonathan Hurley, am directly using check_stack_feature here because even if we 
pass version as ---> 2.3.4.0-1234, the code has format flag as true, which 
calls format_stack_version() function which can be used for comparision.
"compare_versions(stack_version, min_version, format = True) < 0"

Also I have added the check for "version" in this way "if upgrade_direction == 
Direction.DOWNGRADE and version"


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py,
> >  line 61
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.

Same as above


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/params.py,
> >  line 58
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.

Same as above


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py,
> >  line 57
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.

Same as above


> On July 13, 2016, 2:33 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py,
> >  line 47
> > 
> >
> > No need for `stack_version_formatted and` here ... just call 
> > `check_stack_feature` directly.

Same as above


- Mugdha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49987/#review142063
---


On July 13, 2016, 12:25 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49987/
> ---
> 
> (Updated July 13, 2016, 12:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Jonathan 
> Hurley, Jayush Luniya, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17687
> https://issues.apache.org/jira/browse/AMBARI-17687
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Scenario:
> 1. install ranger with stack 2.4 and ambari-2.4.0
> 2. register the version of stack 2.5
> 3. perform the upgrade
> at the step where it where zookeepers are upgraded , just downgraded.
> 4. downgrade was successful 
> 5. check ranger service status , it was down
> try to restart the ranger but ranger startup is failing
> 
> 
> Diffs
> -
> 
>   
> 

Re: Review Request 49826: AUTH_TO_LOCAL rules are not updated when adding services to a Blueprint-installed cluster

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49826/#review142210
---




ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 (line 313)


In ```State.isValidStateTransition``` and 
```ServiceComponentHostImpl.daemonStateMachineFactory``` the DISABLED state can 
be reached from INSTALLED state.

Should we consider here DISABLED state as well? Or not as on the state 
transition to DISABLE the KerberosHelperImpl would never be called.


- Sebastian Toader


On July 13, 2016, 10:33 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49826/
> ---
> 
> (Updated July 13, 2016, 10:33 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sebastian Toader.
> 
> 
> Bugs: AMBARI-17629
> https://issues.apache.org/jira/browse/AMBARI-17629
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When adding new services and components to a cluster that was initially 
> created via Blueprints (rather than via the Ambari UI), auth-to-local rules 
> that are expected to be created as indicated by the Kerberos descriptor are 
> not.
> 
> It occurs because the components being installed are in the `INIT` state 
> where the logic to determine whether to include the new auth-to-local rules 
> or not expects the components to be in either the `INSTALLED` or `STARTED` 
> states. This is due to logic added when resolving AMBARI-14232.
> 
> #Solution:
> Update logic building auth-to-local rules to occur when new services and 
> components are added to the cluster.  Logic assumes caller passes around 
> explict lists of currently installed services and component (including added 
> components), previously installed services and components, and a filter to 
> indicate which services and components are newly added. 
> 
> This data will help avoid generating regressions for AMBARI-14232 and 
> AMBARI-14506
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  872ec2d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  8170f17 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
>  cc48240 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
>  f70c546 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java
>  88f0e3c 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  66ed68d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java
>  58919b9 
> 
> Diff: https://reviews.apache.org/r/49826/diff/
> 
> 
> Testing
> ---
> 
> Manually tested using various Blueprint and UI-based scenarios.
> 
> # Local unit tests: 
> 
> Tests run: 4536, Failures: 0, Errors: 0, Skipped: 34
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:12:50.002s
> [INFO] Finished at: Wed Jul 13 13:33:27 EDT 2016
> [INFO] Final Memory: 61M/1918M
> [INFO] 
> 
> 
> # Jenkins unit tests: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>