date:20160705


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48855/#review140924
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 5, 2016, 7:53 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48855/
> ---
> 
> (Updated July 5, 2016, 7:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Richard Zang, 
> Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-12697
> https://issues.apache.org/jira/browse/AMBARI-12697
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari UI shows operation history in the ops prompt. All upgrade and 
> downgrade attempts are not shown on the UI once user finalizes it and close 
> the upgrade/downgrade UI.
> 
> The information is in the database already. This JIRA is to propose a way to 
> add UI elements to display a list of upgrades/downgrades associated with the 
> cluster, including both express and rolling ones.
> 
> UI BEHAVIOR:
> 
> 1. The new tab is the "Upgrade History" tabe shown on the Version and Stack 
> page. Users will see three tabs now, the Version tab, the Stack tab and the 
> Upgrade History tab. 
> 2. The Upgrade History tab shows a filter widget and a table showing the list 
> of upgrades/downgrades based on the filter select. Default filter select is 
> 'All', so all upgrade and downgrade records from the Ambari database 
> (upgrades tables) are shown on the UI. 
> 3. User clicks on a given record to open the detail view popup dialog that 
> resembles the upgrade wizard UI. on the detail view popup dialog, user can 
> drill up and down on upgrade steps for details.
> 
> UI SCREENSHOT:
> 1. Upgrade_History view: 
> https://issues.apache.org/jira/secure/attachment/12811384/Upgrade_History_view.png
> 2. Upgrade details view: 
> https://issues.apache.org/jira/secure/attachment/12811385/Upgrade_Details_view.png
> 
> 
> Diffs
> -
> 
>   ambari-web/app/assets/test/tests.js 005c97b 
>   ambari-web/app/config.js 06faf27 
>   ambari-web/app/controllers.js 2537f62 
>   ambari-web/app/controllers/global/cluster_controller.js 0d9d41d 
>   ambari-web/app/controllers/main/admin/stack_upgrade_history_controller.js 
> PRE-CREATION 
>   ambari-web/app/mappers.js 96193bc 
>   ambari-web/app/mappers/stack_upgrade_history_mapper.js PRE-CREATION 
>   ambari-web/app/messages.js dcf0426 
>   ambari-web/app/models.js b877255 
>   ambari-web/app/models/finished_upgrade_entity.js PRE-CREATION 
>   ambari-web/app/models/stack_version/stack_upgrade_history.js PRE-CREATION 
>   ambari-web/app/routes/main.js 8d33a68 
>   ambari-web/app/styles/stack_versions.less 604b655 
>   ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history.hbs 
> PRE-CREATION 
>   
> ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history_details.hbs 
> PRE-CREATION 
>   ambari-web/app/views.js ab79b98 
>   ambari-web/app/views/main/admin/stack_upgrade/menu_view.js 1e84f1c 
>   
> ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_details_view.js 
> PRE-CREATION 
>   ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_view.js 
> PRE-CREATION 
>   
> ambari-web/test/controllers/main/admin/stack_upgrade_history_controller_test.js
>  PRE-CREATION 
>   ambari-web/test/mappers/stack_upgrade_history_mapper_test.js PRE-CREATION 
>   ambari-web/test/models/finished_upgrade_entity_test.js PRE-CREATION 
>   
> ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_details_view_test.js
>  PRE-CREATION 
>   ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_view_test.js 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48855/diff/
> 
> 
> Testing
> ---
> 
> 1. patch a trunk cluster with the code change, ran express upgrades, express 
> downgrades, rolling upgrades and rolling downgrades. Check upgrade and 
> downgrade history displayed on the UI
> 2. new unit tests
> 3. existing unit tests
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 49677: YARN Summary Panel shows ResourceManager heap incorrectly

2016-07-05 Thread Zhe (Joe) Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49677/#review140923
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On July 6, 2016, 1:13 a.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49677/
> ---
> 
> (Updated July 6, 2016, 1:13 a.m.)
> 
> 
> Review request for Ambari, Zhe (Joe) Wang and Xi Wang.
> 
> 
> Bugs: AMBARI-17574
> https://issues.apache.org/jira/browse/AMBARI-17574
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Use correct template.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/messages.js dcf0426 
>   ambari-web/app/views/main/service/services/yarn.js 7ccbfcf 
> 
> Diff: https://reviews.apache.org/r/49677/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> All unit tests passed.
>   28941 tests complete (27 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Review Request 49676: Add atlas-application config sections to all services that run Atlas hook, e.g., Hive, Falcon, Storm, Sqoop


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49676/
---

Review request for Ambari, Robert Levas, Sumit Mohanty, and Swapan Shridhar.


Bugs: AMBARI-17573
https://issues.apache.org/jira/browse/AMBARI-17573


Repository: ambari


Description
---

Currently, Atlas hooks that run in Hive, Falconm Storm, Sqoop processes 
reference atlas-application.properties file from Atlas server config location - 
/etc/atlas/conf/atlas-application.properties.
Not all properties in /etc/atlas/conf/atlas-application.properties are required 
in hooks and some of these properties are sensitive enough not to expose them 
to hooks/clients.

To address this concern:
1. atlas-application.properties should be added as a config section in each of 
the host component's that run Atlas hook - Hive, Storm, Falcon, Sqoop
2. These new config sections will only include properties that are required to 
the respective hooks
3. During initial deployment, Ambari will initialize these properties with 
values in Atlas server configuration.
For each one of those services, create a config type called $
{service}-atlas-application.properties that will be saved to /etc/${service}
/conf/application.properties

These are the default values,

Falcon
atlas.hook.falcon.synchronous=false
atlas.hook.falcon.numRetries=3
atlas.hook.falcon.minThreads=5
atlas.hook.falcon.maxThreads=5
atlas.hook.falcon.keepAliveTime=10
atlas.hook.falcon.queueSize

Storm
atlas.hook.storm.numRetries=3

Hive
atlas.hook.hive.synchronous=false
atlas.hook.hive.numRetries=3
atlas.hook.hive.minThreads=5
atlas.hook.hive.maxThreads=5
atlas.hook.hive.keepAliveTime=10
atlas.hook.hive.queueSize=1

Common for all hooks
atlas.kafka.zookeeper.connect=
atlas.kafka.bootstrap.servers=
atlas.kafka.zookeeper.session.timeout.ms=400
atlas.kafka.zookeeper.connection.timeout.ms=200
atlas.kafka.zookeeper.sync.time.ms=20
atlas.kafka.hook.group.id=atlas
atlas.notification.create.topics=true
atlas.notification.replicas=1
atlas.notification.topics=ATLAS_HOOK,ATLAS_ENTITIES
atlas.notification.kafka.service.principal=kafka/_h...@example.com
atlas.notification.kafka.keytab.location=/etc/security/keytabs/kafka.service.keytab
atlas.jaas.KafkaClient.loginModuleName = 
com.sun.security.auth.module.Krb5LoginModule
atlas.jaas.KafkaClient.loginModuleControlFlag = required
atlas.jaas.KafkaClient.option.useKeyTab = true
atlas.jaas.KafkaClient.option.storeKey = true
atlas.jaas.KafkaClient.option.serviceName = kafka
atlas.jaas.KafkaClient.option.keyTab = 
/etc/security/keytabs/atlas.service.keytab
atlas.jaas.KafkaClient.option.principal = atlas/_h...@example.com
atlas.cluster.name=


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 1437251 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-atlas-application.properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/metainfo.xml 
602144b 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 c2f1f53 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 fc9d8b9 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/setup_atlas_falcon.py
 1dce515 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
273133a 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
 839ab04 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 ea2af62 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 17f7380 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/setup_atlas_hive.py
 d1bd8ea 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
 816b6af 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
 b2a6802 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/setup_atlas_sqoop.py
 76c1cda 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
 bac836c 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 fac6331 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/setup_atlas_storm.py
 76ca1d3 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
 2d50767

Review Request 49677: YARN Summary Panel shows ResourceManager heap incorrectly

2016-07-05 Thread Richard Zang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49677/
---

Review request for Ambari, Zhe (Joe) Wang and Xi Wang.


Bugs: AMBARI-17574
https://issues.apache.org/jira/browse/AMBARI-17574


Repository: ambari


Description
---

Use correct template.


Diffs
-

  ambari-web/app/messages.js dcf0426 
  ambari-web/app/views/main/service/services/yarn.js 7ccbfcf 

Diff: https://reviews.apache.org/r/49677/diff/


Testing
---

Manually tested on live cluster.
All unit tests passed.
  28941 tests complete (27 seconds)
  154 tests pending


Thanks,

Richard Zang

Re: Review Request 49664: HDFS Summary Panel is not showing the NameNode heap size correctly

2016-07-05 Thread Zhe (Joe) Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49664/#review140893
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On July 5, 2016, 11:10 p.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49664/
> ---
> 
> (Updated July 5, 2016, 11:10 p.m.)
> 
> 
> Review request for Ambari, Zhe (Joe) Wang and Xi Wang.
> 
> 
> Bugs: AMBARI-17572
> https://issues.apache.org/jira/browse/AMBARI-17572
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix parameter name.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/views/main/service/services/hdfs.js 4bc2cce 
> 
> Diff: https://reviews.apache.org/r/49664/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster. All unit tests passed.
> 
>   28941 tests complete (30 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Review Request 49664: HDFS Summary Panel is not showing the NameNode heap size correctly

2016-07-05 Thread Richard Zang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49664/
---

Review request for Ambari, Zhe (Joe) Wang and Xi Wang.


Bugs: AMBARI-17572
https://issues.apache.org/jira/browse/AMBARI-17572


Repository: ambari


Description
---

Fix parameter name.


Diffs
-

  ambari-web/app/views/main/service/services/hdfs.js 4bc2cce 

Diff: https://reviews.apache.org/r/49664/diff/


Testing
---

Manually tested on live cluster. All unit tests passed.

  28941 tests complete (30 seconds)
  154 tests pending


Thanks,

Richard Zang

Review Request 49665: authorizer.class.name not being set on secure kafka clusters

2016-07-05 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49665/
---

Review request for Ambari, Alejandro Fernandez, Srimanth Gunturi, Tim Thorpe, 
and Vitalyi Brodetskyi.


Bugs: AMBARI-17479
https://issues.apache.org/jira/browse/AMBARI-17479


Repository: ambari


Description
---

The `kafka-broker/authorizer.class.name` property is not being set properly 
when Kerberos is enabled.

The following logic should be followed:
```
if Kerberos is enabled
  if ranger-kafka-plugin-properties/ranger-kafka-plugin-enabled == yes
set authorizer.class.name to 
"org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer"
  else
set authorizer.class.name to "kafka.security.auth.SimpleAclAuthorizer"
else
  remove authorizer.class.name
```

This should be updated in the stack advisor code. 

While at it, configurations from Kafka's `kerberos.json` file should be moved 
to the stack advisor to help ensure properties are set in the the same place to 
help with code maintenance and consistency.


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
06f7cfe 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
879008b 
  ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 2944f6f 

Diff: https://reviews.apache.org/r/49665/diff/


Testing
---

Manually tested

#Jenkins test results: PENDING


Thanks,

Robert Levas

Review Request 49662: Show gpadmin as hawq service account user on Ambari

2016-07-05 Thread bhuvnesh chaudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49662/
---

Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.


Bugs: AMBARI-17571
https://issues.apache.org/jira/browse/AMBARI-17571


Repository: ambari


Description
---

Show gpadmin as hawq service account user on Ambari


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
 cfab63f 

Diff: https://reviews.apache.org/r/49662/diff/


Testing
---

manual.


Thanks,

bhuvnesh chaudhary

Review Request 49659: Update config update message for Activate HAWQ Standby wizard

2016-07-05 Thread bhuvnesh chaudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49659/
---

Review request for Ambari, Alexander Denissov, Lav Jain, and Matt.


Bugs: AMBARI-17569
https://issues.apache.org/jira/browse/AMBARI-17569


Repository: ambari


Description
---

Update config update message for Activate HAWQ Standby wizard


Diffs
-

  
ambari-web/app/controllers/main/admin/highAvailability/hawq/activateStandby/step3_controller.js
 326b704 
  ambari-web/app/messages.js dcf0426 

Diff: https://reviews.apache.org/r/49659/diff/


Testing
---

ok


Thanks,

bhuvnesh chaudhary

Re: Review Request 49249: AMBARI-17445: Storm metrics sink: expand metrics name to contain additional informations

2016-07-05 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49249/#review140873
---


Ship it!




Ship It!

- Aravindan Vijayan


On July 4, 2016, 12:23 p.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49249/
> ---
> 
> (Updated July 4, 2016, 12:23 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Prajwal Rao, Sriharsha 
> Chintalapani, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17445
> https://issues.apache.org/jira/browse/AMBARI-17445
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With AMBARI-16946 we expanded metric name to have component name, task id, 
> and key when type of metric value is Map. Topology name was placed to appId.
> 
> While talking with Prajwal Rao and sriharsha chintalapani, we found that we 
> should move out all necessary informations like topology name, worker host, 
> worker port to metric name so that they can be templatized.
> 
> This also supercedes AMBARI-17249 since all metrics will include worker host 
> and worker port to metric name.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  9266e5f 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  4ea7396 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  d04bc15 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  3e9ed34 
> 
> Diff: https://reviews.apache.org/r/49249/diff/
> 
> 
> Testing
> ---
> 
> Tested locally. Attached screenshot to origin issue.
> https://issues.apache.org/jira/secure/attachment/12813582/AMBARI-17445-changes.png
> 
> 
> Thanks,
> 
> Jungtaek Lim
> 
>

Re: Review Request 49645: With wire-encryption enabled, quick links in ambari UI points to incorrect atlas web url

2016-07-05 Thread Zhe (Joe) Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49645/#review140870
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On July 5, 2016, 6:42 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49645/
> ---
> 
> (Updated July 5, 2016, 6:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Zhe (Joe) Wang, and Yusaku 
> Sako.
> 
> 
> Bugs: AMBARI-17565
> https://issues.apache.org/jira/browse/AMBARI-17565
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With wire-encryption enabled, quick links in ambari UI points to incorrect 
> atlas web url.
> 
> Currently ambari points to http://:21000/.  where as 
> wire-encryption enabled atlas runs on port 21443, so the correct url is 
> https://:21443/.
> 
> 
> The issue occured becaue the property deciding Atlas to be configured for 
> HTTPS or not was renamed from metadata.enableTLS to atlas.enableTLS
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml 
> 88c1915 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/quicklinks/quicklinks.json
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49645/diff/
> 
> 
> Testing
> ---
> 
> Tested manually on a cluster that patch fixes the issue.
> The patch only includes metadata changes in the Atlas stack definitions and 
> so does not require unit tests.
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 48855: AMBARI-12697 Rolling upgrade: Ambari UI should be able to display rolling upgrade history

2016-07-05 Thread Di Li



> On July 5, 2016, 6:01 p.m., Alejandro Fernandez wrote:
> > ambari-web/app/controllers/global/cluster_controller.js, line 309
> > 
> >
> > If an upgrade is paused, I believe the state will be ABORTED. Try 
> > checking that scenario.

Hello Alejandro,

Thanks for the review. I modified the code a little to check the ABORTED state.

please review.


Di.


- Di


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48855/#review140829
---


On July 5, 2016, 7:53 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48855/
> ---
> 
> (Updated July 5, 2016, 7:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Richard Zang, 
> Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-12697
> https://issues.apache.org/jira/browse/AMBARI-12697
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari UI shows operation history in the ops prompt. All upgrade and 
> downgrade attempts are not shown on the UI once user finalizes it and close 
> the upgrade/downgrade UI.
> 
> The information is in the database already. This JIRA is to propose a way to 
> add UI elements to display a list of upgrades/downgrades associated with the 
> cluster, including both express and rolling ones.
> 
> UI BEHAVIOR:
> 
> 1. The new tab is the "Upgrade History" tabe shown on the Version and Stack 
> page. Users will see three tabs now, the Version tab, the Stack tab and the 
> Upgrade History tab. 
> 2. The Upgrade History tab shows a filter widget and a table showing the list 
> of upgrades/downgrades based on the filter select. Default filter select is 
> 'All', so all upgrade and downgrade records from the Ambari database 
> (upgrades tables) are shown on the UI. 
> 3. User clicks on a given record to open the detail view popup dialog that 
> resembles the upgrade wizard UI. on the detail view popup dialog, user can 
> drill up and down on upgrade steps for details.
> 
> UI SCREENSHOT:
> 1. Upgrade_History view: 
> https://issues.apache.org/jira/secure/attachment/12811384/Upgrade_History_view.png
> 2. Upgrade details view: 
> https://issues.apache.org/jira/secure/attachment/12811385/Upgrade_Details_view.png
> 
> 
> Diffs
> -
> 
>   ambari-web/app/assets/test/tests.js 005c97b 
>   ambari-web/app/config.js 06faf27 
>   ambari-web/app/controllers.js 2537f62 
>   ambari-web/app/controllers/global/cluster_controller.js 0d9d41d 
>   ambari-web/app/controllers/main/admin/stack_upgrade_history_controller.js 
> PRE-CREATION 
>   ambari-web/app/mappers.js 96193bc 
>   ambari-web/app/mappers/stack_upgrade_history_mapper.js PRE-CREATION 
>   ambari-web/app/messages.js dcf0426 
>   ambari-web/app/models.js b877255 
>   ambari-web/app/models/finished_upgrade_entity.js PRE-CREATION 
>   ambari-web/app/models/stack_version/stack_upgrade_history.js PRE-CREATION 
>   ambari-web/app/routes/main.js 8d33a68 
>   ambari-web/app/styles/stack_versions.less 604b655 
>   ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history.hbs 
> PRE-CREATION 
>   
> ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history_details.hbs 
> PRE-CREATION 
>   ambari-web/app/views.js ab79b98 
>   ambari-web/app/views/main/admin/stack_upgrade/menu_view.js 1e84f1c 
>   
> ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_details_view.js 
> PRE-CREATION 
>   ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_view.js 
> PRE-CREATION 
>   
> ambari-web/test/controllers/main/admin/stack_upgrade_history_controller_test.js
>  PRE-CREATION 
>   ambari-web/test/mappers/stack_upgrade_history_mapper_test.js PRE-CREATION 
>   ambari-web/test/models/finished_upgrade_entity_test.js PRE-CREATION 
>   
> ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_details_view_test.js
>  PRE-CREATION 
>   ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_view_test.js 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/48855/diff/
> 
> 
> Testing
> ---
> 
> 1. patch a trunk cluster with the code change, ran express upgrades, express 
> downgrades, rolling upgrades and rolling downgrades. Check upgrade and 
> downgrade history displayed on the UI
> 2. new unit tests
> 3. existing unit tests
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 48855: AMBARI-12697 Rolling upgrade: Ambari UI should be able to display rolling upgrade history

2016-07-05 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48855/
---

(Updated July 5, 2016, 7:53 p.m.)


Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Richard Zang, Xi 
Wang, and Yusaku Sako.


Bugs: AMBARI-12697
https://issues.apache.org/jira/browse/AMBARI-12697


Repository: ambari


Description
---

Ambari UI shows operation history in the ops prompt. All upgrade and downgrade 
attempts are not shown on the UI once user finalizes it and close the 
upgrade/downgrade UI.

The information is in the database already. This JIRA is to propose a way to 
add UI elements to display a list of upgrades/downgrades associated with the 
cluster, including both express and rolling ones.

UI BEHAVIOR:

1. The new tab is the "Upgrade History" tabe shown on the Version and Stack 
page. Users will see three tabs now, the Version tab, the Stack tab and the 
Upgrade History tab. 
2. The Upgrade History tab shows a filter widget and a table showing the list 
of upgrades/downgrades based on the filter select. Default filter select is 
'All', so all upgrade and downgrade records from the Ambari database (upgrades 
tables) are shown on the UI. 
3. User clicks on a given record to open the detail view popup dialog that 
resembles the upgrade wizard UI. on the detail view popup dialog, user can 
drill up and down on upgrade steps for details.

UI SCREENSHOT:
1. Upgrade_History view: 
https://issues.apache.org/jira/secure/attachment/12811384/Upgrade_History_view.png
2. Upgrade details view: 
https://issues.apache.org/jira/secure/attachment/12811385/Upgrade_Details_view.png


Diffs (updated)
-

  ambari-web/app/assets/test/tests.js 005c97b 
  ambari-web/app/config.js 06faf27 
  ambari-web/app/controllers.js 2537f62 
  ambari-web/app/controllers/global/cluster_controller.js 0d9d41d 
  ambari-web/app/controllers/main/admin/stack_upgrade_history_controller.js 
PRE-CREATION 
  ambari-web/app/mappers.js 96193bc 
  ambari-web/app/mappers/stack_upgrade_history_mapper.js PRE-CREATION 
  ambari-web/app/messages.js dcf0426 
  ambari-web/app/models.js b877255 
  ambari-web/app/models/finished_upgrade_entity.js PRE-CREATION 
  ambari-web/app/models/stack_version/stack_upgrade_history.js PRE-CREATION 
  ambari-web/app/routes/main.js 8d33a68 
  ambari-web/app/styles/stack_versions.less 604b655 
  ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history.hbs 
PRE-CREATION 
  ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history_details.hbs 
PRE-CREATION 
  ambari-web/app/views.js ab79b98 
  ambari-web/app/views/main/admin/stack_upgrade/menu_view.js 1e84f1c 
  ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_details_view.js 
PRE-CREATION 
  ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_view.js 
PRE-CREATION 
  
ambari-web/test/controllers/main/admin/stack_upgrade_history_controller_test.js 
PRE-CREATION 
  ambari-web/test/mappers/stack_upgrade_history_mapper_test.js PRE-CREATION 
  ambari-web/test/models/finished_upgrade_entity_test.js PRE-CREATION 
  
ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_details_view_test.js
 PRE-CREATION 
  ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_view_test.js 
PRE-CREATION 

Diff: https://reviews.apache.org/r/48855/diff/


Testing (updated)
---

1. patch a trunk cluster with the code change, ran express upgrades, express 
downgrades, rolling upgrades and rolling downgrades. Check upgrade and 
downgrade history displayed on the UI
2. new unit tests
3. existing unit tests


Thanks,

Di Li

Re: Review Request 48628: AMBARI-17184: HBase doesn't start because of lacking of variable

2016-07-05 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48628/#review140861
---


Ship it!




Ship It!

- Sid Wagle


On July 5, 2016, 4:38 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48628/
> ---
> 
> (Updated July 5, 2016, 4:38 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-17184
> https://issues.apache.org/jira/browse/AMBARI-17184
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When we wanted to restart HBase, we got an error
> ```
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py",
>  line 159, in 
> HbaseMaster().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 257, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 686, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py",
>  line 86, in start
> self.configure(env) # for security
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py",
>  line 41, in configure
> hbase(name='master')
>   File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
> line 89, in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py",
>  line 166, in hbase
> tag = 'GANGLIA-MASTER' if name == 'master' else 'GANGLIA-RS'
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py",
>  line 233, in hbase_TemplateConfig
> template_tag = tag
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/template_config.py",
>  line 46, in action_create
> content = Template(template_name, 
> extra_imports=self.resource.extra_imports)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 123, in action_create
> content = self._get_content()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 160, in _get_content
> return content()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", 
> line 51, in __call__
> return self.get_content()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", 
> line 142, in get_content
> rendered = self.template.render(self.context)
>   File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 
> 891, in render
> return self.environment.handle_exception(exc_info, True)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2",
>  line 76, in top-level template code
> hbase.sink.timeline.truststore.password = {{metric_truststore_password}}
> ambari_jinja2.exceptions.UndefinedError: 'metric_collector_hosts' is undefined
> ```
> 
> 
> We should define metric_collector_hosts in 
> ambari/ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
>  6d304d4 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/templates/hadoop-metrics2-hbase.properties-GANGLIA-RS.j2
>  dcec3d4 
> 
> Diff: https://reviews.apache.org/r/48628/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test && manual test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-05 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140859
---




ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
(line 1476)


Based on this impl, we would get 1 warning at a time vs getting all dir 
with issues. Is this pattern consistent with other validations?


- Sid Wagle


On July 5, 2016, 2:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 5, 2016, 2:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 2944f6f 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 48973: AMBARI-17234. kafka should set zookeeper.set.acl to true when kerberos enabled

2016-07-05 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48973/#review140857
---


Ship it!




Ship It!

- Robert Levas


On June 20, 2016, 6:47 p.m., Sriharsha Chintalapani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48973/
> ---
> 
> (Updated June 20, 2016, 6:47 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-17234
> https://issues.apache.org/jira/browse/AMBARI-17234
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> kafka should set zookeeper.set.acl to true when kerberos enabled
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
> eaa3d9d 
> 
> Diff: https://reviews.apache.org/r/48973/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sriharsha Chintalapani
> 
>

Re: Review Request 48999: Fix Spark2 history server quick link still point to spark1 url


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48999/#review140852
---


Ship it!




Ship It!

- Alejandro Fernandez


On June 21, 2016, 3:14 a.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48999/
> ---
> 
> (Updated June 21, 2016, 3:14 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17328
> https://issues.apache.org/jira/browse/AMBARI-17328
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix Spark2 history server quick link still point to spark1 url
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/quicklinks/quicklinks.json
>  076755c 
> 
> Diff: https://reviews.apache.org/r/48999/diff/
> 
> 
> Testing
> ---
> 
> Local verification is done.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>

Review Request 49645: With wire-encryption enabled, quick links in ambari UI points to incorrect atlas web url

2016-07-05 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49645/
---

Review request for Ambari, Alejandro Fernandez, Zhe (Joe) Wang, and Yusaku Sako.


Bugs: AMBARI-17565
https://issues.apache.org/jira/browse/AMBARI-17565


Repository: ambari


Description
---

With wire-encryption enabled, quick links in ambari UI points to incorrect 
atlas web url.

Currently ambari points to http://:21000/.  where as 
wire-encryption enabled atlas runs on port 21443, so the correct url is 
https://:21443/.


The issue occured becaue the property deciding Atlas to be configured for HTTPS 
or not was renamed from metadata.enableTLS to atlas.enableTLS


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml 
88c1915 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/quicklinks/quicklinks.json
 PRE-CREATION 

Diff: https://reviews.apache.org/r/49645/diff/


Testing
---

Tested manually on a cluster that patch fixes the issue.
The patch only includes metadata changes in the Atlas stack definitions and so 
does not require unit tests.


Thanks,

Jaimin Jetly

Re: Review Request 49295: remove spark.driver.extraJavaOptions and spark.yarn.am.extraJavaOptions property in HDP2.5 definition

2016-07-05 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49295/#review140838
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/spark-defaults.xml
 (line 1)


The self detection mechanism for hdp.version was added in HDP-2.3.4. So 
shouldnt be remove these properties from HDP-2.4 onwards instead of HDP-2.5 
onwards?


- Jayush Luniya


On June 29, 2016, 1:29 a.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49295/
> ---
> 
> (Updated June 29, 2016, 1:29 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17459
> https://issues.apache.org/jira/browse/AMBARI-17459
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> remove spark.driver.extraJavaOptions and spark.yarn.am.extraJavaOptions 
> property in HDP2.5 definition
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/spark-defaults.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49295/diff/
> 
> 
> Testing
> ---
> 
> Local verification.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>

Re: Review Request 48971: AMBARI-17291 zookeeper.quorum in storm-metrics2.properties is broken

2016-07-05 Thread Masahiro Tanaka

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48971/
---

(Updated July 5, 2016, 6:20 p.m.)

Review request for Ambari, Andrew Onischuk, Gautam Borad, Sumit Mohanty, and
Sid Wagle.

Bugs: AMBARI-17291
https://issues.apache.org/jira/browse/AMBARI-17291

Repository: ambari

Description (updated)
---

When installed Storm and Ambari Metrics, `zookeeper.quorum` in
/etc/storm/conf/storm-metrics2.properties is looks like this.

```
zookeeper.quorum=[:2181,':2181,c:2181,7:2181,2:2181,0:2181,1:2181,.:2181,a:2181,m:2181,b:2181,a:2181,r:2181,i:2181,.:2181,a:2181,p:2181,a:2181,c:2181,h:2181,e:2181,.:2181,o:2181,r:2181,g:2181,':2181,,:2181,':2181,c:2181,7:2181,2:2181,0:2181,2:2181,.:2181,a:2181,m:2181,b:2181,a:2181,r:2181,i:2181,.:2181,a:2181,p:2181,a:2181,c:2181,h:2181,e:2181,.:2181,o:2181,r:2181,g:2181,':2181,,:2181,':2181,c:2181,7:2181,2:2181,0:2181,3:2181,.:2181,a:2181,m:2181,b:2181,a:2181,r:2181,i:2181,.:2181,a:2181,p:2181,a:2181,c:2181,h:2181,e:2181,.:2181,o:2181,r:2181,g:2181,':2181,]:2181
```

storm.zookeeper.servers is
`'c7201.ambari.apache.org','c7202.ambari.apache.org','c7203.ambari.apache.org']`

Steps to reproduce.

1. Install Ambari Server (I used
`http://s3.amazonaws.com/dev.hortonworks.com/ambari/centos7/2.x/latest/trunk/ambaribn.repo`)
2. Setup and start Ambari Server (`ambari-server setup -s` and `ambari-server
start`)
3. Install Storm and ZooKeeper via Ambari Server (HDP2.4)
4. Install Ambari Metrics
5. Restart all required

Diffs
-

ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
fac6331

Diff: https://reviews.apache.org/r/48971/diff/

Testing
---

mvn clean test && manual test
```
--1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12811836/AMBARI-17291.1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The test build failed in ambari-server
Test results:
https://builds.apache.org/job/Ambari-trunk-test-patch/7426//testReport/
Console output:
https://builds.apache.org/job/Ambari-trunk-test-patch/7426//console
This message is automatically generated.
```

Core test fails seems unrelated to this patch

Thanks,

Masahiro Tanaka

Re: Review Request 48971: AMBARI-17291 zookeeper.quorum in storm-metrics2.properties is broken

2016-07-05 Thread Masahiro Tanaka

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48971/
---

(Updated July 5, 2016, 6:16 p.m.)

Review request for Ambari, Andrew Onischuk, Gautam Borad, Sumit Mohanty, and
Sid Wagle.

Bugs: AMBARI-17291
https://issues.apache.org/jira/browse/AMBARI-17291

Repository: ambari

Description (updated)
---

When installed Storm and Ambari Metrics, `zookeeper.quorum` in
/etc/storm/conf/storm-metrics2.properties is looks like this.

storm.zookeeper.servers is
`'c7201.ambari.apache.org','c7202.ambari.apache.org','c7203.ambari.apache.org']`

Steps to reproduce.

Diffs
-

ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
fac6331

Diff: https://reviews.apache.org/r/48971/diff/

Testing
---

Core test fails seems unrelated to this patch

Thanks,

Masahiro Tanaka

Re: Review Request 49507: AMBARI-17520: Update the policy_user property to use storm user principal specified in Storms config


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49507/#review140828
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 5, 2016, 1:01 p.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49507/
> ---
> 
> (Updated July 5, 2016, 1:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Mugdha Varadkar, Robert 
> Levas, Sriharsha Chintalapani, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17520
> https://issues.apache.org/jira/browse/AMBARI-17520
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update the policy_user property in Advanced ranger-storm-plugin-properties of 
> Ranger with the value of the storm user bare principal specified in Storms 
> Ambari config.
> With this the principal used for storm will also be added to default ranger 
> policy and will prevent Storm service check failures.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
>  2fee04f 
> 
> Diff: https://reviews.apache.org/r/49507/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger storm plugin on centos6 cluster. Kerberized the cluster and 
> checked that Storm service check is working fine.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 49640: Identify config changes added to Ambari-2.4.0 and mark them to not get added during Ambari upgrade

2016-07-05 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49640/#review140827
---


Ship it!




Ship It!

- Jonathan Hurley


On July 5, 2016, 12:29 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49640/
> ---
> 
> (Updated July 5, 2016, 12:29 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17564
> https://issues.apache.org/jira/browse/AMBARI-17564
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now that we have the mechanism to prevent configs from getting added as part 
> of Ambari upgrade, lets identify the configs that got added in 2.4.0 and mark 
> them as not to be added during Ambari upgrade.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  b4eecec 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  871e571 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  3578d43 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
>  b8f7715 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  150b2c6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml
>  290239e 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
>  9811191 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  61437d5 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
>  d8012dd 
>   
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
>  29c46e9 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml
>  d0e51eb 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/configuration/tez-site.xml
>  e7a851c 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
>  6951db0 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
>  152c463 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  e1aab8a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
>  f3bbce8 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/OOZIE/configuration/oozie-site.xml
>  f2c41c3 
> 
> Diff: https://reviews.apache.org/r/49640/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 49535: Ambari Agent memory Leak fix.

2016-07-05 Thread Jonathan Hurley



> On July 5, 2016, 9:22 a.m., Jonathan Hurley wrote:
> > I don't see how this actually fixes the problem; importing gc from 
> > subprocess would be the same as importing it normally, no? There isn't a 
> > different gc module that subprocess uses. Also, after you patch it, you 
> > then re-import it - wouldn't that break the existing patching?
> 
> Andrew Onischuk wrote:
> "I don't see how this actually fixes the problem" - this was tested on 
> the instance where I could reproduce this and is a sure fix.
> 
> Jonathan Hurley wrote:
> So, let's say that it does fix the issue by always reporting that GC is 
> enabled. Wouldn't this then cause side effects since Popen is specifically 
> disabling it for a reason?
> 
> Andrew Onischuk wrote:
> popen disables gc anyway no matter if it is enabled or not.
> 
> Andrew Onischuk wrote:
> And even with fix it still disables it. The fix the for the race 
> condition, when it doesn't re-enable it again after the work is done.

OK, so `subprocess` doesn't check before it disables it; I can see that. 
However, we're changing the behavior of a function provided by the gc and we 
don't know what kinds of issues this could cause on other distributions of 
Python. There could be other versions which check this function and/or turning 
on the gc when it's already on could cause a problem.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/#review140761
---


On July 4, 2016, 4:45 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49535/
> ---
> 
> (Updated July 4, 2016, 4:45 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17539
> https://issues.apache.org/jira/browse/AMBARI-17539
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Reason of memory leak:*
> Race condition in subprocess python module. 
> Due to this race condition at some unlucky cases python garbage collection 
> was disabled. 
> This usually happened when running alerts, as a bunch of our alerts run shell 
> commands and they do it in a different threads.
> 
> *Fix for the issue:*
> Synchronizing subprocess is not the best option. Since some people can still 
> use it without synchronization not knowing about the issue. 
> Also synchronizing will provide some unnecessary slowdown. So for this issue 
> the proposed fix is to monkey patch subprocess.gc.isenabled.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 
> 
> Diff: https://reviews.apache.org/r/49535/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49429: Logsearch shows all the services not only those which are available in the stack

2016-07-05 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49429/#review140825
---


Ship it!




Ship It!

- Jayush Luniya


On June 30, 2016, 11:48 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49429/
> ---
> 
> (Updated June 30, 2016, 11:48 a.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Jayush Luniya, Oliver Szabo, 
> Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17492
> https://issues.apache.org/jira/browse/AMBARI-17492
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Cluster with HDF stack should show only HDF services, HDP cluster should not 
> show NiFi
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-assembly/pom.xml 958232e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  bdb5fb1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  8bb6225 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  fe7e757 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  a89b3d6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  bfab139 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/HadoopServiceConfig.json.j2
>  PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> 48e071d 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json 233a733 
> 
> Diff: https://reviews.apache.org/r/49429/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> ambari-server:
> OK
> --
> Total run:1085
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 49328: AMBARI-17465 - Management packs should be able to install extensions

2016-07-05 Thread Jayush Luniya


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49328/#review140824
---




ambari-server/src/main/python/ambari_server/serverConfiguration.py (line 187)


Aldo add property to ambari.properties


- Jayush Luniya


On June 30, 2016, 5:16 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49328/
> ---
> 
> (Updated June 30, 2016, 5:16 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17465
> https://issues.apache.org/jira/browse/AMBARI-17465
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently management packs (AMBARI-14854) can only add stacks and addon 
> services. Now that AMBARI-12885 has been resolved, the management packs 
> should be able to add extensions as well.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py e868f96 
>   ambari-server/src/main/python/ambari_server/setupMpacks.py 98811f5 
> 
> Diff: https://reviews.apache.org/r/49328/diff/
> 
> 
> Testing
> ---
> 
> Manual testing
> 
> ambari-server install-mpack 
> --mpack=/root/mpacks/hdp-ambari-mpack-1.0.0.0.tar.gz --purge -v
> ambari-server install-mpack 
> --mpack=/root/mpacks/myservice-ambari-mpack-1.0.0.0.tar.gz -v
> ambari-server install-mpack 
> --mpack=/root/mpacks/myservice2-ambari-mpack-1.0.0.0.tar.gz -v
> ambari-server install-mpack --mpack=/root/mpacks/myextension-1.0.0.0.tar.gz -v
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Re: Review Request 49173: AMBARI-17416: AMS-Grafana: show 'alias' when there's no datapoint available

2016-07-05 Thread Prajwal Rao



> On June 24, 2016, 4:54 p.m., Prajwal Rao wrote:
> > ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js, lines 
> > 128-133
> > 
> >
> > All graphs on HBase-Users have legend hidden.
> > 
> > We do need to handle this on aggregate level dashboards and graphs, and 
> > I already have a separate fix for that.
> 
> Jungtaek Lim wrote:
> Prajwal Rao
> I'm sorry but I don't get your feedback.
> I just would like to show alias instead of raw metric name since it could 
> be too long and not beauty to show when we introduce series aggregation ('%' 
> will be used to metric name). But I may miss some spots in patch since I 
> understand this javascript partially.
> 
> If you already have a patch to cover this issue, please let me know. I'd 
> be happy to close the review. You can also close issue AMBARI-17416 as 
> duplicated or take over.

Jungtaek, You can close this review. I have taken over AMBARI-17416 and will 
update the JIRA accordingly.


- Prajwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49173/#review139384
---


On June 24, 2016, 1:42 a.m., Jungtaek Lim wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49173/
> ---
> 
> (Updated June 24, 2016, 1:42 a.m.)
> 
> 
> Review request for Ambari, Prajwal Rao, Sriharsha Chintalapani, and Yusaku 
> Sako.
> 
> 
> Bugs: AMBARI-17416
> https://issues.apache.org/jira/browse/AMBARI-17416
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When there's no datapoint available, AMS Grafana plugin just ignores alias 
> and shows metric name instead. It seems to be not consistent in point of 
> view, and not good UX I think.
> 
> It'd be great AMS Grafana shows alias whenever there're datapoints available 
> or not.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-grafana/ambari-metrics/datasource.js 6a4f01b 
> 
> Diff: https://reviews.apache.org/r/49173/diff/
> 
> 
> Testing
> ---
> 
> Tested manually. Actually this is tiny fix so we can imagine how it will make 
> change.
> 
> 
> Thanks,
> 
> Jungtaek Lim
> 
>

Re: Review Request 48855: AMBARI-12697 Rolling upgrade: Ambari UI should be able to display rolling upgrade history

2016-07-05 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48855/
---

(Updated July 5, 2016, 5:07 p.m.)


Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Richard Zang, Xi 
Wang, and Yusaku Sako.


Bugs: AMBARI-12697
https://issues.apache.org/jira/browse/AMBARI-12697


Repository: ambari


Description
---

Ambari UI shows operation history in the ops prompt. All upgrade and downgrade 
attempts are not shown on the UI once user finalizes it and close the 
upgrade/downgrade UI.

The information is in the database already. This JIRA is to propose a way to 
add UI elements to display a list of upgrades/downgrades associated with the 
cluster, including both express and rolling ones.

UI BEHAVIOR:

1. The new tab is the "Upgrade History" tabe shown on the Version and Stack 
page. Users will see three tabs now, the Version tab, the Stack tab and the 
Upgrade History tab. 
2. The Upgrade History tab shows a filter widget and a table showing the list 
of upgrades/downgrades based on the filter select. Default filter select is 
'All', so all upgrade and downgrade records from the Ambari database (upgrades 
tables) are shown on the UI. 
3. User clicks on a given record to open the detail view popup dialog that 
resembles the upgrade wizard UI. on the detail view popup dialog, user can 
drill up and down on upgrade steps for details.

UI SCREENSHOT:
1. Upgrade_History view: 
https://issues.apache.org/jira/secure/attachment/12811384/Upgrade_History_view.png
2. Upgrade details view: 
https://issues.apache.org/jira/secure/attachment/12811385/Upgrade_Details_view.png


Diffs (updated)
-

  ambari-web/app/assets/test/tests.js 005c97b 
  ambari-web/app/controllers.js 2537f62 
  ambari-web/app/controllers/global/cluster_controller.js 0d9d41d 
  ambari-web/app/controllers/main/admin/stack_upgrade_history_controller.js 
PRE-CREATION 
  ambari-web/app/mappers.js 96193bc 
  ambari-web/app/mappers/stack_upgrade_history_mapper.js PRE-CREATION 
  ambari-web/app/messages.js dcf0426 
  ambari-web/app/models.js b877255 
  ambari-web/app/models/finished_upgrade_entity.js PRE-CREATION 
  ambari-web/app/models/stack_version/stack_upgrade_history.js PRE-CREATION 
  ambari-web/app/routes/main.js 8d33a68 
  ambari-web/app/styles/stack_versions.less 604b655 
  ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history.hbs 
PRE-CREATION 
  ambari-web/app/templates/main/admin/stack_upgrade/upgrade_history_details.hbs 
PRE-CREATION 
  ambari-web/app/views.js ab79b98 
  ambari-web/app/views/main/admin/stack_upgrade/menu_view.js 1e84f1c 
  ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_details_view.js 
PRE-CREATION 
  ambari-web/app/views/main/admin/stack_upgrade/upgrade_history_view.js 
PRE-CREATION 
  
ambari-web/test/controllers/main/admin/stack_upgrade_history_controller_test.js 
PRE-CREATION 
  ambari-web/test/mappers/stack_upgrade_history_mapper_test.js PRE-CREATION 
  ambari-web/test/models/finished_upgrade_entity_test.js PRE-CREATION 
  
ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_details_view_test.js
 PRE-CREATION 
  ambari-web/test/views/main/admin/stack_upgrade/upgrade_history_view_test.js 
PRE-CREATION 

Diff: https://reviews.apache.org/r/48855/diff/


Testing
---

1. patch a trunk cluster with the code change, ran express upgrades, express 
downgrades, rolling upgrades and rolling downgrades. Check upgrade and 
downgrade history displayed on the UI
2. new unit tests


Thanks,

Di Li

Re: Review Request 49640: Identify config changes added to Ambari-2.4.0 and mark them to not get added during Ambari upgrade

2016-07-05 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49640/#review140813
---


Ship it!




Ship It!

- Nate Cole


On July 5, 2016, 12:29 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49640/
> ---
> 
> (Updated July 5, 2016, 12:29 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17564
> https://issues.apache.org/jira/browse/AMBARI-17564
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now that we have the mechanism to prevent configs from getting added as part 
> of Ambari upgrade, lets identify the configs that got added in 2.4.0 and mark 
> them as not to be added during Ambari upgrade.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  b4eecec 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  871e571 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  3578d43 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
>  b8f7715 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  150b2c6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml
>  290239e 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
>  9811191 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  61437d5 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
>  d8012dd 
>   
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
>  29c46e9 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml
>  d0e51eb 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/configuration/tez-site.xml
>  e7a851c 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
>  6951db0 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
>  152c463 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  e1aab8a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
>  f3bbce8 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/OOZIE/configuration/oozie-site.xml
>  f2c41c3 
> 
> Diff: https://reviews.apache.org/r/49640/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 49618: Hive WebHCat Service Check Fails During Upgrade Due To Missing Configurations Files

2016-07-05 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49618/#review140812
---


Ship it!




Ship It!

- Nate Cole


On July 4, 2016, 9:02 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49618/
> ---
> 
> (Updated July 4, 2016, 9:02 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-17557
> https://issues.apache.org/jira/browse/AMBARI-17557
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A leftover variable from a prior loop was causing problems for 
> WebHCat/HCatalog symlinking from HDP 2.2 - Other components didn't have a 
> problem since they only have 1 directory to worry about correcting per 
> component. But Hive has two:
> 
> ```
>   "hive-hcatalog": [
> {
>   "conf_dir": "/etc/hive-webhcat/conf",
>   "prefix": "/etc/hive-webhcat",
>   "current_dir": 
> "{0}/current/hive-webhcat/etc/webhcat".format(STACK_ROOT_PATTERN)
> },
> {
>   "conf_dir": "/etc/hive-hcatalog/conf",
>   "prefix": "/etc/hive-hcatalog",
>   "current_dir": 
> "{0}/current/hive-webhcat/etc/hcatalog".format(STACK_ROOT_PATTERN)
> }
>   ]
> ```
> 
> As a result, we had a wrong symlink:
> ```
> [root@c6402 hive-webhcat]# ll /etc/hive-webhcat/
> total 8
> drwxr-xr-x 3 root root   4096 Jul  5 00:49 2.4.2.0-236
> lrwxrwxrwx 1 root root 29 Jul  5 00:49 conf -> 
> /etc/hive-hcatalog/conf.backup
> drwxr-xr-x 2 hcat hadoop 4096 Jul  4 23:25 conf.backup
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
>  c810438 
> 
> Diff: https://reviews.apache.org/r/49618/diff/
> 
> 
> Testing
> ---
> 
> --
> Total run:1095
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 49597: true was removed from stack definition

2016-07-05 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49597/#review140809
---


Ship it!




Ship It!

- Sumit Mohanty


On July 5, 2016, 3:02 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49597/
> ---
> 
> (Updated July 5, 2016, 3:02 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17555
> https://issues.apache.org/jira/browse/AMBARI-17555
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As of now, delete=false directive of on-ambari-upgrade is used to indicate 
> that properties are no longer available.
> We should bring back true. This is a directive for stack 
> inheritance. on-ambari-upgrade should only be used to indicate what happens 
> on ambari upgrade.
> Also, I do not think there is any property as of now that needs to be deleted 
> during ambari upgrade. So on-ambari-upgrade directives should have 
> delete=false.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
>  543c1ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 167fefa 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  97c9967 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  8c586a7 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
>  7100389 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
>  d281569 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
>  6519296 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
>  ca7fe2f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
>  57052b8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  1fd3e83 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  6b6c0bc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  082cdde 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
>  eda0fce 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
>  17bc454 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
>  72c864d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-site.xml
>  56b5752 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
>  1743bf8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  d0ee332 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml
>  03c8c83 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml
>  116a8b2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/ranger-hive-plugin-properties.xml
>  116a8b2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KNOX/configuration/ranger-knox-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/capacity-scheduler.xml
>  a71c111 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-site.xml
>  c2bc06d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  517a418 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/configuration/ranger-hbase-audit.xml
>  72c864d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/ranger-hdfs-audit.xml
>  bf81ff9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/ranger-hive-audit.xml
>  72c864d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/KAFKA/configuration/ranger-kafka-audit.xml
>  72c864d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/configuration/ranger-knox-audit.xml
>  72c864d 
>

Re: Review Request 49640: Identify config changes added to Ambari-2.4.0 and mark them to not get added during Ambari upgrade


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49640/
---

(Updated July 5, 2016, 7:29 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.


Bugs: AMBARI-17564
https://issues.apache.org/jira/browse/AMBARI-17564


Repository: ambari


Description
---

Now that we have the mechanism to prevent configs from getting added as part of 
Ambari upgrade, lets identify the configs that got added in 2.4.0 and mark them 
as not to be added during Ambari upgrade.


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
 b4eecec 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 871e571 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 3578d43 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
 b8f7715 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 150b2c6 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml
 290239e 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
 9811191 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
 61437d5 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
 d8012dd 
  
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
 29c46e9 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml
 d0e51eb 
  
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/configuration/tez-site.xml
 e7a851c 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
 6951db0 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
 152c463 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
89e05d7 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
 e1aab8a 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
 f3bbce8 
  
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/OOZIE/configuration/oozie-site.xml
 f2c41c3 

Diff: https://reviews.apache.org/r/49640/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 49626: NN HA enabling failed at 'Stop all Services' step


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49626/#review140805
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 5, 2016, 11 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49626/
> ---
> 
> (Updated July 5, 2016, 11 a.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, Nate 
> Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17559
> https://issues.apache.org/jira/browse/AMBARI-17559
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1) If a command gets aborted, it remains in "IN_PROGRESS" state on the agent. 
> Agent sends it in each heartbeat until agent gets restarted.
> 
> 2) Because of an ordering issue in the heartbeat report processing, server 
> sometimes mismatches command statuses. (If we have COMPLETED command and 
> ABORTED in the same heartbeat, it may think that completed command was 
> aborted and rejects state change). This situation was rare previously, but 
> because of the first issue aborted commands come in each Heartbeat.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 3832b81 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
>  2b121dc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/CommandUtils.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
>  50021c2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionManager.java
>  f85b95d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
>  e0f67af 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
>  913c4ea 
> 
> Diff: https://reviews.apache.org/r/49626/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49385: Hive and Oozie db displayed incorrectly on the installer review page

2016-07-05 Thread Andrii Tkach


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49385/#review140804
---


Ship it!




Ship It!

- Andrii Tkach


On June 30, 2016, 8:57 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49385/
> ---
> 
> (Updated June 30, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Andrii Tkach.
> 
> 
> Bugs: AMBARI-17469
> https://issues.apache.org/jira/browse/AMBARI-17469
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During Hive install, in the review page, the default value of 
> hive_admin_database (MySQL) is concatenated to the selected database type, no 
> matter which database is selected.  For e.g. if Existing PostgreSQL Database 
> is selected as the Hive database, the review page displays the following for 
> Hive database: 
> 
> Database : MySQL (Existing PostgreSQL Database)
>  
> In case of Oozie, because there is no oozie_admin_database property, a blank 
> is displayed for database although an existing database was selected
>  
> Database :  
>  
> This seems to be because of the logic in the method that determines the 
> database value to be displayed.
>  
> var dbFull = serviceConfigProperties.findProperty('name', 
> serviceName.toLowerCase() + '_database'),
>  db = serviceConfigProperties.findProperty('name', 
> serviceName.toLowerCase() + '_ambari_database');
> return db && dbFull ? db.value + ' (' + dbFull.value + ')' : '';
> 
> The value of hive_ambari_database returns MySQL and hence in case of Hive, 
> MySQL always gets appended.
>  
> There is no oozie_ambari_database property defined. Hence db is undefined and 
> an emtpy string is returned instead of the actual database type selected.
>  
> Fix involves changing the logic to not include the value of 
> serviceName_ambari_database since it will not have the right value unless the 
> default value is selected for Hive/Oozie database.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step8_controller.js 3971cf5 
>   ambari-web/test/controllers/wizard/step8_test.js 74e042b 
> 
> Diff: https://reviews.apache.org/r/49385/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> Added a test case to verify the value displayed for database.
> Ran mvn test
> 
> 28979 tests complete (48 seconds)
> 154 tests pending
> 
> 
> File Attachments
> 
> 
> Updated Patch with review comments incorporated
>   
> https://reviews.apache.org/media/uploaded/files/2016/06/30/e5fa58fa-3940-4c82-ad92-c8070c824528__AMBARI-17469.patch
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 49640: Identify config changes added to Ambari-2.4.0 and mark them to not get added during Ambari upgrade


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49640/#review140803
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 5, 2016, 3:53 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49640/
> ---
> 
> (Updated July 5, 2016, 3:53 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17564
> https://issues.apache.org/jira/browse/AMBARI-17564
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now that we have the mechanism to prevent configs from getting added as part 
> of Ambari upgrade, lets identify the configs that got added in 2.4.0 and mark 
> them as not to be added during Ambari upgrade.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
>  b4eecec 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  871e571 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  3578d43 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
>  b8f7715 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  150b2c6 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml
>  290239e 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
>  9811191 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  61437d5 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
>  d8012dd 
>   
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
>  29c46e9 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml
>  d0e51eb 
>   
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/configuration/tez-site.xml
>  e7a851c 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
>  6951db0 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
>  152c463 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  e1aab8a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
>  f3bbce8 
>   
> ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/OOZIE/configuration/oozie-site.xml
>  f2c41c3 
> 
> Diff: https://reviews.apache.org/r/49640/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140801
---




ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
(line 1453)


Can we move this out of stack advisor and into common services? We already 
have a file called mounted_dirs_helper.py


- Alejandro Fernandez


On July 5, 2016, 2:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 5, 2016, 2:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 2944f6f 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49590: While changing NN, DN directories from UI, proper warning should be present for invalid values

2016-07-05 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49590/#review140798
---


Ship it!




Ship It!

- Sid Wagle


On July 5, 2016, 2:38 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49590/
> ---
> 
> (Updated July 5, 2016, 2:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17550
> https://issues.apache.org/jira/browse/AMBARI-17550
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While changing NN and DN directories from ambari, for example:  
> changing dn directories from **/grid/0/hadoop/hdfs/data** to **/grid/0/hadoop/
> hdfs/data,/grid/0/hadoop/hdfs/data1,/grid/0/hadoop/hdfs/data2**
> 
> The values are changed without being validated  
> This leads to datanodes start failing
> 
> 
> 
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 174, in 
> DataNode().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 709, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 60, in start
> self.configure(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py",
>  line 55, in configure
> datanode(action="configure")
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py",
>  line 53, in datanode
> data_dir_to_mount_file_content = handle_mounted_dirs(create_dirs, 
> params.dfs_data_dirs, params.data_dir_mount_file, params)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/mounted_dirs_helper.py",
>  line 158, in handle_mounted_dirs
> raise Fail(message + " . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.")
> resource_management.core.exceptions.Fail: Trying to create another 
> directory on the following mount: /grid/0 . Please turn off 
> cluster-env/one_dir_per_partition or handle the situation manually.
> 
> 
> The test fails because it set invalid value, and the directory was not 
> created  
> A warning message informing why the new directory name is invalid could be
> useful
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py 
> 06f7cfe 
>   ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
> 7a092fc 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 08b9554 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 2944f6f 
> 
> Diff: https://reviews.apache.org/r/49590/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 49640: Identify config changes added to Ambari-2.4.0 and mark them to not get added during Ambari upgrade


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49640/
---

Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.


Bugs: AMBARI-17564
https://issues.apache.org/jira/browse/AMBARI-17564


Repository: ambari


Description
---

Now that we have the mechanism to prevent configs from getting added as part of 
Ambari upgrade, lets identify the configs that got added in 2.4.0 and mark them 
as not to be added during Ambari upgrade.


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-hbase-env.xml
 b4eecec 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 871e571 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 3578d43 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-env.xml
 b8f7715 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 150b2c6 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml
 290239e 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/configuration/hbase-env.xml
 9811191 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
 61437d5 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/webhcat-site.xml
 d8012dd 
  
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
 29c46e9 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml
 d0e51eb 
  
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/configuration/tez-site.xml
 e7a851c 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration-mapred/mapred-site.xml
 6951db0 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-env.xml
 152c463 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
89e05d7 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
 e1aab8a 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml
 f3bbce8 
  
ambari-server/src/main/resources/stacks/HDPWIN/2.3/services/OOZIE/configuration/oozie-site.xml
 f2c41c3 

Diff: https://reviews.apache.org/r/49640/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 49626: NN HA enabling failed at 'Stop all Services' step

2016-07-05 Thread Myroslav Papirkovskyy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49626/#review140797
---


Ship it!




Ship It!

- Myroslav Papirkovskyy


On Липень 5, 2016, 2 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49626/
> ---
> 
> (Updated Липень 5, 2016, 2 після полудня)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, Nate 
> Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17559
> https://issues.apache.org/jira/browse/AMBARI-17559
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1) If a command gets aborted, it remains in "IN_PROGRESS" state on the agent. 
> Agent sends it in each heartbeat until agent gets restarted.
> 
> 2) Because of an ordering issue in the heartbeat report processing, server 
> sometimes mismatches command statuses. (If we have COMPLETED command and 
> ABORTED in the same heartbeat, it may think that completed command was 
> aborted and rejects state change). This situation was rare previously, but 
> because of the first issue aborted commands come in each Heartbeat.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 3832b81 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
>  2b121dc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/CommandUtils.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
>  50021c2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionManager.java
>  f85b95d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
>  e0f67af 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
>  913c4ea 
> 
> Diff: https://reviews.apache.org/r/49626/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49597: true was removed from stack definition



> On July 4, 2016, 10:09 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml,
> >  line 144
> > 
> >
> > Can we say -  or simply keep it 
> > empty 
> 
> Dmitro Lisnichenko wrote:
> Empty  or  whould 
> mean add="true" (that is the default as of now, going to change in 2.5.0). 
> Also, as of now  tag implies  
> during ambari upgrade. Both proposed statements may look confusing.
> 
> Sumit Mohanty wrote:
> Ah! I missed that part where add has the default "true". 
> 
> Related question:
> {quote}as of now  tag implies  delete="true"/>{quote}
> 
> Was that the behavior before 2.4.0, where if a property is marked 
> "true, Amabri upgrade will delete these property?
> 
> Dmitro Lisnichenko wrote:
> As I see from code, before 2.4.0 we did not handle that at all. If 
> property became deleted in stack, it was not processed (e.g. added/removed) 
> during Ambari upgrade. That does not sound right, I think.
> 
> Sumit Mohanty wrote:
> Generaly, properties are marked deleted for the "next" stack version - 
> e.g. its in HDP-2.3 but starting HDP-2.4, this property is no longer 
> available. So it needs to be processed when stack is being upgraded. So 
> Ambari upgrade need not delete this property. 
> 
> Another argument is that before Ambari upgrade the stack is functional 
> with the property - so after upgrade it should also remain functional even if 
> the property is not deleted.
> 
> So keeping the same goal of Ambari upgrade should not impact stack 
> configurations, it seems alright to me that a stack "deleted" property should 
> not be automatically deleted on Ambari upgrade.

makes sense. Published new patch


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49597/#review140697
---


On July 5, 2016, 6:02 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49597/
> ---
> 
> (Updated July 5, 2016, 6:02 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17555
> https://issues.apache.org/jira/browse/AMBARI-17555
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As of now, delete=false directive of on-ambari-upgrade is used to indicate 
> that properties are no longer available.
> We should bring back true. This is a directive for stack 
> inheritance. on-ambari-upgrade should only be used to indicate what happens 
> on ambari upgrade.
> Also, I do not think there is any property as of now that needs to be deleted 
> during ambari upgrade. So on-ambari-upgrade directives should have 
> delete=false.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
>  543c1ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 167fefa 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  97c9967 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  8c586a7 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
>  7100389 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
>  d281569 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
>  6519296 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
>  ca7fe2f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
>  57052b8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  1fd3e83 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  6b6c0bc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  082cdde 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
>  eda0fce 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
>  17bc454 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
>  72c864d 
>   
>

Re: Review Request 49597: true was removed from stack definition


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49597/
---

(Updated July 5, 2016, 6:02 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.


Changes
---

true should not remove property during ambari upgrade


Bugs: AMBARI-17555
https://issues.apache.org/jira/browse/AMBARI-17555


Repository: ambari


Description
---

As of now, delete=false directive of on-ambari-upgrade is used to indicate that 
properties are no longer available.
We should bring back true. This is a directive for stack 
inheritance. on-ambari-upgrade should only be used to indicate what happens on 
ambari upgrade.
Also, I do not think there is any property as of now that needs to be deleted 
during ambari upgrade. So on-ambari-upgrade directives should have delete=false.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
 543c1ec 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
167fefa 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
 97c9967 
  
ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
 8c586a7 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
 7100389 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
 d281569 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
 6519296 
  
ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
 ca7fe2f 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
 57052b8 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
 1fd3e83 
  
ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
 6b6c0bc 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
 082cdde 
  
ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
 eda0fce 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
 c19d5ec 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
 17bc454 
  
ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
 72c864d 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-site.xml
 56b5752 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
 1743bf8 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
 d0ee332 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml
 03c8c83 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml
 116a8b2 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
 c19d5ec 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/ranger-hive-plugin-properties.xml
 116a8b2 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/KNOX/configuration/ranger-knox-plugin-properties.xml
 c19d5ec 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/capacity-scheduler.xml
 a71c111 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-site.xml
 c2bc06d 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
 517a418 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/configuration/ranger-hbase-audit.xml
 72c864d 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/ranger-hdfs-audit.xml
 bf81ff9 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/ranger-hive-audit.xml
 72c864d 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/KAFKA/configuration/ranger-kafka-audit.xml
 72c864d 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/configuration/ranger-knox-audit.xml
 72c864d 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/configuration/ranger-kms-audit.xml
 8a80485 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/spark-defaults.xml
 61c938e 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/ranger-yarn-audit.xml
 72c864d 
  
ambari-server/src/main/resources/stacks/HDPWIN/2.1/services/AMBARI_METRICS/configuration/ams-env.xml
 909e3fc

Re: Review Request 49507: AMBARI-17520: Update the policy_user property to use storm user principal specified in Storms config

2016-07-05 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49507/#review140788
---


Ship it!




Ship It!

- Robert Levas


On July 5, 2016, 9:01 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49507/
> ---
> 
> (Updated July 5, 2016, 9:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Mugdha Varadkar, Robert 
> Levas, Sriharsha Chintalapani, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17520
> https://issues.apache.org/jira/browse/AMBARI-17520
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update the policy_user property in Advanced ranger-storm-plugin-properties of 
> Ranger with the value of the storm user bare principal specified in Storms 
> Ambari config.
> With this the principal used for storm will also be added to default ranger 
> policy and will prevent Storm service check failures.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
>  2fee04f 
> 
> Diff: https://reviews.apache.org/r/49507/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger storm plugin on centos6 cluster. Kerberized the cluster and 
> checked that Storm service check is working fine.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 49393: Incorrect error message for view URL name and short URL validation

2016-07-05 Thread Alexandr Antonenko



> On July 5, 2016, 2:23 p.m., Alexandr Antonenko wrote:
> > Ship It!
> 
> Sangeeta Ravindran wrote:
> Thank you Alexandr. Can you help commit the change.

done


- Alexandr


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49393/#review140777
---


On June 29, 2016, 7:31 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49393/
> ---
> 
> (Updated June 29, 2016, 7:31 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-16785
> https://issues.apache.org/jira/browse/AMBARI-16785
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The create url UI flags an error about minimum length required for URL name 
> being 2 characters even if the URL name specified is 2 characters long.
> The create url UI flags an error about maximum length allowed for URL name 
> being 25 characters when the length of the URL name specified is greater than 
> 20 characters long.
> The create/edit url UI flags an error about minimum length required for short 
> URL being 2 characters even if the short URL specified is 2 characters long.
> 
> This seems to be because the minimum length for URL name is set to 3 via 
> ng-minlength="3" instead of 2 as the error message indicates.
> Also, the maximum length for URL name is set to 20 via ng-maxlength="20" 
> instead of 25 as the error message indicates.
> Also, the minimum length for short URL is set to 3 via ng-minlength="3" 
> instead of 2 as the error message indicates.
> 
> Fix involves updating the html files with values that match the error message 
> displayed.
> 
> 
> Diffs
> -
> 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/urls/create.html 
> a01024d 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/urls/edit.html 
> 1d0ff8c 
> 
> Diff: https://reviews.apache.org/r/49393/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> Ran mvn test.
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 49597: true was removed from stack definition

2016-07-05 Thread Sumit Mohanty



> On July 4, 2016, 7:09 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml,
> >  line 144
> > 
> >
> > Can we say -  or simply keep it 
> > empty 
> 
> Dmitro Lisnichenko wrote:
> Empty  or  whould 
> mean add="true" (that is the default as of now, going to change in 2.5.0). 
> Also, as of now  tag implies  
> during ambari upgrade. Both proposed statements may look confusing.
> 
> Sumit Mohanty wrote:
> Ah! I missed that part where add has the default "true". 
> 
> Related question:
> {quote}as of now  tag implies  delete="true"/>{quote}
> 
> Was that the behavior before 2.4.0, where if a property is marked 
> "true, Amabri upgrade will delete these property?
> 
> Dmitro Lisnichenko wrote:
> As I see from code, before 2.4.0 we did not handle that at all. If 
> property became deleted in stack, it was not processed (e.g. added/removed) 
> during Ambari upgrade. That does not sound right, I think.

Generaly, properties are marked deleted for the "next" stack version - e.g. its 
in HDP-2.3 but starting HDP-2.4, this property is no longer available. So it 
needs to be processed when stack is being upgraded. So Ambari upgrade need not 
delete this property. 

Another argument is that before Ambari upgrade the stack is functional with the 
property - so after upgrade it should also remain functional even if the 
property is not deleted.

So keeping the same goal of Ambari upgrade should not impact stack 
configurations, it seems alright to me that a stack "deleted" property should 
not be automatically deleted on Ambari upgrade.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49597/#review140697
---


On July 4, 2016, 4:13 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49597/
> ---
> 
> (Updated July 4, 2016, 4:13 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17555
> https://issues.apache.org/jira/browse/AMBARI-17555
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As of now, delete=false directive of on-ambari-upgrade is used to indicate 
> that properties are no longer available.
> We should bring back true. This is a directive for stack 
> inheritance. on-ambari-upgrade should only be used to indicate what happens 
> on ambari upgrade.
> Also, I do not think there is any property as of now that needs to be deleted 
> during ambari upgrade. So on-ambari-upgrade directives should have 
> delete=false.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
>  543c1ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 167fefa 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  97c9967 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  8c586a7 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
>  7100389 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
>  d281569 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
>  6519296 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
>  ca7fe2f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
>  57052b8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  1fd3e83 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  6b6c0bc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  082cdde 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
>  eda0fce 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
>  17bc454 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
>  72c864d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-site.xml
>  56b5752 
>   
>

Re: Review Request 49393: Incorrect error message for view URL name and short URL validation

2016-07-05 Thread Alexandr Antonenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49393/#review140777
---


Ship it!




Ship It!

- Alexandr Antonenko


On June 29, 2016, 7:31 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49393/
> ---
> 
> (Updated June 29, 2016, 7:31 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-16785
> https://issues.apache.org/jira/browse/AMBARI-16785
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The create url UI flags an error about minimum length required for URL name 
> being 2 characters even if the URL name specified is 2 characters long.
> The create url UI flags an error about maximum length allowed for URL name 
> being 25 characters when the length of the URL name specified is greater than 
> 20 characters long.
> The create/edit url UI flags an error about minimum length required for short 
> URL being 2 characters even if the short URL specified is 2 characters long.
> 
> This seems to be because the minimum length for URL name is set to 3 via 
> ng-minlength="3" instead of 2 as the error message indicates.
> Also, the maximum length for URL name is set to 20 via ng-maxlength="20" 
> instead of 25 as the error message indicates.
> Also, the minimum length for short URL is set to 3 via ng-minlength="3" 
> instead of 2 as the error message indicates.
> 
> Fix involves updating the html files with values that match the error message 
> displayed.
> 
> 
> Diffs
> -
> 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/urls/create.html 
> a01024d 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/urls/edit.html 
> 1d0ff8c 
> 
> Diff: https://reviews.apache.org/r/49393/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> Ran mvn test.
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 49597: true was removed from stack definition



> On July 4, 2016, 10:09 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml,
> >  line 144
> > 
> >
> > Can we say -  or simply keep it 
> > empty 
> 
> Dmitro Lisnichenko wrote:
> Empty  or  whould 
> mean add="true" (that is the default as of now, going to change in 2.5.0). 
> Also, as of now  tag implies  
> during ambari upgrade. Both proposed statements may look confusing.
> 
> Sumit Mohanty wrote:
> Ah! I missed that part where add has the default "true". 
> 
> Related question:
> {quote}as of now  tag implies  delete="true"/>{quote}
> 
> Was that the behavior before 2.4.0, where if a property is marked 
> "true, Amabri upgrade will delete these property?

As I see from code, before 2.4.0 we did not handle that at all. If property 
became deleted in stack, it was not processed (e.g. added/removed) during 
Ambari upgrade. That does not sound right, I think.


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49597/#review140697
---


On July 4, 2016, 7:13 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49597/
> ---
> 
> (Updated July 4, 2016, 7:13 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17555
> https://issues.apache.org/jira/browse/AMBARI-17555
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As of now, delete=false directive of on-ambari-upgrade is used to indicate 
> that properties are no longer available.
> We should bring back true. This is a directive for stack 
> inheritance. on-ambari-upgrade should only be used to indicate what happens 
> on ambari upgrade.
> Also, I do not think there is any property as of now that needs to be deleted 
> during ambari upgrade. So on-ambari-upgrade directives should have 
> delete=false.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
>  543c1ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 167fefa 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  97c9967 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  8c586a7 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
>  7100389 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
>  d281569 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
>  6519296 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
>  ca7fe2f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
>  57052b8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  1fd3e83 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  6b6c0bc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  082cdde 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
>  eda0fce 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
>  17bc454 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
>  72c864d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-site.xml
>  56b5752 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
>  1743bf8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  d0ee332 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml
>  03c8c83 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml
>  116a8b2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/ranger-hive-plugin-properties.xml
>  116a8b2 
>   
>

Review Request 49637: Zeppelin: service install failure on Suse due to bash error

2016-07-05 Thread Renjith Kamath


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49637/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17558
https://issues.apache.org/jira/browse/AMBARI-17558


Repository: ambari


Description
---

Fix the script failure on Suse with the following error due to outdated bash
/setup_snapshot.sh: line 44: ${SETUP_VIEW,,}: bad substitution


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/setup_snapshot.sh
 8612d64 

Diff: https://reviews.apache.org/r/49637/diff/


Testing
---

Manually tested


Thanks,

Renjith Kamath

Re: Review Request 49597: true was removed from stack definition

2016-07-05 Thread Sumit Mohanty



> On July 4, 2016, 7:09 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml,
> >  line 144
> > 
> >
> > Can we say -  or simply keep it 
> > empty 
> 
> Dmitro Lisnichenko wrote:
> Empty  or  whould 
> mean add="true" (that is the default as of now, going to change in 2.5.0). 
> Also, as of now  tag implies  
> during ambari upgrade. Both proposed statements may look confusing.

Ah! I missed that part where add has the default "true". 

Related question:
{quote}as of now  tag implies {quote}

Was that the behavior before 2.4.0, where if a property is marked 
"true, Amabri upgrade will delete these property?


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49597/#review140697
---


On July 4, 2016, 4:13 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49597/
> ---
> 
> (Updated July 4, 2016, 4:13 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17555
> https://issues.apache.org/jira/browse/AMBARI-17555
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> As of now, delete=false directive of on-ambari-upgrade is used to indicate 
> that properties are no longer available.
> We should bring back true. This is a directive for stack 
> inheritance. on-ambari-upgrade should only be used to indicate what happens 
> on ambari upgrade.
> Also, I do not think there is any property as of now that needs to be deleted 
> during ambari upgrade. So on-ambari-upgrade directives should have 
> delete=false.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationModule.java
>  543c1ec 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
> 167fefa 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  97c9967 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  8c586a7 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/admin-properties.xml
>  7100389 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-env.xml
>  d281569 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/ranger-site.xml
>  6519296 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.5.0/configuration/usersync-properties.xml
>  ca7fe2f 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/admin-properties.xml
>  57052b8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-admin-site.xml
>  1fd3e83 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.6.0/configuration/ranger-env.xml
>  6b6c0bc 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  082cdde 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-thrift-sparkconf.xml
>  eda0fce 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-site.xml
>  17bc454 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/ranger-storm-audit.xml
>  72c864d 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-site.xml
>  56b5752 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
>  1743bf8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/configuration/yarn-site.xml
>  d0ee332 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-site.xml
>  03c8c83 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-plugin-properties.xml
>  116a8b2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/ranger-hive-plugin-properties.xml
>  116a8b2 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/KNOX/configuration/ranger-knox-plugin-properties.xml
>  c19d5ec 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/capacity-scheduler.xml
>  a71c111 
>   
>

Re: Review Request 49635: Enable simulating logfeeder inputs

2016-07-05 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49635/#review140772
---


Ship it!




Ship It!

- Oliver Szabo


On July 5, 2016, 1:57 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49635/
> ---
> 
> (Updated July 5, 2016, 1:57 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17561
> https://issues.apache.org/jira/browse/AMBARI-17561
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enable simulating input files in a configurable way. The parameters has to be 
> set at custom logfeeder.properties
> 
> Available parameters:
> logfeeder.simulate.input_number - number of paralell inputs (threads) loading 
> the logs, if it is set and not 0 then all the rest of the configured inputs 
> are ignored (running in simulation mode)!!
> 
> logfeeder.simulate.log_ids - comma separated list of the log ids to propagate 
> at random, if not set by default all the available logs are propagated at 
> random
> 
> logfeeder.simulate.log_level - the level of the simulated log messages, by 
> default WARN
> 
> logfeeder.simulate.log_message_size - the length of the simulated log 
> messages, can't be less then 50 due to log message prefix, the rest is filled 
> with 'X' characters
> 
> logfeeder.simulate.sleep_milliseconds - the time interval at which each 
> simulated inputs writes one log message at random
> 
> The text of the log message is like this:
> Simulated log message for testing, line 0001 XX
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
>  88a6737 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json
>  978f581 
> 
> Diff: https://reviews.apache.org/r/49635/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 49429: Logsearch shows all the services not only those which are available in the stack

2016-07-05 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49429/#review140771
---


Ship it!




Ship It!

- Sebastian Toader


On June 30, 2016, 1:48 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49429/
> ---
> 
> (Updated June 30, 2016, 1:48 p.m.)
> 
> 
> Review request for Ambari, Don Bosco Durai, Jayush Luniya, Oliver Szabo, 
> Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17492
> https://issues.apache.org/jira/browse/AMBARI-17492
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Cluster with HDF stack should show only HDF services, HDP cluster should not 
> show NiFi
> 
> 
> Diffs
> -
> 
>   ambari-logsearch/ambari-logsearch-assembly/pom.xml 958232e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  bdb5fb1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  8bb6225 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  fe7e757 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/params.py
>  a89b3d6 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/setup_logsearch.py
>  bfab139 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/HadoopServiceConfig.json.j2
>  PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logsearch.py 
> 48e071d 
>   ambari-server/src/test/python/stacks/2.4/configs/default.json 233a733 
> 
> Diff: https://reviews.apache.org/r/49429/diff/
> 
> 
> Testing
> ---
> 
> Tested on local cluster.
> 
> ambari-server:
> OK
> --
> Total run:1085
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Review Request 49635: Enable simulating logfeeder inputs

2016-07-05 Thread Miklos Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49635/
---

Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.


Bugs: AMBARI-17561
https://issues.apache.org/jira/browse/AMBARI-17561


Repository: ambari


Description
---

Enable simulating input files in a configurable way. The parameters has to be 
set at custom logfeeder.properties

Available parameters:
logfeeder.simulate.input_number - number of paralell inputs (threads) loading 
the logs, if it is set and not 0 then all the rest of the configured inputs are 
ignored (running in simulation mode)!!

logfeeder.simulate.log_ids - comma separated list of the log ids to propagate 
at random, if not set by default all the available logs are propagated at random

logfeeder.simulate.log_level - the level of the simulated log messages, by 
default WARN

logfeeder.simulate.log_message_size - the length of the simulated log messages, 
can't be less then 50 due to log message prefix, the rest is filled with 'X' 
characters

logfeeder.simulate.sleep_milliseconds - the time interval at which each 
simulated inputs writes one log message at random

The text of the log message is like this:
Simulated log message for testing, line 0001 XX


Diffs
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
 88a6737 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputSimulate.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/alias_config.json
 978f581 

Diff: https://reviews.apache.org/r/49635/diff/


Testing
---

Tested on local cluster.


Thanks,

Miklos Gergely

Re: Review Request 49535: Ambari Agent memory Leak fix.



> On July 5, 2016, 1:22 p.m., Jonathan Hurley wrote:
> > I don't see how this actually fixes the problem; importing gc from 
> > subprocess would be the same as importing it normally, no? There isn't a 
> > different gc module that subprocess uses. Also, after you patch it, you 
> > then re-import it - wouldn't that break the existing patching?
> 
> Andrew Onischuk wrote:
> "I don't see how this actually fixes the problem" - this was tested on 
> the instance where I could reproduce this and is a sure fix.
> 
> Jonathan Hurley wrote:
> So, let's say that it does fix the issue by always reporting that GC is 
> enabled. Wouldn't this then cause side effects since Popen is specifically 
> disabling it for a reason?
> 
> Andrew Onischuk wrote:
> popen disables gc anyway no matter if it is enabled or not.

And even with fix it still disables it. The fix the for the race condition, 
when it doesn't re-enable it again after the work is done.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/#review140761
---


On July 4, 2016, 8:45 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49535/
> ---
> 
> (Updated July 4, 2016, 8:45 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17539
> https://issues.apache.org/jira/browse/AMBARI-17539
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Reason of memory leak:*
> Race condition in subprocess python module. 
> Due to this race condition at some unlucky cases python garbage collection 
> was disabled. 
> This usually happened when running alerts, as a bunch of our alerts run shell 
> commands and they do it in a different threads.
> 
> *Fix for the issue:*
> Synchronizing subprocess is not the best option. Since some people can still 
> use it without synchronization not knowing about the issue. 
> Also synchronizing will provide some unnecessary slowdown. So for this issue 
> the proposed fix is to monkey patch subprocess.gc.isenabled.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 
> 
> Diff: https://reviews.apache.org/r/49535/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49535: Ambari Agent memory Leak fix.



> On July 5, 2016, 1:22 p.m., Jonathan Hurley wrote:
> > I don't see how this actually fixes the problem; importing gc from 
> > subprocess would be the same as importing it normally, no? There isn't a 
> > different gc module that subprocess uses. Also, after you patch it, you 
> > then re-import it - wouldn't that break the existing patching?
> 
> Andrew Onischuk wrote:
> "I don't see how this actually fixes the problem" - this was tested on 
> the instance where I could reproduce this and is a sure fix.
> 
> Jonathan Hurley wrote:
> So, let's say that it does fix the issue by always reporting that GC is 
> enabled. Wouldn't this then cause side effects since Popen is specifically 
> disabling it for a reason?

popen disables gc anyway no matter if it is enabled or not.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/#review140761
---


On July 4, 2016, 8:45 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49535/
> ---
> 
> (Updated July 4, 2016, 8:45 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17539
> https://issues.apache.org/jira/browse/AMBARI-17539
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Reason of memory leak:*
> Race condition in subprocess python module. 
> Due to this race condition at some unlucky cases python garbage collection 
> was disabled. 
> This usually happened when running alerts, as a bunch of our alerts run shell 
> commands and they do it in a different threads.
> 
> *Fix for the issue:*
> Synchronizing subprocess is not the best option. Since some people can still 
> use it without synchronization not knowing about the issue. 
> Also synchronizing will provide some unnecessary slowdown. So for this issue 
> the proposed fix is to monkey patch subprocess.gc.isenabled.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 
> 
> Diff: https://reviews.apache.org/r/49535/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49535: Ambari Agent memory Leak fix.



> On July 5, 2016, 1:22 p.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/main.py, lines 30-31
> > 
> >
> > I don't think that subprocess has a different version of gc than normal 
> > python flow; how is subprocess.gc different than gc?
> 
> Andrew Onischuk wrote:
> It is for readability purposes only

And also helps to avoid sutiations when subprocess was already imported and has 
a reference to different gc module. (oposed to simply patching gc)


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/#review140761
---


On July 4, 2016, 8:45 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49535/
> ---
> 
> (Updated July 4, 2016, 8:45 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17539
> https://issues.apache.org/jira/browse/AMBARI-17539
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Reason of memory leak:*
> Race condition in subprocess python module. 
> Due to this race condition at some unlucky cases python garbage collection 
> was disabled. 
> This usually happened when running alerts, as a bunch of our alerts run shell 
> commands and they do it in a different threads.
> 
> *Fix for the issue:*
> Synchronizing subprocess is not the best option. Since some people can still 
> use it without synchronization not knowing about the issue. 
> Also synchronizing will provide some unnecessary slowdown. So for this issue 
> the proposed fix is to monkey patch subprocess.gc.isenabled.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 
> 
> Diff: https://reviews.apache.org/r/49535/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49535: Ambari Agent memory Leak fix.



> On July 5, 2016, 1:22 p.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/main.py, lines 30-31
> > 
> >
> > I don't think that subprocess has a different version of gc than normal 
> > python flow; how is subprocess.gc different than gc?

It is for readability purposes only


> On July 5, 2016, 1:22 p.m., Jonathan Hurley wrote:
> > ambari-agent/src/main/python/ambari_agent/main.py, lines 33-36
> > 
> >
> > Could you just do a reload(gc) here?

reload python module does more complicated things than this, here we simply 
need to reimport existing compiled module.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/#review140761
---


On July 4, 2016, 8:45 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49535/
> ---
> 
> (Updated July 4, 2016, 8:45 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17539
> https://issues.apache.org/jira/browse/AMBARI-17539
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Reason of memory leak:*
> Race condition in subprocess python module. 
> Due to this race condition at some unlucky cases python garbage collection 
> was disabled. 
> This usually happened when running alerts, as a bunch of our alerts run shell 
> commands and they do it in a different threads.
> 
> *Fix for the issue:*
> Synchronizing subprocess is not the best option. Since some people can still 
> use it without synchronization not knowing about the issue. 
> Also synchronizing will provide some unnecessary slowdown. So for this issue 
> the proposed fix is to monkey patch subprocess.gc.isenabled.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 
> 
> Diff: https://reviews.apache.org/r/49535/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49535: Ambari Agent memory Leak fix.

2016-07-05 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/#review140761
---



I don't see how this actually fixes the problem; importing gc from subprocess 
would be the same as importing it normally, no? There isn't a different gc 
module that subprocess uses. Also, after you patch it, you then re-import it - 
wouldn't that break the existing patching?


ambari-agent/src/main/python/ambari_agent/main.py (lines 30 - 31)


I don't think that subprocess has a different version of gc than normal 
python flow; how is subprocess.gc different than gc?



ambari-agent/src/main/python/ambari_agent/main.py (lines 33 - 36)


Could you just do a reload(gc) here?


- Jonathan Hurley


On July 4, 2016, 4:45 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49535/
> ---
> 
> (Updated July 4, 2016, 4:45 a.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17539
> https://issues.apache.org/jira/browse/AMBARI-17539
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Reason of memory leak:*
> Race condition in subprocess python module. 
> Due to this race condition at some unlucky cases python garbage collection 
> was disabled. 
> This usually happened when running alerts, as a bunch of our alerts run shell 
> commands and they do it in a different threads.
> 
> *Fix for the issue:*
> Synchronizing subprocess is not the best option. Since some people can still 
> use it without synchronization not knowing about the issue. 
> Also synchronizing will provide some unnecessary slowdown. So for this issue 
> the proposed fix is to monkey patch subprocess.gc.isenabled.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 
> 
> Diff: https://reviews.apache.org/r/49535/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49507: AMBARI-17520: Update the policy_user property to use storm user principal specified in Storms config

2016-07-05 Thread Gautam Borad


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49507/
---

(Updated July 5, 2016, 1:01 p.m.)


Review request for Ambari, Alejandro Fernandez, Mugdha Varadkar, Robert Levas, 
Sriharsha Chintalapani, Srimanth Gunturi, and Velmurugan Periasamy.


Changes
---

Handle Robert's comment


Bugs: AMBARI-17520
https://issues.apache.org/jira/browse/AMBARI-17520


Repository: ambari


Description
---

Update the policy_user property in Advanced ranger-storm-plugin-properties of 
Ranger with the value of the storm user bare principal specified in Storms 
Ambari config.
With this the principal used for storm will also be added to default ranger 
policy and will prevent Storm service check failures.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 073bb1c 
  
ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
 2fee04f 

Diff: https://reviews.apache.org/r/49507/diff/


Testing
---

Tested Ranger storm plugin on centos6 cluster. Kerberized the cluster and 
checked that Storm service check is working fine.


Thanks,

Gautam Borad

Re: Review Request 49393: Incorrect error message for view URL name and short URL validation

2016-07-05 Thread Sangeeta Ravindran



> On June 29, 2016, 7:35 p.m., Di Li wrote:
> > Ship It!

Thanks Di. 

Hi Alexander, can you please review the changes?

Thanks,
Sangeeta


- Sangeeta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49393/#review140043
---


On June 29, 2016, 7:31 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49393/
> ---
> 
> (Updated June 29, 2016, 7:31 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-16785
> https://issues.apache.org/jira/browse/AMBARI-16785
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The create url UI flags an error about minimum length required for URL name 
> being 2 characters even if the URL name specified is 2 characters long.
> The create url UI flags an error about maximum length allowed for URL name 
> being 25 characters when the length of the URL name specified is greater than 
> 20 characters long.
> The create/edit url UI flags an error about minimum length required for short 
> URL being 2 characters even if the short URL specified is 2 characters long.
> 
> This seems to be because the minimum length for URL name is set to 3 via 
> ng-minlength="3" instead of 2 as the error message indicates.
> Also, the maximum length for URL name is set to 20 via ng-maxlength="20" 
> instead of 25 as the error message indicates.
> Also, the minimum length for short URL is set to 3 via ng-minlength="3" 
> instead of 2 as the error message indicates.
> 
> Fix involves updating the html files with values that match the error message 
> displayed.
> 
> 
> Diffs
> -
> 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/urls/create.html 
> a01024d 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/urls/edit.html 
> 1d0ff8c 
> 
> Diff: https://reviews.apache.org/r/49393/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> Ran mvn test.
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 49385: Hive and Oozie db displayed incorrectly on the installer review page

2016-07-05 Thread Sangeeta Ravindran



> On June 30, 2016, 10:52 a.m., Andrii Tkach wrote:
> >

Hello Andrii,

I have updated the patch with the changes suggested. Can you please review the 
changes.

Thanks,
Sangeeta


- Sangeeta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49385/#review140134
---


On June 30, 2016, 8:57 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49385/
> ---
> 
> (Updated June 30, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Andrii Tkach.
> 
> 
> Bugs: AMBARI-17469
> https://issues.apache.org/jira/browse/AMBARI-17469
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During Hive install, in the review page, the default value of 
> hive_admin_database (MySQL) is concatenated to the selected database type, no 
> matter which database is selected.  For e.g. if Existing PostgreSQL Database 
> is selected as the Hive database, the review page displays the following for 
> Hive database: 
> 
> Database : MySQL (Existing PostgreSQL Database)
>  
> In case of Oozie, because there is no oozie_admin_database property, a blank 
> is displayed for database although an existing database was selected
>  
> Database :  
>  
> This seems to be because of the logic in the method that determines the 
> database value to be displayed.
>  
> var dbFull = serviceConfigProperties.findProperty('name', 
> serviceName.toLowerCase() + '_database'),
>  db = serviceConfigProperties.findProperty('name', 
> serviceName.toLowerCase() + '_ambari_database');
> return db && dbFull ? db.value + ' (' + dbFull.value + ')' : '';
> 
> The value of hive_ambari_database returns MySQL and hence in case of Hive, 
> MySQL always gets appended.
>  
> There is no oozie_ambari_database property defined. Hence db is undefined and 
> an emtpy string is returned instead of the actual database type selected.
>  
> Fix involves changing the logic to not include the value of 
> serviceName_ambari_database since it will not have the right value unless the 
> default value is selected for Hive/Oozie database.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step8_controller.js 3971cf5 
>   ambari-web/test/controllers/wizard/step8_test.js 74e042b 
> 
> Diff: https://reviews.apache.org/r/49385/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> Added a test case to verify the value displayed for database.
> Ran mvn test
> 
> 28979 tests complete (48 seconds)
> 154 tests pending
> 
> 
> File Attachments
> 
> 
> Updated Patch with review comments incorporated
>   
> https://reviews.apache.org/media/uploaded/files/2016/06/30/e5fa58fa-3940-4c82-ad92-c8070c824528__AMBARI-17469.patch
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 49626: NN HA enabling failed at 'Stop all Services' step


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49626/#review140756
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On July 5, 2016, 2 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49626/
> ---
> 
> (Updated July 5, 2016, 2 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, Nate 
> Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17559
> https://issues.apache.org/jira/browse/AMBARI-17559
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1) If a command gets aborted, it remains in "IN_PROGRESS" state on the agent. 
> Agent sends it in each heartbeat until agent gets restarted.
> 
> 2) Because of an ordering issue in the heartbeat report processing, server 
> sometimes mismatches command statuses. (If we have COMPLETED command and 
> ABORTED in the same heartbeat, it may think that completed command was 
> aborted and rejects state change). This situation was rare previously, but 
> because of the first issue aborted commands come in each Heartbeat.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 3832b81 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
>  2b121dc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
>  c6036c2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/CommandUtils.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
>  50021c2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionManager.java
>  f85b95d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
>  e0f67af 
>   
> ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
>  913c4ea 
> 
> Diff: https://reviews.apache.org/r/49626/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49594: 'Restart all' services fails due to dependency on other service components

2016-07-05 Thread Dmytro Sen



> On Июль 4, 2016, 3 п.п., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/role_command_order.json, 
> > line 46
> > 
> >
> > is this just duplicating 'start' dependencies? Shouldn't we rewrite 
> > restart to use start dependencies?

Will be implemented in a separate jira


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49594/#review140664
---


On Июль 4, 2016, 2:52 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49594/
> ---
> 
> (Updated Июль 4, 2016, 2:52 п.п.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-17552
> https://issues.apache.org/jira/browse/AMBARI-17552
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Looks like we have a dependency mechanism in place due to which the newly 
> added "Restart all Services" operation does not work as expected in some cases
> Here is my scenario:
> 1. Deploy a cluster (HDP-2.4.2.0) with core services (HDFS, YARN, ZK, MR) and 
> Ambari-2.4.0.0 (ambari-server-2.4.0.0-680.x86_64)
> 2. Add Spark, Hive, Tez, Pig to the cluster and let it finish
> 3. At the end of service addition, there is an icon against HDFS, YARN, MR 
> indicating these three services need restart
> 4. Try 'Restart all Required' operation
> Result:
> Failures at ATS start, which aborts the whole restart operation.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.0.6/role_command_order.json 
> 710b0dd 
>   ambari-server/src/main/resources/stacks/HDP/2.1/role_command_order.json 
> 3adf5e1 
> 
> Diff: https://reviews.apache.org/r/49594/diff/
> 
> 
> Testing
> ---
> 
> Manual unit tests
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49521: Move service advisor tests for HAWQ and PXF

2016-07-05 Thread Tim Thorpe


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49521/#review140754
---


Ship it!




Ship It!

- Tim Thorpe


On July 4, 2016, 12:33 a.m., Lav Jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49521/
> ---
> 
> (Updated July 4, 2016, 12:33 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
> Luniya, Matt, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-16647
> https://issues.apache.org/jira/browse/AMBARI-16647
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the service advisor tests for HAWQ and PXF are under 
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py
> Move the tests to the service specific service_advisor files:
> ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py
> ambari-server/src/test/python/common-services/PXF/test_service_advisor.py
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  e254094 
>   ambari-server/src/main/resources/stacks/stack_advisor.py d8685c3 
>   ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/common-services/PXF/test_service_advisor.py 
> PRE-CREATION 
>   ambari-server/src/test/python/common-services/configs/hawq_default.json 
> PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-hawq-3-hosts.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-hawq-pxf-hdfs.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-master_ambari_colo-3-hosts.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-master_standby_colo-3-hosts.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-nohawq-3-hosts.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-normal-hawq-3-hosts.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-normal-nohawq-3-hosts.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/common-services/configs/services-standby_ambari_colo-3-hosts.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_component_status.py  
>   
> ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_segment_registration_status.py
>  6bb5930 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_sync_status.py 
> fd4f474 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 88fb008 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 8639ca5 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py b406723 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 
> 8d97baa 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_utils.py  
>   ambari-server/src/test/python/stacks/2.3/PXF/test_alerts_api_status.py  
>   ambari-server/src/test/python/stacks/2.3/PXF/test_pxf.py 1147a7e 
>   ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py 
> 4ea3bfb 
>   ambari-server/src/test/python/stacks/2.3/common/hosts-1-host.json  
>   ambari-server/src/test/python/stacks/2.3/common/hosts-3-hosts.json  
>   ambari-server/src/test/python/stacks/2.3/common/services-hawq-1-host.json 
> 515ba7d 
>   ambari-server/src/test/python/stacks/2.3/common/services-hawq-3-hosts.json 
> 515ba7d 
>   ambari-server/src/test/python/stacks/2.3/common/services-hawq-pxf-hdfs.json 
> 0bf459d 
>   
> ambari-server/src/test/python/stacks/2.3/common/services-master_ambari_colo-3-hosts.json
>  1657ccf 
>   
> ambari-server/src/test/python/stacks/2.3/common/services-master_standby_colo-3-hosts.json
>  cd5d02c 
>   
> ambari-server/src/test/python/stacks/2.3/common/services-nohawq-3-hosts.json 
> beeb62d 
>   
> ambari-server/src/test/python/stacks/2.3/common/services-normal-hawq-3-hosts.json
>  5495d77 
>   
> ambari-server/src/test/python/stacks/2.3/common/services-normal-nohawq-3-hosts.json
>  2149317 
>   
> ambari-server/src/test/python/stacks/2.3/common/services-standby_ambari_colo-3-hosts.json
>  92a8e58 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 2944f6f 
>   ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json ebff461 
>   ambari-server/src/test/python/stacks/2.3/configs/pxf_default.json  
> 
> Diff: https://reviews.apache.org/r/49521/diff/
> 
> 
> Testing
> ---
> 
> Total run:1005
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Lav Jain
> 
>

Review Request 49626: NN HA enabling failed at 'Stop all Services' step


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49626/
---

Review request for Ambari and Myroslav Papirkovskyy.


Bugs: AMBARI-17559
https://issues.apache.org/jira/browse/AMBARI-17559


Repository: ambari


Description
---

1) If a command gets aborted, it remains in "IN_PROGRESS" state on the agent. 
Agent sends it in each heartbeat until agent gets restarted.

2) Because of an ordering issue in the heartbeat report processing, server 
sometimes mismatches command statuses. (If we have COMPLETED command and 
ABORTED in the same heartbeat, it may think that completed command was aborted 
and rejects state change). This situation was rare previously, but because of 
the first issue aborted commands come in each Heartbeat.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py 3832b81 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionManager.java
 2b121dc 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatProcessor.java
 c6036c2 
  ambari-server/src/main/java/org/apache/ambari/server/utils/CommandUtils.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionDBAccessorImpl.java
 50021c2 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionManager.java
 f85b95d 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
 e0f67af 
  
ambari-server/src/test/java/org/apache/ambari/server/agent/HeartbeatProcessorTest.java
 913c4ea 

Diff: https://reviews.apache.org/r/49626/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49597: true was removed from stack definition