Re: Review Request 50777: AMBARI-18014: PXF service definition is missing pxf-json profile

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50777/#review144711
---


Ship it!




Ship It!

- Lav Jain


On Aug. 4, 2016, 1:24 a.m., Alexander Denissov wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50777/
> ---
> 
> (Updated Aug. 4, 2016, 1:24 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Lav Jain, and 
> Matt.
> 
> 
> Bugs: AMBARI-18014
> https://issues.apache.org/jira/browse/AMBARI-18014
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PXF service definition is missing pxf-json profile
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/PXF/3.0.0/configuration/pxf-profiles.xml
>  69e3228 
>   ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 
> 5978d0b 
>   ambari-server/src/test/python/common-services/PXF/test_pxf.py 49df75d 
>   ambari-server/src/test/python/common-services/configs/pxf_default.json 
> 10cd9a4 
> 
> Diff: https://reviews.apache.org/r/50777/diff/
> 
> 
> Testing
> ---
> 
> --
> Ran 16 tests in 0.050s
> 
> OK
> 
> 
> Thanks,
> 
> Alexander Denissov
> 
>

Re: Review Request 50731: HAWQ service advisor shows wrong recommendations on edge cases

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50731/#review144709
---


Ship it!




Ship It!

- Lav Jain


On Aug. 4, 2016, 1:11 a.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50731/
> ---
> 
> (Updated Aug. 4, 2016, 1:11 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov and Lav Jain.
> 
> 
> Bugs: AMBARI-17996
> https://issues.apache.org/jira/browse/AMBARI-17996
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HAWQ service advisor shows wrong recommendations on edge cases
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  9baaa1c 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  0777dc6 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 437fe4f 
>   ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
> 7ececa0 
> 
> Diff: https://reviews.apache.org/r/50731/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> 
> Added unit tests
> ```
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-08-02 18:01:23,293 - Removing 
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-08-02 18:01:23,295 - HAWQ 
> Standby Master Re-Sync started in fast mode...
> ok
> test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
> Kerberized ... 2016-08-02 18:01:23,298 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
> Kerberized ... 2016-08-02 18:01:23,304 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not 
> Kerberized ... 2016-08-02 18:01:23,309 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case12 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 12: HDFS-HA, YARN Resource Management YARN_HA, 
> Kerberized ... 2016-08-02 18:01:23,315 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case2 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 2: Non HDFS-HA, Standalone Resource Management, 
> Kerberized ... 

Review Request 50777: AMBARI-18014: PXF service definition is missing pxf-json profile

[Alexander Denissov]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50777/
---

Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Lav Jain, and 
Matt.


Bugs: AMBARI-18014
https://issues.apache.org/jira/browse/AMBARI-18014


Repository: ambari


Description
---

PXF service definition is missing pxf-json profile


Diffs
-

  
ambari-server/src/main/resources/common-services/PXF/3.0.0/configuration/pxf-profiles.xml
 69e3228 
  ambari-server/src/main/resources/common-services/PXF/3.0.0/metainfo.xml 
5978d0b 
  ambari-server/src/test/python/common-services/PXF/test_pxf.py 49df75d 
  ambari-server/src/test/python/common-services/configs/pxf_default.json 
10cd9a4 

Diff: https://reviews.apache.org/r/50777/diff/


Testing
---

--
Ran 16 tests in 0.050s

OK


Thanks,

Alexander Denissov

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Jameel Mohamed]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/
---

(Updated Aug. 4, 2016, 1:24 a.m.)


Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.


Changes
---

Changed Interface to use Precision


Bugs: AMBARI-10145
https://issues.apache.org/jira/browse/AMBARI-10145


Repository: ambari


Description
---

Added new Interface to drain aggregated metrics to external storage. Added unit 
test and sample implementation (TimelineMetricsAggregatorMemorySink.java) to 
store metrics in memory.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 dfd730d 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
 e82d65a 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
 0ea668a 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/50666/diff/


Testing
---

Manually tested.


Added unit test.


mvn clean test


Thanks,

Jameel Mohamed

Re: Review Request 50731: HAWQ service advisor shows wrong recommendations on edge cases

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50731/
---

(Updated Aug. 3, 2016, 6:11 p.m.)


Review request for Ambari, Alexander Denissov and Lav Jain.


Bugs: AMBARI-17996
https://issues.apache.org/jira/browse/AMBARI-17996


Repository: ambari


Description
---

HAWQ service advisor shows wrong recommendations on edge cases


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 9baaa1c 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
0777dc6 
  ambari-server/src/main/resources/stacks/stack_advisor.py 437fe4f 
  ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
7ececa0 

Diff: https://reviews.apache.org/r/50731/diff/


Testing
---

Manually tested on live cluster.

Added unit tests
```
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Remove HAWQ Standby ... 2016-08-02 18:01:23,293 - Removing 
HAWQ Standby Master ...
ok
test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Resync HAWQ Standby ... 2016-08-02 18:01:23,295 - HAWQ 
Standby Master Re-Sync started in fast mode...
ok
test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
Kerberized ... 2016-08-02 18:01:23,298 - Executing HAWQ Check ...
ok
test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-08-02 18:01:23,304 - Executing HAWQ Check ...
ok
test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not 
Kerberized ... 2016-08-02 18:01:23,309 - Executing HAWQ Check ...
ok
test_run_hawq_check_case12 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 12: HDFS-HA, YARN Resource Management YARN_HA, 
Kerberized ... 2016-08-02 18:01:23,315 - Executing HAWQ Check ...
ok
test_run_hawq_check_case2 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 2: Non HDFS-HA, Standalone Resource Management, 
Kerberized ... 2016-08-02 18:01:23,320 - Executing HAWQ Check ...
ok
test_run_hawq_check_case3 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 3: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
Not Kerberized ... 2016-08-02 18:01:23,327 - Executing HAWQ Check ...
ok
test_run_hawq_check_case4 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 4: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-08-02 18:01:23,332 - Executing HAWQ Check ...
ok
test_run_hawq_check_case5 (test_hawqmaster.TestHawqMaster)

Re: Review Request 50731: HAWQ service advisor shows wrong recommendations on edge cases

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50731/
---

(Updated Aug. 3, 2016, 5:23 p.m.)


Review request for Ambari, Alexander Denissov and Lav Jain.


Changes
---

Review Feedback


Bugs: AMBARI-17996
https://issues.apache.org/jira/browse/AMBARI-17996


Repository: ambari


Description
---

HAWQ service advisor shows wrong recommendations on edge cases


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 9baaa1c 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
0777dc6 
  ambari-server/src/main/resources/stacks/stack_advisor.py 437fe4f 
  ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
7ececa0 

Diff: https://reviews.apache.org/r/50731/diff/


Testing
---

Manually tested on live cluster.

Added unit tests
```
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Remove HAWQ Standby ... 2016-08-02 18:01:23,293 - Removing 
HAWQ Standby Master ...
ok
test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
Run custom command Resync HAWQ Standby ... 2016-08-02 18:01:23,295 - HAWQ 
Standby Master Re-Sync started in fast mode...
ok
test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
Kerberized ... 2016-08-02 18:01:23,298 - Executing HAWQ Check ...
ok
test_run_hawq_check_case10 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 10: HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-08-02 18:01:23,304 - Executing HAWQ Check ...
ok
test_run_hawq_check_case11 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 11: HDFS-HA, YARN Resource Management YARN_HA, Not 
Kerberized ... 2016-08-02 18:01:23,309 - Executing HAWQ Check ...
ok
test_run_hawq_check_case12 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 12: HDFS-HA, YARN Resource Management YARN_HA, 
Kerberized ... 2016-08-02 18:01:23,315 - Executing HAWQ Check ...
ok
test_run_hawq_check_case2 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 2: Non HDFS-HA, Standalone Resource Management, 
Kerberized ... 2016-08-02 18:01:23,320 - Executing HAWQ Check ...
ok
test_run_hawq_check_case3 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 3: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
Not Kerberized ... 2016-08-02 18:01:23,327 - Executing HAWQ Check ...
ok
test_run_hawq_check_case4 (test_hawqmaster.TestHawqMaster)
Running HAWQ Check Case 4: Non HDFS-HA, YARN Resource Management Non YARN_HA, 
Kerberized ... 2016-08-02 18:01:23,332 - Executing HAWQ Check ...
ok

Re: Review Request 50731: HAWQ service advisor shows wrong recommendations on edge cases

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50731/
---

(Updated Aug. 3, 2016, 5:21 p.m.)


Review request for Ambari, Alexander Denissov and Lav Jain.


Changes
---

Review Feedback


Bugs: AMBARI-17996
https://issues.apache.org/jira/browse/AMBARI-17996


Repository: ambari


Description
---

HAWQ service advisor shows wrong recommendations on edge cases


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 0d6f2b2 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java
 ab43898 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 e8b0f15 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/AbstractKerberosDescriptorContainer.java
 39ebdaf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/KerberosIdentityDescriptor.java
 fb6cb97 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 6c0fb70 
  
ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationUtility.java
 2906100 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
 9efa4f9 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-solrconfig.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
 7e9c0f6 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 7064de3 
  
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/atlas-solrconfig.xml
 cba4a4e 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-atlas-application.properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
 9baaa1c 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 529c492 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py
 95c3dcc 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
0777dc6 
  
ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-audit.xml
 b4c0790 
  
ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-plugin-properties.xml
 3949402 
  
ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-policymgr-ssl.xml
 cf4a82e 
  
ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-security.xml
 91061d1 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/ranger-knox-plugin-properties.xml
 ae9314b 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
 44dc61a 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml
 ecb103c 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 eded4bd 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py
 50c1555 
  
ambari-server/src/main/resources/common-services/SPARK/1.5.2/configuration/spark-thrift-sparkconf.xml
 865aa73 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-audit.xml
 4dc51eb 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-policymgr-ssl.xml
 b1f6e1e 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-security.xml
 983702f 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml
 e504c2d 
  
ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
 faeac96 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
f80fa8a 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py
 bc8947a 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 b2c635d 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
 93c3aaa 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 da577ac 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-env.xml
 97283e0 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-site.xml
 b4fb928 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/ranger-hbase-plugin-properties.xml
 960c751 
  

Re: Review Request 50773: HiveHook fails to post messages to kafka due to missing keytab config in /etc/hive/conf/atlas-application.properties in kerberized cluster

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50773/#review144695
---


Ship it!




Ship It!

- Robert Levas


On Aug. 3, 2016, 7:46 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50773/
> ---
> 
> (Updated Aug. 3, 2016, 7:46 p.m.)
> 
> 
> Review request for Ambari, Robert Levas, Sumit Mohanty, and Swapan Shridhar.
> 
> 
> Bugs: AMBARI-18013
> https://issues.apache.org/jira/browse/AMBARI-18013
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> * Install Ambari 2.4
> * HDP 2.5 with Hive and Atlas
> * Kerberize the cluster
> 
> The hive hook fails because 2 configs are missing from 
> hive-atlas-application.properties, 
> 
> atlas.jaas.KafkaClient.option.keyTab=/etc/security/keytabs/hive.service.keytab
> atlas.jaas.KafkaClient.option.principal=hive/_h...@example.com
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/kerberos.json 
> 37a7e82 
> 
> Diff: https://reviews.apache.org/r/50773/diff/
> 
> 
> Testing
> ---
> 
> Python unit tests passed on these tese cases but failed due to a known issue 
> in test_nodemanager.py
> Verified on live cluster with Ambari 2.4, HDP 2.5, and Hive kerberized.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Review Request 50773: HiveHook fails to post messages to kafka due to missing keytab config in /etc/hive/conf/atlas-application.properties in kerberized cluster

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50773/
---

Review request for Ambari, Robert Levas, Sumit Mohanty, and Swapan Shridhar.


Bugs: AMBARI-18013
https://issues.apache.org/jira/browse/AMBARI-18013


Repository: ambari


Description
---

STR:
* Install Ambari 2.4
* HDP 2.5 with Hive and Atlas
* Kerberize the cluster

The hive hook fails because 2 configs are missing from 
hive-atlas-application.properties, 

atlas.jaas.KafkaClient.option.keyTab=/etc/security/keytabs/hive.service.keytab
atlas.jaas.KafkaClient.option.principal=hive/_h...@example.com


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/kerberos.json 
37a7e82 

Diff: https://reviews.apache.org/r/50773/diff/


Testing
---

Python unit tests passed on these tese cases but failed due to a known issue in 
test_nodemanager.py
Verified on live cluster with Ambari 2.4, HDP 2.5, and Hive kerberized.


Thanks,

Alejandro Fernandez

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Jameel Mohamed]

> On Aug. 3, 2016, 9:31 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java,
> >  line 40
> > 
> >
> > Aggre with Aravindan's comment, tableName should not be part of the 
> > interface.
> 
> Jameel Mohamed wrote:
> Fixed. Added this method to get the precision from table name.
> 
> /**
>* Get precision for a table
>* @param tableName
>* @return precision
>*/
>   private Precision getTablePrecision(String tableName) {
> Precision tablePrecision = null;
> switch (tableName) {
> case METRICS_RECORD_TABLE_NAME:
>   tablePrecision = Precision.SECONDS;
>   break;
> case METRICS_AGGREGATE_MINUTE_TABLE_NAME:
> case METRICS_CLUSTER_AGGREGATE_MINUTE_TABLE_NAME:
>   tablePrecision = Precision.MINUTES;
>   break;
> case METRICS_AGGREGATE_HOURLY_TABLE_NAME:
> case METRICS_CLUSTER_AGGREGATE_HOURLY_TABLE_NAME:
>   tablePrecision = Precision.HOURS;
>   break;
> case METRICS_AGGREGATE_DAILY_TABLE_NAME:
> case METRICS_CLUSTER_AGGREGATE_DAILY_TABLE_NAME:
>   tablePrecision = Precision.DAYS;
>   break;
> }
> return tablePrecision;
>   }

Is the above code looks ok to get the precision?


- Jameel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144673
---


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 48766: MySQL service status needs to be more robust

[Juanjo Marron]

> On June 16, 2016, 10:52 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py,
> >  line 63
> > 
> >
> > Else should go into for, not if (in case break didn't execute). 
> > 
> > Also (if code == 0) line is not needed since checked_call checks for 
> > non-zero return code itself and throws a reasonable exception in that case.
> 
> Andrew Onischuk wrote:
> Just to be easier to understand here is what I mean:
> 
> def get_mysql_pid_file():
>   cmd = 'mysqladmin variables'
>   code, output = shell.checked_call(cmd)
> 
>   for line in output.splitlines():
> if (line.find("pid_file") != -1):
>   mysql_pid_file  = line.split()[3].strip()
>   break
> else:
>   raise Fail(format("Output of '{cmd}' didn't contain pid_file. 
> Output: {output}"))  
>   return mysql_pid_file
> 
> Andrew Onischuk wrote:
> Juanjo sorry for bothering you so much with this issues. I just feel like 
> this is something which we should fix to be able debug problems in this place 
> more simply in future.
> 
> Juanjo  Marron wrote:
> No problem Andrew! I totally understand the process and it does not 
> bother me at all. These reviews the best way to find a good solution.
> 
> Correct me if I'm wrong, but I dont think this last solution is not  
> working since status action will fail to get the mysqld file. 
> Iterating line by line the mysqladmin variables output, which looks like 
> following: 
> 
> 
> +-+---+
> | Variable_name   | Value 
> |
> 
> +-+---+
> | auto_increment_increment| 1 
> |
> | auto_increment_offset   | 1 
> |
> | autocommit  | ON
> |
> | automatic_sp_privileges | ON
> |
> | back_log| 50
> |
> | basedir | /usr/ 
> |
> | big_tables  | OFF   
> |
> | binlog_cache_size   | 32768 
> |
> 
>  . 
>  
> | old_passwords   | OFF   
> |
> | open_files_limit| 1024  
> |
> | optimizer_prune_level   | 1 
> |
> | optimizer_search_depth  | 62
> |
> | optimizer_switch| 
> index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on
>  |
> | pid_file| /var/run/mysqld/mysqld.pid
> |
> | plugin_dir  | /usr/lib64/mysql/plugin   
> |
> | port| 3306  
> |
> | preload_buffer_size | 32768 
> |
> | profiling   | OFF   
> |
> 
> . 
> 
> if the first line does not include pid_file,  else will run and hence 
> raise the Fail exception.
> 
> Additionally, even if we solved this last issue with this proposal:
> 
> 

Re: Review Request 50653: AMBARI-17308 Ambari Logfeeder outputs a lot of errors due to parse date

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50653/#review144684
---



Thank you for the contribution.

Merged into trunk and branch-2.4:
trunk: bfa24d139aa9d10925850165702024e7a094e473 
branch-2.4: c41565852d93f1af6cf0278d752b839689948f15

please add "trunk, branch-2.4" to the "Branch:" tag in the review request.
Then you can close (submit) this review request.

- Oliver Szabo


On Aug. 3, 2016, 12:36 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50653/
> ---
> 
> (Updated Aug. 3, 2016, 12:36 p.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Dmytro Sen, Oliver Szabo, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-17308
> https://issues.apache.org/jira/browse/AMBARI-17308
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In logsearch_feeder service log, we got errors like below
> ```
> 2016-06-20 15:28:09,368 ERROR file=ambari-audit.log 
> org.apache.ambari.logfeeder.mapper.MapperDate LogFeederUtil.java:356 - Error 
> applying date transformation. isEpoch=false, 
> dateFormat=-MM-dd'T'HH:mm:ss.SSSZ, value=2016-06-20T15:28:08.000. 
> mapClass=map_date, input=input:source=file, 
> path=/var/log/ambari-server/ambari-audit.log, fieldName=logtime. Messages 
> suppressed before: 2
> java.text.ParseException: Unparseable date: "2016-06-20T15:28:08.000"
>   at java.text.DateFormat.parse(DateFormat.java:366)
>   at 
> org.apache.ambari.logfeeder.mapper.MapperDate.apply(MapperDate.java:83)
>   at org.apache.ambari.logfeeder.filter.Filter.apply(Filter.java:154)
>   at 
> org.apache.ambari.logfeeder.filter.FilterGrok.applyMessage(FilterGrok.java:291)
>   at 
> org.apache.ambari.logfeeder.filter.FilterGrok.flush(FilterGrok.java:320)
>   at org.apache.ambari.logfeeder.input.Input.flush(Input.java:125)
>   at 
> org.apache.ambari.logfeeder.input.InputFile.processFile(InputFile.java:430)
>   at org.apache.ambari.logfeeder.input.InputFile.start(InputFile.java:260)
>   at org.apache.ambari.logfeeder.input.Input.run(Input.java:100)
>   at java.lang.Thread.run(Thread.java:745)
> ```
> ambari-audit.log is like below
> ```
> 2016-07-21T01:52:49.875+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-2.5),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
> id(HDP-2.5), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu14/2.x/BUILDS/2.5.0.0-1025)
> 2016-07-21T01:52:49.905+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu16/repositories/HDP-2.5),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu16), Repo 
> id(HDP-2.5), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu16/2.x/BUILDS/2.5.0.0-1025)
> 2016-07-21T01:52:50.015+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-UTILS-1.1.0.21),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
> id(HDP-UTILS-1.1.0.21), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14)
> ```
> I think date format of the ambari-audit.log (2016-07-21T01:52:49.875+09) 
> should be like 2016-07-21T01:52:49.875+0900, since grok-pattern can't handle 
> 2016-07-21T01:52:49.875+09 format.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java
>  ea6524f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
>  5f751bf 
> 
> Diff: https://reviews.apache.org/r/50653/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Jameel Mohamed]

> On Aug. 3, 2016, 9:31 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java,
> >  line 40
> > 
> >
> > Aggre with Aravindan's comment, tableName should not be part of the 
> > interface.

Fixed. Added this method to get the precision from table name.

/**
   * Get precision for a table
   * @param tableName
   * @return precision
   */
  private Precision getTablePrecision(String tableName) {
Precision tablePrecision = null;
switch (tableName) {
case METRICS_RECORD_TABLE_NAME:
  tablePrecision = Precision.SECONDS;
  break;
case METRICS_AGGREGATE_MINUTE_TABLE_NAME:
case METRICS_CLUSTER_AGGREGATE_MINUTE_TABLE_NAME:
  tablePrecision = Precision.MINUTES;
  break;
case METRICS_AGGREGATE_HOURLY_TABLE_NAME:
case METRICS_CLUSTER_AGGREGATE_HOURLY_TABLE_NAME:
  tablePrecision = Precision.HOURS;
  break;
case METRICS_AGGREGATE_DAILY_TABLE_NAME:
case METRICS_CLUSTER_AGGREGATE_DAILY_TABLE_NAME:
  tablePrecision = Precision.DAYS;
  break;
}
return tablePrecision;
  }


- Jameel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144673
---


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Review Request 50450: API for bulk delete hostcomponent

[Ajit Kumar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50450/
---

Review request for Ambari, Alejandro Fernandez, Zhe (Joe) Wang, Nate Cole, and 
Sumit Mohanty.


Bugs: AMBARI-18011
https://issues.apache.org/jira/browse/AMBARI-18011


Repository: ambari


Description
---

API for bulk delete hostcomponent.
This should do best effort to delete hostcomponents. Current implementation 
fails fast in case of error.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/services/HostComponentService.java
 4990ad71c7dbd0d64ea7e817ace278aea8fe0f1d 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 5cf2de7b95939c9a55c5506988b2a6ba6801936c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 075b85a806fc5de8552f605a858ad153d7e485c6 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java
 df2b476ee030e8de88300e04ecaf08f46cbcfc4d 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 3ad1f1fc7702b60d832e3819829ffd71c30e08f3 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/HostComponentResourceProviderTest.java
 ef1b821ca05a563f0dbb3058877d335015bc6370 

Diff: https://reviews.apache.org/r/50450/diff/


Testing
---

Unit test
Manual Test
delete http://c6401.ambari.apache.org:8080/api/v1/clusters/c1/host_components 
-d 
'{"RequestInfo":{"query":"HostRoles/host_name.in(c6401.ambari.apache.org,c6402.ambari.apache.org)"}}'

Response : http://pastebin.com/GADF2ncH


Thanks,

Ajit Kumar

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Sid Wagle]

> On Aug. 3, 2016, 9:28 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java,
> >  line 33
> > 
> >
> > This look like a test calss. This should be checked into src/main/test 
> > instead.

Ignore this comment for some reason I thought this was part of source.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144671
---


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144673
---




ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
 (line 40)


Aggre with Aravindan's comment, tableName should not be part of the 
interface.


- Sid Wagle


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 50545: Ambari should not recursively chown for HAWQ hdfs upon every start

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50545/
---

(Updated Aug. 3, 2016, 9:28 p.m.)


Review request for Ambari, Alexander Denissov and Matt.


Bugs: AMBARI-17938
https://issues.apache.org/jira/browse/AMBARI-17938


Repository: ambari


Description
---

This results in changing of owner even if the owner value is same. The 
operation is very costly if there are a lot of subdirectories.
The owner value only changes when you switch from regular mode to secure mode 
and vice-versa.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 298513c 

Diff: https://reviews.apache.org/r/50545/diff/


Testing
---

In progress.


Thanks,

Lav Jain

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144671
---




ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
 (line 33)


This look like a test calss. This should be checked into src/main/test 
instead.


- Sid Wagle


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 50545: Ambari should not recursively chown for HAWQ hdfs upon every start

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50545/
---

(Updated Aug. 3, 2016, 9:18 p.m.)


Review request for Ambari, Alexander Denissov and Matt.


Bugs: AMBARI-17938
https://issues.apache.org/jira/browse/AMBARI-17938


Repository: ambari


Description
---

This results in changing of owner even if the owner value is same. The 
operation is very costly if there are a lot of subdirectories.
The owner value only changes when you switch from regular mode to secure mode 
and vice-versa.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 8c43555 

Diff: https://reviews.apache.org/r/50545/diff/


Testing
---

In progress.


Thanks,

Lav Jain

Re: Review Request 50047: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50047/#review144670
---


Ship it!




Ship It!

- Robert Levas


On Aug. 3, 2016, 4:47 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50047/
> ---
> 
> (Updated Aug. 3, 2016, 4:47 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, Nate 
> Cole, Robert Levas, and Sriharsha Chintalapani.
> 
> 
> Bugs: AMBARI-17694
> https://issues.apache.org/jira/browse/AMBARI-17694
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When kerberos is enabled, the protocol for listeners in 
> /etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
> even though the Ambari UI shows otherwise
> 
> 
> The patch was reverted due to a conflict, Based on Robert's comment there are 
> two versions of kerberos.json, 
> 
> …/stacks/HDP/2.5/services/KAFKA/kerberos.json
> …/resources/common-services/KAFKA/0.9.0/kerberos.json
> 
> The new version …/stacks/HDP/2.5/services/KAFKA/kerberos.json (AMBARI-17902) 
> has been added after the patch was committed for this review. I have included 
> the changes in the updated patch.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  66be3bf 
>   ambari-server/src/main/resources/common-services/KAFKA/0.10.0/kerberos.json 
> e1e6461 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
>  ac7b0ae 
>   ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
> 2b1c01b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
>  ee2a671 
> 
> Diff: https://reviews.apache.org/r/50047/diff/
> 
> 
> Testing
> ---
> 
> Added 1 new test case,
>  Ran mvn test
>  Tested in Ambari UI, by enabling kerberos, listeners protocol is updated and 
> kafka started successfully
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 50047: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50047/
---

(Updated Aug. 3, 2016, 8:47 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, Nate 
Cole, Robert Levas, and Sriharsha Chintalapani.


Bugs: AMBARI-17694
https://issues.apache.org/jira/browse/AMBARI-17694


Repository: ambari


Description (updated)
---

When kerberos is enabled, the protocol for listeners in 
/etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
even though the Ambari UI shows otherwise


The patch was reverted due to a conflict, Based on Robert's comment there are 
two versions of kerberos.json, 

…/stacks/HDP/2.5/services/KAFKA/kerberos.json
…/resources/common-services/KAFKA/0.9.0/kerberos.json

The new version …/stacks/HDP/2.5/services/KAFKA/kerberos.json (AMBARI-17902) 
has been added after the patch was committed for this review. I have included 
the changes in the updated patch.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 66be3bf 
  ambari-server/src/main/resources/common-services/KAFKA/0.10.0/kerberos.json 
e1e6461 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 ac7b0ae 
  ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
2b1c01b 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
 ee2a671 

Diff: https://reviews.apache.org/r/50047/diff/


Testing (updated)
---

Added 1 new test case,
 Ran mvn test
 Tested in Ambari UI, by enabling kerberos, listeners protocol is updated and 
kafka started successfully


Thanks,

Anita Jebaraj

Re: Review Request 50731: HAWQ service advisor shows wrong recommendations on edge cases

[Alexander Denissov]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50731/#review144642
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 167)


move these after if statement ?



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 173)


probably need to analyze if the changed configs are only for HAWQ service



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
(line 237)


might be an edge case arouns YARN RM-HA properties



ambari-server/src/main/resources/stacks/stack_advisor.py (line 900)


in names.keys() ?


- Alexander Denissov


On Aug. 3, 2016, 1:01 a.m., Matt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50731/
> ---
> 
> (Updated Aug. 3, 2016, 1:01 a.m.)
> 
> 
> Review request for Ambari, Alexander Denissov and Lav Jain.
> 
> 
> Bugs: AMBARI-17996
> https://issues.apache.org/jira/browse/AMBARI-17996
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HAWQ service advisor shows wrong recommendations on edge cases
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
>  9baaa1c 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
>  0777dc6 
>   ambari-server/src/main/resources/stacks/stack_advisor.py 437fe4f 
>   ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
> 7ececa0 
> 
> Diff: https://reviews.apache.org/r/50731/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster.
> 
> Added unit tests
> ```
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_remove_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Remove HAWQ Standby ... 2016-08-02 18:01:23,293 - Removing 
> HAWQ Standby Master ...
> ok
> test_resync_hawq_standby (test_hawqmaster.TestHawqMaster)
> Run custom command Resync HAWQ Standby ... 2016-08-02 18:01:23,295 - HAWQ 
> Standby Master Re-Sync started in fast mode...
> ok
> test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster)
> Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not 
> Kerberized ... 2016-08-02 18:01:23,298 - Executing HAWQ Check ...
> ok
> test_run_hawq_check_case10 

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Jameel Mohamed]

> On Aug. 3, 2016, 6:20 p.m., Aravindan Vijayan wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java,
> >  line 37
> > 
> >
> > Do the sink interface APIs need to take in a "tableName" ? Shouldn't 
> > the sink inteface just take in the data to be written, and decide what to 
> > do with the data in the implementation?

Reason for this is by just giving the data interface will not know what level 
of aggregates were performed. like minuter, hour or day.. So this is required.


- Jameel


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144637
---


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144637
---




ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
 (line 37)


Do the sink interface APIs need to take in a "tableName" ? Shouldn't the 
sink inteface just take in the data to be written, and decide what to do with 
the data in the implementation?


- Aravindan Vijayan


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Jameel Mohamed]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144636
---




ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 (line 1232)


Intention here is to have extension to write to different storage but not 
changing the default behavior. What you are say is to use this interface to 
write to HBase store. If we do that then user who want to write to additional 
storage apart from Hbase store (default) will have to make sure default 
scenario also work. This change will not alert the default behavior. Also 
better way to what you are asking is to make PhoenixHBaseAccessor Interface 
based implementation


- Jameel Mohamed


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 50666: AMBARI-10145 : Add support for tee to another Storage service?

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50666/#review144635
---




ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 (line 1232)


I guess this kind of implementation will work. But, can we have a design 
where the interface can be in such a way that AMS can write to any store. 
(PhoenixHBaseAccessor and HBaseTimelineMetric Store can be the default 
implemenation of those interfaces).


- Aravindan Vijayan


On Aug. 1, 2016, 10:11 p.m., Jameel Mohamed wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50666/
> ---
> 
> (Updated Aug. 1, 2016, 10:11 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-10145
> https://issues.apache.org/jira/browse/AMBARI-10145
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added new Interface to drain aggregated metrics to external storage. Added 
> unit test and sample implementation 
> (TimelineMetricsAggregatorMemorySink.java) to store metrics in memory.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  dfd730d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  e82d65a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  0ea668a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricsAggregatorMemorySink.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50666/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> 
> 
> Added unit test.
> 
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jameel Mohamed
> 
>

Re: Review Request 50756: RU Downgrade failure while downgrading from erie to DAL-m10

[Alejandro Fernandez]

> On Aug. 3, 2016, 5:41 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml, 
> > line 797
> > 
> >
> > This does not look like a correct fix.
> > 
> > EU/RU from HDP 2.3/2.4 to 2.5 is adding a new config type, 
> > sqoop-atlas-application.properties, so a downgrade must remove that new 
> > config type.
> > 
> > As a temporary solution to get EU working, I'm ok with this, but we 
> > need a Jira to remove newly added config types on downgrade.
> 
> Dmitro Lisnichenko wrote:
> The downgrade removes this config type. The trouble is that downgrade 
> still attempts to execute actions from pre-upgrade during downgrade. So 
> adding empty pre-downgrade section solves the issue. Also added null check 
> for sanity, but it is not strictly required for fixing this issue.
> 
> Here is a part of our javadoc:
> ```
> /**
>* Determine the list of tasks given these rules
>* 
>*   When performing an upgrade, only use upgrade tasks
>*   When performing a downgrade, use the downgrade tasks if they 
> are defined
>*   When performing a downgrade, but no downgrade tasks exist, 
> reuse the upgrade tasks
>* 
>* @param context the upgrade context
>* @param preTasks{@code true} if loading pre-upgrade or 
> pre-downgrade
>* @param pc  the processing component holding task definitions
>* @return A collection, potentially empty, of the tasks to run, which 
> may contain either
>* pre or post tasks if they exist, and the order depends on whether 
> it's an upgrade or downgrade.
>*/
>   protected List resolveTasks(final UpgradeContext context, boolean 
> preTasks, ProcessingComponent pc) {
> ```

So why does a downgrade read elements from a pre-upgrade?
I'm ok with this going in for now, but we should create a Jira so that we don't 
have to annotate this in the future.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50756/#review144629
---


On Aug. 3, 2016, 4:44 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50756/
> ---
> 
> (Updated Aug. 3, 2016, 4:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18008
> https://issues.apache.org/jira/browse/AMBARI-18008
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> RU Downgrade failure while downgrading from 2.5 to 2.3. Downgrade failed at 
> the step
> ' Client Components' > ' Updating configuration 
> sqoop-atlas-application.properties'.
> 
> 
> 1) Install HDP-2.3.2.0-2950 with ambari-server-2.1.2-377
> 2) upgrade ambari to 2.4.0.0-1054
> 3) Start stack upgrade to erie.
> 4) Before finalize step, start the downgrade.
> The downgrade is failing at above mentioned step.
> 
> {code}
> 02 Aug 2016 05:27:58,527  WARN [Server Action Executor Worker 4525] 
> ServerActionExecutor:497 - Task #4525 failed to complete execution due to 
> thrown exception: java.lang.NullPointerException:null
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.serveraction.upgrades.ConfigureAction.execute(ConfigureAction.java:239)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:555)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:492)
> at java.lang.Thread.run(Thread.java:745){code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
>  ab6614d 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 6390f1f 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> e7bf162 
> 
> Diff: https://reviews.apache.org/r/50756/diff/
> 
> 
> Testing
> ---
> 
> on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 50756: RU Downgrade failure while downgrading from erie to DAL-m10

[Dmitro Lisnichenko]

> On Aug. 3, 2016, 8:41 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml, 
> > line 797
> > 
> >
> > This does not look like a correct fix.
> > 
> > EU/RU from HDP 2.3/2.4 to 2.5 is adding a new config type, 
> > sqoop-atlas-application.properties, so a downgrade must remove that new 
> > config type.
> > 
> > As a temporary solution to get EU working, I'm ok with this, but we 
> > need a Jira to remove newly added config types on downgrade.

The downgrade removes this config type. The trouble is that downgrade still 
attempts to execute actions from pre-upgrade during downgrade. So adding empty 
pre-downgrade section solves the issue. Also added null check for sanity, but 
it is not strictly required for fixing this issue.

Here is a part of our javadoc:
```
/**
   * Determine the list of tasks given these rules
   * 
   *   When performing an upgrade, only use upgrade tasks
   *   When performing a downgrade, use the downgrade tasks if they are 
defined
   *   When performing a downgrade, but no downgrade tasks exist, reuse the 
upgrade tasks
   * 
   * @param context the upgrade context
   * @param preTasks{@code true} if loading pre-upgrade or pre-downgrade
   * @param pc  the processing component holding task definitions
   * @return A collection, potentially empty, of the tasks to run, which may 
contain either
   * pre or post tasks if they exist, and the order depends on whether it's an 
upgrade or downgrade.
   */
  protected List resolveTasks(final UpgradeContext context, boolean 
preTasks, ProcessingComponent pc) {
```


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50756/#review144629
---


On Aug. 3, 2016, 7:44 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50756/
> ---
> 
> (Updated Aug. 3, 2016, 7:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18008
> https://issues.apache.org/jira/browse/AMBARI-18008
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> RU Downgrade failure while downgrading from 2.5 to 2.3. Downgrade failed at 
> the step
> ' Client Components' > ' Updating configuration 
> sqoop-atlas-application.properties'.
> 
> 
> 1) Install HDP-2.3.2.0-2950 with ambari-server-2.1.2-377
> 2) upgrade ambari to 2.4.0.0-1054
> 3) Start stack upgrade to erie.
> 4) Before finalize step, start the downgrade.
> The downgrade is failing at above mentioned step.
> 
> {code}
> 02 Aug 2016 05:27:58,527  WARN [Server Action Executor Worker 4525] 
> ServerActionExecutor:497 - Task #4525 failed to complete execution due to 
> thrown exception: java.lang.NullPointerException:null
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.serveraction.upgrades.ConfigureAction.execute(ConfigureAction.java:239)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:555)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:492)
> at java.lang.Thread.run(Thread.java:745){code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
>  ab6614d 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 6390f1f 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> e7bf162 
> 
> Diff: https://reviews.apache.org/r/50756/diff/
> 
> 
> Testing
> ---
> 
> on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 50756: RU Downgrade failure while downgrading from erie to DAL-m10

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50756/#review144629
---




ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml (line 
797)


This does not look like a correct fix.

EU/RU from HDP 2.3/2.4 to 2.5 is adding a new config type, 
sqoop-atlas-application.properties, so a downgrade must remove that new config 
type.

As a temporary solution to get EU working, I'm ok with this, but we need a 
Jira to remove newly added config types on downgrade.


- Alejandro Fernandez


On Aug. 3, 2016, 4:44 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50756/
> ---
> 
> (Updated Aug. 3, 2016, 4:44 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18008
> https://issues.apache.org/jira/browse/AMBARI-18008
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> RU Downgrade failure while downgrading from 2.5 to 2.3. Downgrade failed at 
> the step
> ' Client Components' > ' Updating configuration 
> sqoop-atlas-application.properties'.
> 
> 
> 1) Install HDP-2.3.2.0-2950 with ambari-server-2.1.2-377
> 2) upgrade ambari to 2.4.0.0-1054
> 3) Start stack upgrade to erie.
> 4) Before finalize step, start the downgrade.
> The downgrade is failing at above mentioned step.
> 
> {code}
> 02 Aug 2016 05:27:58,527  WARN [Server Action Executor Worker 4525] 
> ServerActionExecutor:497 - Task #4525 failed to complete execution due to 
> thrown exception: java.lang.NullPointerException:null
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.serveraction.upgrades.ConfigureAction.execute(ConfigureAction.java:239)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:555)
> at 
> org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:492)
> at java.lang.Thread.run(Thread.java:745){code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
>  ab6614d 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 6390f1f 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> e7bf162 
> 
> Diff: https://reviews.apache.org/r/50756/diff/
> 
> 
> Testing
> ---
> 
> on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 50594: Fix Spark hdp.version issues in upgrading and fresh install

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50594/#review144625
---


Ship it!




Ship It!

- Jayush Luniya


On Aug. 2, 2016, 8:37 a.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50594/
> ---
> 
> (Updated Aug. 2, 2016, 8:37 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17954
> https://issues.apache.org/jira/browse/AMBARI-17954
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix Spark hdp.version issues in upgrading and fresh install
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  6537e43 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml
>  646239e 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
>  86aeb70 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py
>  2b5d36b 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.5.2/configuration/spark-thrift-sparkconf.xml
>  e27c986 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  b297fa6 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> b295cc9 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
>  5b20a72 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  e046849 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml 
> c9f9823 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 6390f1f 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SPARK/configuration/spark-defaults.xml
>  b3d05d8 
>   ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 
> b2ed5df 
>   ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py ed331ea 
>   ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py 
> a1abdfa 
> 
> Diff: https://reviews.apache.org/r/50594/diff/
> 
> 
> Testing
> ---
> 
> Manual test with different scenarios:
> 
> 1. Fresh install of HDP 2.3.6, 2.4.3, 2.5.0
> 2. Upgrade for 2.3.6 to 2.5.0.
> 3. Downgrade from 2.5.0 to 2.3.6.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>

Re: Review Request 50710: Ambari agent unable to register with server when server response is too big

[Dmytro Sen]

> On Авг. 2, 2016, 3:35 п.п., Sid Wagle wrote:
> > ambari-server/pom.xml, line 1117
> > 
> >
> > Where is this being used ?

I think it's a views dependency. It's been there before replacing jetty 8 with 
jetty 9


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50710/#review144497
---


On Авг. 3, 2016, 4:44 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50710/
> ---
> 
> (Updated Авг. 3, 2016, 4:44 п.п.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17991
> https://issues.apache.org/jira/browse/AMBARI-17991
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Use Jetty 8 instead of 9 in Ambari 2.4.0
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/security.py 72ae954 
>   ambari-funtest/pom.xml ebc34ec 
>   ambari-logsearch/ambari-logsearch-portal/pom.xml 0a24759 
>   ambari-project/pom.xml c7a0a6a 
>   ambari-server/pom.xml 814973b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/AmbariErrorHandler.java
>  373face 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  bd96868 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
>  44cdf2b 
>   ambari-server/src/main/java/org/apache/ambari/server/utils/JsonUtils.java 
> 63c88d8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/api/AmbariErrorHandlerTest.java
>  30c22a2 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/request/DefaultEventCreatorTest.java
>  1e063ec 
> 
> Diff: https://reviews.apache.org/r/50710/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 50756: RU Downgrade failure while downgrading from erie to DAL-m10

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50756/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-18008
https://issues.apache.org/jira/browse/AMBARI-18008


Repository: ambari


Description
---

RU Downgrade failure while downgrading from 2.5 to 2.3. Downgrade failed at the 
step
' Client Components' > ' Updating configuration 
sqoop-atlas-application.properties'.


1) Install HDP-2.3.2.0-2950 with ambari-server-2.1.2-377
2) upgrade ambari to 2.4.0.0-1054
3) Start stack upgrade to erie.
4) Before finalize step, start the downgrade.
The downgrade is failing at above mentioned step.

{code}
02 Aug 2016 05:27:58,527  WARN [Server Action Executor Worker 4525] 
ServerActionExecutor:497 - Task #4525 failed to complete execution due to 
thrown exception: java.lang.NullPointerException:null
java.lang.NullPointerException
at 
org.apache.ambari.server.serveraction.upgrades.ConfigureAction.execute(ConfigureAction.java:239)
at 
org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.execute(ServerActionExecutor.java:555)
at 
org.apache.ambari.server.serveraction.ServerActionExecutor$Worker.run(ServerActionExecutor.java:492)
at java.lang.Thread.run(Thread.java:745){code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/ConfigureAction.java
 ab6614d 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
6390f1f 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
e7bf162 

Diff: https://reviews.apache.org/r/50756/diff/


Testing
---

on live cluster


Thanks,

Dmitro Lisnichenko

Re: Review Request 50710: Ambari agent unable to register with server when server response is too big

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50710/
---

(Updated Авг. 3, 2016, 4:44 п.п.)


Review request for Ambari, Jaimin Jetly, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-17991
https://issues.apache.org/jira/browse/AMBARI-17991


Repository: ambari


Description
---

Use Jetty 8 instead of 9 in Ambari 2.4.0


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/security.py 72ae954 
  ambari-funtest/pom.xml ebc34ec 
  ambari-logsearch/ambari-logsearch-portal/pom.xml 0a24759 
  ambari-project/pom.xml c7a0a6a 
  ambari-server/pom.xml 814973b 
  
ambari-server/src/main/java/org/apache/ambari/server/api/AmbariErrorHandler.java
 373face 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 bd96868 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
 44cdf2b 
  ambari-server/src/main/java/org/apache/ambari/server/utils/JsonUtils.java 
63c88d8 
  
ambari-server/src/test/java/org/apache/ambari/server/api/AmbariErrorHandlerTest.java
 30c22a2 
  
ambari-server/src/test/java/org/apache/ambari/server/audit/request/DefaultEventCreatorTest.java
 1e063ec 

Diff: https://reviews.apache.org/r/50710/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 50753: Cleared cached resources from ambari-server on host removal

[Sandor Magyari]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50753/#review144619
---


Ship it!




Ship It!

- Sandor Magyari


On Aug. 3, 2016, 3:17 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50753/
> ---
> 
> (Updated Aug. 3, 2016, 3:17 p.m.)
> 
> 
> Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18005
> https://issues.apache.org/jira/browse/AMBARI-18005
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a host is removed from the cluster and later from ambari there's a 
> chance the agent registers back to the ambari server before the agent is 
> stopped.
> Stopping the machine running the agent without the host being deleted again 
> leads to an inconsistent state in the ambari-server due to cached state.
> Resolution:
> The cached resources get cleared on host delete event.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> a757010 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  0190478 
> 
> Diff: https://reviews.apache.org/r/50753/diff/
> 
> 
> Testing
> ---
> 
> Manually, in progress.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 50753: Cleared cached resources from ambari-server on host removal

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50753/
---

(Updated Aug. 3, 2016, 3:17 p.m.)


Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
Toader.


Changes
---

Applied review notes.


Bugs: AMBARI-18005
https://issues.apache.org/jira/browse/AMBARI-18005


Repository: ambari


Description
---

When a host is removed from the cluster and later from ambari there's a chance 
the agent registers back to the ambari server before the agent is stopped.
Stopping the machine running the agent without the host being deleted again 
leads to an inconsistent state in the ambari-server due to cached state.
Resolution:
The cached resources get cleared on host delete event.


Diffs (updated)
-

  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
a757010 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 0190478 

Diff: https://reviews.apache.org/r/50753/diff/


Testing
---

Manually, in progress.


Thanks,

Laszlo Puskas

Re: Review Request 50753: Cleared cached resources from ambari-server on host removal

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50753/#review144618
---


Ship it!




Ship It!

- Robert Nettleton


On Aug. 3, 2016, 2:47 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50753/
> ---
> 
> (Updated Aug. 3, 2016, 2:47 p.m.)
> 
> 
> Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18005
> https://issues.apache.org/jira/browse/AMBARI-18005
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a host is removed from the cluster and later from ambari there's a 
> chance the agent registers back to the ambari server before the agent is 
> stopped.
> Stopping the machine running the agent without the host being deleted again 
> leads to an inconsistent state in the ambari-server due to cached state.
> Resolution:
> The cached resources get cleared on host delete event.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> a757010 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  0190478 
> 
> Diff: https://reviews.apache.org/r/50753/diff/
> 
> 
> Testing
> ---
> 
> Manually, in progress.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 50753: Cleared cached resources from ambari-server on host removal

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50753/#review144617
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 (line 974)


add some javadoc



ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 (line 990)


Let's make the message more explicit that the host is removed from 
available hosts.


- Sebastian Toader


On Aug. 3, 2016, 4:47 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50753/
> ---
> 
> (Updated Aug. 3, 2016, 4:47 p.m.)
> 
> 
> Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-18005
> https://issues.apache.org/jira/browse/AMBARI-18005
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a host is removed from the cluster and later from ambari there's a 
> chance the agent registers back to the ambari server before the agent is 
> stopped.
> Stopping the machine running the agent without the host being deleted again 
> leads to an inconsistent state in the ambari-server due to cached state.
> Resolution:
> The cached resources get cleared on host delete event.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> a757010 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  0190478 
> 
> Diff: https://reviews.apache.org/r/50753/diff/
> 
> 
> Testing
> ---
> 
> Manually, in progress.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Review Request 50753: Cleared cached resources from ambari-server on host removal

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50753/
---

Review request for Ambari, Robert Nettleton, Sandor Magyari, and Sebastian 
Toader.


Bugs: AMBARI-18005
https://issues.apache.org/jira/browse/AMBARI-18005


Repository: ambari


Description
---

When a host is removed from the cluster and later from ambari there's a chance 
the agent registers back to the ambari server before the agent is stopped.
Stopping the machine running the agent without the host being deleted again 
leads to an inconsistent state in the ambari-server due to cached state.
Resolution:
The cached resources get cleared on host delete event.


Diffs
-

  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
a757010 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 0190478 

Diff: https://reviews.apache.org/r/50753/diff/


Testing
---

Manually, in progress.


Thanks,

Laszlo Puskas

Re: Review Request 50752: Tune HDFS opts parameters to trigger GC more predicitably

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50752/#review144615
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Aug. 3, 2016, 5:34 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50752/
> ---
> 
> (Updated Aug. 3, 2016, 5:34 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18006
> https://issues.apache.org/jira/browse/AMBARI-18006
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> DN Heap usage alert is set to 80% warning and 90% critical. This alert is
> causing a lot of alerts to be fired off and needs to be tuned.
> 
> One thing we need to make sure is that DN gc is happening. I am not sure at
> what % of heap usage will the gc be fired but we need to make sure our alerts
> are for a % higher than that as when i manually ran a GC the heap usage went
> below the threshold.
> 
> Based on discussions with hdfs dev's it was determined we should be adding the
> following
> 
> 
> 
> 
> XX:+UseCMSInitiatingOccupancyOnly and 
> -XX:CMSInitiatingOccupancyFraction=
> 
> 
> to the namenode and datanode opts. Based on this setting hopefully we can also
> determine what % to set the alerts to.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml
>  40483b8 
> 
> Diff: https://reviews.apache.org/r/50752/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 50752: Tune HDFS opts parameters to trigger GC more predicitably

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50752/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18006
https://issues.apache.org/jira/browse/AMBARI-18006


Repository: ambari


Description
---

DN Heap usage alert is set to 80% warning and 90% critical. This alert is
causing a lot of alerts to be fired off and needs to be tuned.

One thing we need to make sure is that DN gc is happening. I am not sure at
what % of heap usage will the gc be fired but we need to make sure our alerts
are for a % higher than that as when i manually ran a GC the heap usage went
below the threshold.

Based on discussions with hdfs dev's it was determined we should be adding the
following




XX:+UseCMSInitiatingOccupancyOnly and 
-XX:CMSInitiatingOccupancyFraction=


to the namenode and datanode opts. Based on this setting hopefully we can also
determine what % to set the alerts to.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml
 40483b8 

Diff: https://reviews.apache.org/r/50752/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 50742: operationStatus and taskStatus audit log should contain remoteIp message

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50742/#review144605
---




ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
 (lines 47 - 57)


Can you provide a description of the use case this patch implements?


- Sebastian Toader


On Aug. 3, 2016, 12:04 p.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50742/
> ---
> 
> (Updated Aug. 3, 2016, 12:04 p.m.)
> 
> 
> Review request for Ambari, Laszlo Puskas, Oliver Szabo, Sandor Magyari, and 
> Sebastian Toader.
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> OperationStatusAuditEvent and TaskStatusAuditEvent auditlog message should 
> contain the remoteip info, store in the pg table host_role_command
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
>  b44dc78 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
>  ff2ce92 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/OperationStatusAuditEvent.java
>  fd0068e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/TaskStatusAuditEvent.java
>  1682e74 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
>  6288091 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 0843616 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 43bdef9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/OperationStatusAuditEventTest.java
>  038cebc 
>   
> ambari-server/src/test/java/org/apache/ambari/server/audit/TaskStatusAuditEventTest.java
>  8f769c7 
> 
> Diff: https://reviews.apache.org/r/50742/diff/
> 
> 
> Testing
> ---
> 
> done
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 50653: AMBARI-17308 Ambari Logfeeder outputs a lot of errors due to parse date

[Masahiro Tanaka]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50653/
---

(Updated Aug. 3, 2016, 9:36 p.m.)


Review request for Ambari, Daniel Gergely, Dmytro Sen, Oliver Szabo, and 
Sebastian Toader.


Changes
---

Updated the patch


Bugs: AMBARI-17308
https://issues.apache.org/jira/browse/AMBARI-17308


Repository: ambari


Description
---

In logsearch_feeder service log, we got errors like below
```
2016-06-20 15:28:09,368 ERROR file=ambari-audit.log 
org.apache.ambari.logfeeder.mapper.MapperDate LogFeederUtil.java:356 - Error 
applying date transformation. isEpoch=false, 
dateFormat=-MM-dd'T'HH:mm:ss.SSSZ, value=2016-06-20T15:28:08.000. 
mapClass=map_date, input=input:source=file, 
path=/var/log/ambari-server/ambari-audit.log, fieldName=logtime. Messages 
suppressed before: 2
java.text.ParseException: Unparseable date: "2016-06-20T15:28:08.000"
at java.text.DateFormat.parse(DateFormat.java:366)
at 
org.apache.ambari.logfeeder.mapper.MapperDate.apply(MapperDate.java:83)
at org.apache.ambari.logfeeder.filter.Filter.apply(Filter.java:154)
at 
org.apache.ambari.logfeeder.filter.FilterGrok.applyMessage(FilterGrok.java:291)
at 
org.apache.ambari.logfeeder.filter.FilterGrok.flush(FilterGrok.java:320)
at org.apache.ambari.logfeeder.input.Input.flush(Input.java:125)
at 
org.apache.ambari.logfeeder.input.InputFile.processFile(InputFile.java:430)
at org.apache.ambari.logfeeder.input.InputFile.start(InputFile.java:260)
at org.apache.ambari.logfeeder.input.Input.run(Input.java:100)
at java.lang.Thread.run(Thread.java:745)
```
ambari-audit.log is like below
```
2016-07-21T01:52:49.875+09, User(admin), RemoteIp(192.168.72.1), 
Operation(Repository update), RequestType(PUT), 
url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-2.5),
 ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
id(HDP-2.5), Base 
URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu14/2.x/BUILDS/2.5.0.0-1025)
2016-07-21T01:52:49.905+09, User(admin), RemoteIp(192.168.72.1), 
Operation(Repository update), RequestType(PUT), 
url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu16/repositories/HDP-2.5),
 ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu16), Repo 
id(HDP-2.5), Base 
URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu16/2.x/BUILDS/2.5.0.0-1025)
2016-07-21T01:52:50.015+09, User(admin), RemoteIp(192.168.72.1), 
Operation(Repository update), RequestType(PUT), 
url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-UTILS-1.1.0.21),
 ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
id(HDP-UTILS-1.1.0.21), Base 
URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14)
```
I think date format of the ambari-audit.log (2016-07-21T01:52:49.875+09) should 
be like 2016-07-21T01:52:49.875+0900, since grok-pattern can't handle 
2016-07-21T01:52:49.875+09 format.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java
 ea6524f 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
 5f751bf 

Diff: https://reviews.apache.org/r/50653/diff/


Testing
---

mvn clean test


Thanks,

Masahiro Tanaka

Re: Review Request 50725: Kerberos identity definitions in Kerberos descriptors should explicitly declare a reference

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50725/#review144600
---


Ship it!




Ship It!

- Oliver Szabo


On Aug. 3, 2016, 12:57 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50725/
> ---
> 
> (Updated Aug. 3, 2016, 12:57 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Oliver Szabo.
> 
> 
> Bugs: AMBARI-17993
> https://issues.apache.org/jira/browse/AMBARI-17993
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Kerberos identity definitions in Kerberos descriptors should explicitly 
> declare a reference rather than rely on the identity's _name_ attribute. 
> 
> Currently, the set of Kerberos identities declared at a service-level or a 
> component-level can contain identities with unique names.  For example using:
> 
> ```
> "identities": [
> {
>   "name": "identity",
>   "principal": {
> "value": "service/_HOST@${realm}",
> "configuration": "service-site/property1.principal",
> ...
>   },
>   "keytab": {
> "file": "${keytab_dir}/service.service.keytab",
> "configuration": "service-site/property1.keytab",
> ...
>   }
> },
> {
>   "name": "identity",
>   "principal": {
> "value": "service/_HOST@${realm}",
> "configuration": "service-site/property2.principal",
> ...
>   },
>   "keytab": {
> "file": "${keytab_dir}/service.service.keytab",
> "configuration": "service-site/property2.keytab",
> ...
>   }
> }
>   ]
> ```
> 
> Only the first "identity" principal is realized and the additional one is 
> ignored, leaving the configurations {{service-site/property2.principal}} and 
> {{service-site/property2.keytab}} untouched when Kerberos is enabled for the 
> service. 
> 
> To help this, the 2nd instance can be converted to a reference, overriding 
> only the attributes the need to be changed - like the configurations. 
> 
> ```
> "identities": [
> {
>   "name": "identity",
>   "principal": {
> "value": "service/_HOST@${realm}",
> "configuration": "service-site/property1.principal",
> ...
>   },
>   "keytab": {
> "file": "${keytab_dir}/service.service.keytab",
> "configuration": "service-site/property1.keytab",
> ...
>   }
> },
> {
>   "name": "/SERVICE/identity",
>   "principal": {
> "configuration": "service-site/property2.principal"
>   },
>   "keytab": {
> "configuration": "service-site/property2.keytab"
>   }
> }
>   ]
> ```
> 
> This allows for both identity declarations to be realized, however this is 
> limited to only the 2 instances. If a 3rd instance is needed (to set an 
> additional configuration), it must look be:
> 
> ```
> {
>   "name": "/SERVICE/identity",
>   "principal": {
> "configuration": "service-site/property3.principal"
>   },
>   "keytab": {
> "configuration": "service-site/property3.keytab"
>   }
> }
> ```
> 
> However since it's name is the same as the 2nd instance, it will be ignored. 
> 
> If explicit references are specified, then multiple uniquely-named identity 
> blocks will be allowed to reference the same base identity, effectively 
> enabling the ability to declare unlimited configurations for the same 
> identity definition:
> 
> ```
> "identities": [
> {
>   "name": "identity",
>   "principal": {
> "value": "service/_HOST@${realm}",
> "configuration": "service-site/property1.principal",
> ...
>   },
>   "keytab": {
> "file": "${keytab_dir}/service.service.keytab",
> "configuration": "service-site/property1.keytab",
> ...
>   }
> },
> {
>   "name": "identitiy_reference1",
>   "reference": "/SERVICE/identity",
>   "principal": {
> "configuration": "service-site/property2.principal"
>   },
>   "keytab": {
> "configuration": "service-site/property2.keytab"
>   }
> },
> {
>   "name": "identitiy_reference2",
>   "reference": "/SERVICE/identity",
>   "principal": {
> "configuration": "service-site/property3.principal"
>   },
>   "keytab": {
> "configuration": "service-site/property3.keytab"
>   }
> }
>   ]
> ```
> 
> NOTE: Backwards compatibility must be maintained when implementing this as to 
> not break existing Kerberos descriptors. So identity block names the look 
> like paths are to continue to be treated as references.
> 
> 
> Diffs
> -
> 
>   
> 

Re: Review Request 50735: Data can not be migrated from Hive 1.0 to Hive2

[Pallav Kulshreshtha]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50735/#review144597
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On Aug. 3, 2016, 4:55 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50735/
> ---
> 
> (Updated Aug. 3, 2016, 4:55 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, and Rohit Choudhary.
> 
> 
> Bugs: AMBARI-18000
> https://issues.apache.org/jira/browse/AMBARI-18000
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added DataMigrator class for mapping entities between versions.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationUtility.java
>  6f427cc 
>   
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/DataMigrator.java
>  PRE-CREATION 
>   contrib/views/hive-next/src/main/resources/view.xml 52fe3ec 
> 
> Diff: https://reviews.apache.org/r/50735/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>

Review Request 50742: operationStatus and taskStatus audit log should contain remoteIp message

[wang yaoxin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50742/
---

Review request for Ambari, Laszlo Puskas, Oliver Szabo, Sandor Magyari, and 
Sebastian Toader.


Repository: ambari


Description
---

OperationStatusAuditEvent and TaskStatusAuditEvent auditlog message should 
contain the remoteip info, store in the pg table host_role_command


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionDBAccessorImpl.java
 b44dc78 
  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java
 ff2ce92 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/event/OperationStatusAuditEvent.java
 fd0068e 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/event/TaskStatusAuditEvent.java
 1682e74 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java
 6288091 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 0843616 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
43bdef9 
  
ambari-server/src/test/java/org/apache/ambari/server/audit/OperationStatusAuditEventTest.java
 038cebc 
  
ambari-server/src/test/java/org/apache/ambari/server/audit/TaskStatusAuditEventTest.java
 8f769c7 

Diff: https://reviews.apache.org/r/50742/diff/


Testing
---

done


Thanks,

wang yaoxin

Re: Review Request 50595: AMBARI-17955: Enabled download of berkeley DB jar file to be provisioned during install

[Vitalyi Brodetskyi]

> On Сер. 2, 2016, 8:58 до полудня, Vitalyi Brodetskyi wrote:
> > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py,
> >  lines 200-217
> > 
> >
> > I don't see here any condition, is it default db for falcon and user 
> > always should registrate/use jdbc for it?
> 
> Vitalyi Brodetskyi wrote:
> And one more, as i remember we set some special permissions for jdbc 
> which was downloaded because later you could have troubles with customized 
> users/permissions/non root user.
> 
> Venkat Ranganathan wrote:
> Yes.  This is only one and it is really not a JDBC driver.  We just use 
> the download mechanism

Ok, but anyway i think it will be good to set correct permissions to prevent 
related issues.


- Vitalyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50595/#review144468
---


On Сер. 1, 2016, 9:21 після полудня, Venkat Ranganathan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50595/
> ---
> 
> (Updated Сер. 1, 2016, 9:21 після полудня)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-17955
> https://issues.apache.org/jira/browse/AMBARI-17955
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enabled the registration of optionally downloaded berkeley db jar file.   
> Introduce Berekley DB as a jdbc-db (even though it is actually not.   
> Download je-5.0.73.jar if not already present and inform users on how to 
> register
> 
> also, fixed Atlas integration changes with classname and packagename
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/python/ambari_server/dbConfiguration.py fc68a4c 
>   ambari-server/src/main/python/ambari_server/dbConfiguration_linux.py 
> d09bd77 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  dd1cda0 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
>  ebe3634 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> f8ba3e9 
>   ambari-server/src/test/python/TestAmbariServer.py 279a172 
>   ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 
> 2dd0da9 
> 
> Diff: https://reviews.apache.org/r/50595/diff/
> 
> 
> Testing
> ---
> 
> Tested downloading of the jar file by registering with ambari server
> Tested Falcon/Atlas component names
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>

Re: Review Request 50653: AMBARI-17308 Ambari Logfeeder outputs a lot of errors due to parse date

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50653/#review144594
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
 (line 245)


I think we can use XX here too, just keep it consistent


- Oliver Szabo


On Aug. 3, 2016, 1:15 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50653/
> ---
> 
> (Updated Aug. 3, 2016, 1:15 a.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Dmytro Sen, Oliver Szabo, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-17308
> https://issues.apache.org/jira/browse/AMBARI-17308
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In logsearch_feeder service log, we got errors like below
> ```
> 2016-06-20 15:28:09,368 ERROR file=ambari-audit.log 
> org.apache.ambari.logfeeder.mapper.MapperDate LogFeederUtil.java:356 - Error 
> applying date transformation. isEpoch=false, 
> dateFormat=-MM-dd'T'HH:mm:ss.SSSZ, value=2016-06-20T15:28:08.000. 
> mapClass=map_date, input=input:source=file, 
> path=/var/log/ambari-server/ambari-audit.log, fieldName=logtime. Messages 
> suppressed before: 2
> java.text.ParseException: Unparseable date: "2016-06-20T15:28:08.000"
>   at java.text.DateFormat.parse(DateFormat.java:366)
>   at 
> org.apache.ambari.logfeeder.mapper.MapperDate.apply(MapperDate.java:83)
>   at org.apache.ambari.logfeeder.filter.Filter.apply(Filter.java:154)
>   at 
> org.apache.ambari.logfeeder.filter.FilterGrok.applyMessage(FilterGrok.java:291)
>   at 
> org.apache.ambari.logfeeder.filter.FilterGrok.flush(FilterGrok.java:320)
>   at org.apache.ambari.logfeeder.input.Input.flush(Input.java:125)
>   at 
> org.apache.ambari.logfeeder.input.InputFile.processFile(InputFile.java:430)
>   at org.apache.ambari.logfeeder.input.InputFile.start(InputFile.java:260)
>   at org.apache.ambari.logfeeder.input.Input.run(Input.java:100)
>   at java.lang.Thread.run(Thread.java:745)
> ```
> ambari-audit.log is like below
> ```
> 2016-07-21T01:52:49.875+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-2.5),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
> id(HDP-2.5), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu14/2.x/BUILDS/2.5.0.0-1025)
> 2016-07-21T01:52:49.905+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu16/repositories/HDP-2.5),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu16), Repo 
> id(HDP-2.5), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu16/2.x/BUILDS/2.5.0.0-1025)
> 2016-07-21T01:52:50.015+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-UTILS-1.1.0.21),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
> id(HDP-UTILS-1.1.0.21), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14)
> ```
> I think date format of the ambari-audit.log (2016-07-21T01:52:49.875+09) 
> should be like 2016-07-21T01:52:49.875+0900, since grok-pattern can't handle 
> 2016-07-21T01:52:49.875+09 format.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java
>  ea6524f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
>  5f751bf 
> 
> Diff: https://reviews.apache.org/r/50653/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 50645: Ambari upgrade to 2.4.0 adds several config properties that are not necessary and some are incorrect

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50645/#review144592
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Aug. 3, 2016, 4:44 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50645/
> ---
> 
> (Updated Aug. 3, 2016, 4:44 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, Jaimin 
> Jetly, and Mugdha Varadkar.
> 
> 
> Bugs: AMBARI-17971
> https://issues.apache.org/jira/browse/AMBARI-17971
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari upgrade to 2.4.0 adds several config properties that are not necessary 
> and some are incorrect.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  108c7b4 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-log4j.xml
>  a071442 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/atlas-solrconfig.xml
>  35e9f58 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py
>  1a0e797 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  8006158 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/atlas-solrconfig.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-atlas-application.properties.xml
>  e653651 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-audit.xml
>  5257549 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-plugin-properties.xml
>  74cdf63 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-policymgr-ssl.xml
>  f0fc160 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/ranger-kafka-security.xml
>  3967e3e 
>   
> ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/configuration/ranger-knox-plugin-properties.xml
>  f0f6f41 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  f7823d2 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-audit.xml
>  b7cf4c5 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-policymgr-ssl.xml
>  9592914 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/ranger-storm-security.xml
>  5f0e944 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-site.xml
>  ab7d9cb 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
>  9ba58b5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-env.xml
>  7b55225 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-site.xml
>  3f860e0 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/ranger-hbase-plugin-properties.xml
>  adcf116 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/ranger-hdfs-plugin-properties.xml
>  1e65763 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/ranger-hive-plugin-properties.xml
>  d82c8b4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/KNOX/configuration/ranger-knox-plugin-properties.xml
>  ad2b1e4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-audit.xml
>  f670d7e 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-policymgr-ssl.xml
>  79370bc 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/ranger-hbase-security.xml
>  7ef63d8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-audit.xml
>  63154eb 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-policymgr-ssl.xml
>  e14a9e8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/ranger-hdfs-security.xml
>  a5f1e24 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/ranger-hive-audit.xml
>  874d0d5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/ranger-hive-policymgr-ssl.xml
>  14e7b16 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HIVE/configuration/ranger-hive-security.xml
>  61e008f 
>   

Re: Review Request 48766: MySQL service status needs to be more robust

[Andrew Onischuk]

> On June 16, 2016, 10:52 p.m., Andrew Onischuk wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py,
> >  line 63
> > 
> >
> > Else should go into for, not if (in case break didn't execute). 
> > 
> > Also (if code == 0) line is not needed since checked_call checks for 
> > non-zero return code itself and throws a reasonable exception in that case.
> 
> Andrew Onischuk wrote:
> Just to be easier to understand here is what I mean:
> 
> def get_mysql_pid_file():
>   cmd = 'mysqladmin variables'
>   code, output = shell.checked_call(cmd)
> 
>   for line in output.splitlines():
> if (line.find("pid_file") != -1):
>   mysql_pid_file  = line.split()[3].strip()
>   break
> else:
>   raise Fail(format("Output of '{cmd}' didn't contain pid_file. 
> Output: {output}"))  
>   return mysql_pid_file
> 
> Andrew Onischuk wrote:
> Juanjo sorry for bothering you so much with this issues. I just feel like 
> this is something which we should fix to be able debug problems in this place 
> more simply in future.
> 
> Juanjo  Marron wrote:
> No problem Andrew! I totally understand the process and it does not 
> bother me at all. These reviews the best way to find a good solution.
> 
> Correct me if I'm wrong, but I dont think this last solution is not  
> working since status action will fail to get the mysqld file. 
> Iterating line by line the mysqladmin variables output, which looks like 
> following: 
> 
> 
> +-+---+
> | Variable_name   | Value 
> |
> 
> +-+---+
> | auto_increment_increment| 1 
> |
> | auto_increment_offset   | 1 
> |
> | autocommit  | ON
> |
> | automatic_sp_privileges | ON
> |
> | back_log| 50
> |
> | basedir | /usr/ 
> |
> | big_tables  | OFF   
> |
> | binlog_cache_size   | 32768 
> |
> 
>  . 
>  
> | old_passwords   | OFF   
> |
> | open_files_limit| 1024  
> |
> | optimizer_prune_level   | 1 
> |
> | optimizer_search_depth  | 62
> |
> | optimizer_switch| 
> index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on
>  |
> | pid_file| /var/run/mysqld/mysqld.pid
> |
> | plugin_dir  | /usr/lib64/mysql/plugin   
> |
> | port| 3306  
> |
> | preload_buffer_size | 32768 
> |
> | profiling   | OFF   
> |
> 
> . 
> 
> if the first line does not include pid_file,  else will run and hence 
> raise the Fail exception.
> 
> Additionally, even if we solved this last issue with this proposal:
> 
> 

Re: Review Request 50653: AMBARI-17308 Ambari Logfeeder outputs a lot of errors due to parse date

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50653/#review144587
---


Ship it!




Ship It!

- Sebastian Toader


On Aug. 3, 2016, 3:15 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50653/
> ---
> 
> (Updated Aug. 3, 2016, 3:15 a.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Dmytro Sen, Oliver Szabo, and 
> Sebastian Toader.
> 
> 
> Bugs: AMBARI-17308
> https://issues.apache.org/jira/browse/AMBARI-17308
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In logsearch_feeder service log, we got errors like below
> ```
> 2016-06-20 15:28:09,368 ERROR file=ambari-audit.log 
> org.apache.ambari.logfeeder.mapper.MapperDate LogFeederUtil.java:356 - Error 
> applying date transformation. isEpoch=false, 
> dateFormat=-MM-dd'T'HH:mm:ss.SSSZ, value=2016-06-20T15:28:08.000. 
> mapClass=map_date, input=input:source=file, 
> path=/var/log/ambari-server/ambari-audit.log, fieldName=logtime. Messages 
> suppressed before: 2
> java.text.ParseException: Unparseable date: "2016-06-20T15:28:08.000"
>   at java.text.DateFormat.parse(DateFormat.java:366)
>   at 
> org.apache.ambari.logfeeder.mapper.MapperDate.apply(MapperDate.java:83)
>   at org.apache.ambari.logfeeder.filter.Filter.apply(Filter.java:154)
>   at 
> org.apache.ambari.logfeeder.filter.FilterGrok.applyMessage(FilterGrok.java:291)
>   at 
> org.apache.ambari.logfeeder.filter.FilterGrok.flush(FilterGrok.java:320)
>   at org.apache.ambari.logfeeder.input.Input.flush(Input.java:125)
>   at 
> org.apache.ambari.logfeeder.input.InputFile.processFile(InputFile.java:430)
>   at org.apache.ambari.logfeeder.input.InputFile.start(InputFile.java:260)
>   at org.apache.ambari.logfeeder.input.Input.run(Input.java:100)
>   at java.lang.Thread.run(Thread.java:745)
> ```
> ambari-audit.log is like below
> ```
> 2016-07-21T01:52:49.875+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-2.5),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
> id(HDP-2.5), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu14/2.x/BUILDS/2.5.0.0-1025)
> 2016-07-21T01:52:49.905+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu16/repositories/HDP-2.5),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu16), Repo 
> id(HDP-2.5), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu16/2.x/BUILDS/2.5.0.0-1025)
> 2016-07-21T01:52:50.015+09, User(admin), RemoteIp(192.168.72.1), 
> Operation(Repository update), RequestType(PUT), 
> url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-UTILS-1.1.0.21),
>  ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo 
> id(HDP-UTILS-1.1.0.21), Base 
> URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14)
> ```
> I think date format of the ambari-audit.log (2016-07-21T01:52:49.875+09) 
> should be like 2016-07-21T01:52:49.875+0900, since grok-pattern can't handle 
> 2016-07-21T01:52:49.875+09 format.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/AuditLoggerDefaultImpl.java
>  ea6524f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/templates/input.config-ambari.json.j2
>  5f751bf 
> 
> Diff: https://reviews.apache.org/r/50653/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 50735: Data can not be migrated from Hive 1.0 to Hive2

[DIPAYAN BHOWMICK]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50735/#review144585
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On Aug. 3, 2016, 4:55 a.m., Gaurav Nagar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50735/
> ---
> 
> (Updated Aug. 3, 2016, 4:55 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
> Kulshreshtha, and Rohit Choudhary.
> 
> 
> Bugs: AMBARI-18000
> https://issues.apache.org/jira/browse/AMBARI-18000
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Added DataMigrator class for mapping entities between versions.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationUtility.java
>  6f427cc 
>   
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/DataMigrator.java
>  PRE-CREATION 
>   contrib/views/hive-next/src/main/resources/view.xml 52fe3ec 
> 
> Diff: https://reviews.apache.org/r/50735/diff/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Gaurav Nagar
> 
>