Review Request 46294: HAWQ activate standby wizard fails after port number change but before restart.

[jun aoki]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46294/
---

Review request for Ambari.


Bugs: AMBARI-15926
https://issues.apache.org/jira/browse/AMBARI-15926


Repository: ambari


Description
---

Activating standby could start a master with an old port, and the following 
operations in the Activate Standby wizard could be mislead. 
The fix is to stop the newly become master so that the wizard will only have to 
deal with the current value.


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py
 e2114d8 

Diff: https://reviews.apache.org/r/46294/diff/


Testing
---

Unit test passed.
Manually tested.


Thanks,

jun aoki

Re: Review Request 46248: YARN work preserving restart should be recommended and validated to be enabled if Hive interactive mode is turned ON

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46248/
---

(Updated April 16, 2016, 12:57 a.m.)


Review request for Ambari, Jaimin Jetly and Swapan Shridhar.


Bugs: AMBARI-15904
https://issues.apache.org/jira/browse/AMBARI-15904


Repository: ambari


Description
---

YARN work preserving restart should be recommended and validated to be enabled 
if Hive interactive mode is turned ON


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
6680611 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py ccf368f 

Diff: https://reviews.apache.org/r/46248/diff/


Testing
---

Unit tests ran successfully and verified by applying on a running cluster. 
Image attached to the JIRA.


Thanks,

Sumit Mohanty

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and disabling LLAP queue when Interactive query is turned on and off respectively.

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/#review129231
---


Ship it!




Ship It!

- Alejandro Fernandez


On April 15, 2016, 10:29 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46183/
> ---
> 
> (Updated April 15, 2016, 10:29 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15876
> https://issues.apache.org/jira/browse/AMBARI-15876
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> -- When Hive Server Interactive is turned ON:
>   Ambari creates OR "modifies 'llap' queue 'state and/or capacity' " based on 
> below conditions:
>- if only 1 queue exists at root level and is 'default' queue and has 
> 100% cap -> Creates 'llap' queue,  OR
>- if 2 queues exists at root level ('llap' and 'default') :
>- 'llap' queue state is STOPPED -> Modifies 'llap' queue state to 
> RUNNING, adjusts capacity, OR
>- 'llap' queue state is RUNNING and 'llap_queue_capacity' prop != 
> 'llap' queue current running capacity ->
>   Modifies 'llap' queue capacity to 'llap_queue_capacity'.
>   
> -- When Hive Server Interactive is turned OFF:
>-  Checks and sees (1). If only two leaf queues exist at root level, 
> namely: 'default' and 'llap',
>   and (2). 'llap' is in RUNNING state.
> 
>   If yes, performs the following actions:   (1). 'llap' queue state set 
> to STOPPED,
> (2). 'llap' queue capacity 
> set to 0 %,
> (3). 'default' queue capacity 
> set to 100 %
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  d518c8b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ba46f00 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46183/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Manual testing done. Various scenarios tested.
> - Test cases added to test the creation and disabling logic.
> 
> - Screenshots attached.
> 
> 
> File Attachments
> 
> 
> LLAP queue created
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
> HSI On and llap and default queues shown in dropdown
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
> YARn queue manager with llap and default queues 
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 46280: Client START while suspended puts entire host to wrong version

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46280/#review129205
---


Ship it!




Ship It!

- Alejandro Fernandez


On April 15, 2016, 9:13 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46280/
> ---
> 
> (Updated April 15, 2016, 9:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-15929
> https://issues.apache.org/jira/browse/AMBARI-15929
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When an upgrade is suspended, the user can initiate any command.  Issuing a 
> START against a client actually invokes INSTALL.  When that happens, the 
> after-INSTALL hook will execute {{hdp-select set all}}.  Check to see if the 
> upgrade is suspended before making that call.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  9870441 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
>  7b51fc0 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
>  b188c16 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  955404c 
> 
> Diff: https://reviews.apache.org/r/46280/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> --
> Total run:960
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 46283: hawq init master fails to syncup hawq-site xml if there is a segment host down

[Lav Jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46283/#review129202
---


Ship it!




Ship It!

- Lav Jain


On April 15, 2016, 10:15 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46283/
> ---
> 
> (Updated April 15, 2016, 10:15 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, jun aoki, 
> Jayush Luniya, Matt, Oleksandr Diachenko, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-15930
> https://issues.apache.org/jira/browse/AMBARI-15930
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In hawq init command, default bucket number is calculated and the updated 
> hawq-site.xml is copied to all the other nodes, if there is a segment host 
> down it will fail to syncup the fail. 
> Including the option to skip bad hosts as ambari will override this file 
> whenever the segment host is brought online and started.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  b929430 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 5f7f085 
> 
> Diff: https://reviews.apache.org/r/46283/diff/
> 
> 
> Testing
> ---
> 
> yes.
> bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-15 
> 15:16:22,910 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys 
> is either set to false or is not available in hawq-env.xml
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok
> 
> --
> Ran 35 tests in 0.148s
> 
> OK
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 46283: hawq init master fails to syncup hawq-site xml if there is a segment host down

[jun aoki]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46283/#review129201
---




ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 302)


```
# utils.exec_hawq_operation(hawq_constants.INIT, "{0}".format(options_str))
utils.exec_hawq_operation(hawq_constants.INIT, options_str)
```
Do we need format()?


- jun aoki


On April 15, 2016, 10:15 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46283/
> ---
> 
> (Updated April 15, 2016, 10:15 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, jun aoki, 
> Jayush Luniya, Matt, Oleksandr Diachenko, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-15930
> https://issues.apache.org/jira/browse/AMBARI-15930
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In hawq init command, default bucket number is calculated and the updated 
> hawq-site.xml is copied to all the other nodes, if there is a segment host 
> down it will fail to syncup the fail. 
> Including the option to skip bad hosts as ambari will override this file 
> whenever the segment host is brought online and started.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  b929430 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 5f7f085 
> 
> Diff: https://reviews.apache.org/r/46283/diff/
> 
> 
> Testing
> ---
> 
> yes.
> bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-15 
> 15:16:22,910 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys 
> is either set to false or is not available in hawq-env.xml
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and disabling LLAP queue when Interactive query is turned on and off respectively.

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/
---

(Updated April 15, 2016, 10:29 p.m.)


Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit Mohanty.


Changes
---

Alejandro's suggestion incorporated.


Bugs: AMBARI-15876
https://issues.apache.org/jira/browse/AMBARI-15876


Repository: ambari


Description
---

-- When Hive Server Interactive is turned ON:
  Ambari creates OR "modifies 'llap' queue 'state and/or capacity' " based on 
below conditions:
   - if only 1 queue exists at root level and is 'default' queue and has 
100% cap -> Creates 'llap' queue,  OR
   - if 2 queues exists at root level ('llap' and 'default') :
   - 'llap' queue state is STOPPED -> Modifies 'llap' queue state to 
RUNNING, adjusts capacity, OR
   - 'llap' queue state is RUNNING and 'llap_queue_capacity' prop != 
'llap' queue current running capacity ->
  Modifies 'llap' queue capacity to 'llap_queue_capacity'.
  
-- When Hive Server Interactive is turned OFF:
   -  Checks and sees (1). If only two leaf queues exist at root level, namely: 
'default' and 'llap',
  and (2). 'llap' is in RUNNING state.

  If yes, performs the following actions:   (1). 'llap' queue state set to 
STOPPED,
(2). 'llap' queue capacity set 
to 0 %,
(3). 'default' queue capacity 
set to 100 %


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
 d518c8b 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
 ba46f00 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
7407da3 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py c21bad7 

Diff: https://reviews.apache.org/r/46183/diff/


Testing
---

- Python UT passes.
- Manual testing done. Various scenarios tested.
- Test cases added to test the creation and disabling logic.

- Screenshots attached.


File Attachments


LLAP queue created
  
https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
HSI On and llap and default queues shown in dropdown
  
https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
YARn queue manager with llap and default queues 
  
https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png


Thanks,

Swapan Shridhar

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and disabling LLAP queue when Interactive query is turned on and off respectively.

[Swapan Shridhar]

> On April 15, 2016, 6:30 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py, 
> > line 24
> > 
> >
> > Can we do this in the constructor?

Done.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/#review129170
---


On April 15, 2016, 10:29 p.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46183/
> ---
> 
> (Updated April 15, 2016, 10:29 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15876
> https://issues.apache.org/jira/browse/AMBARI-15876
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> -- When Hive Server Interactive is turned ON:
>   Ambari creates OR "modifies 'llap' queue 'state and/or capacity' " based on 
> below conditions:
>- if only 1 queue exists at root level and is 'default' queue and has 
> 100% cap -> Creates 'llap' queue,  OR
>- if 2 queues exists at root level ('llap' and 'default') :
>- 'llap' queue state is STOPPED -> Modifies 'llap' queue state to 
> RUNNING, adjusts capacity, OR
>- 'llap' queue state is RUNNING and 'llap_queue_capacity' prop != 
> 'llap' queue current running capacity ->
>   Modifies 'llap' queue capacity to 'llap_queue_capacity'.
>   
> -- When Hive Server Interactive is turned OFF:
>-  Checks and sees (1). If only two leaf queues exist at root level, 
> namely: 'default' and 'llap',
>   and (2). 'llap' is in RUNNING state.
> 
>   If yes, performs the following actions:   (1). 'llap' queue state set 
> to STOPPED,
> (2). 'llap' queue capacity 
> set to 0 %,
> (3). 'default' queue capacity 
> set to 100 %
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  d518c8b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ba46f00 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46183/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Manual testing done. Various scenarios tested.
> - Test cases added to test the creation and disabling logic.
> 
> - Screenshots attached.
> 
> 
> File Attachments
> 
> 
> LLAP queue created
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
> HSI On and llap and default queues shown in dropdown
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
> YARn queue manager with llap and default queues 
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Review Request 46283: hawq init master fails to syncup hawq-site xml if there is a segment host down

[bhuvnesh chaudhary]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46283/
---

Review request for Ambari, Alexander Denissov, Alejandro Fernandez, jun aoki, 
Jayush Luniya, Matt, Oleksandr Diachenko, and Srimanth Gunturi.


Bugs: AMBARI-15930
https://issues.apache.org/jira/browse/AMBARI-15930


Repository: ambari


Description
---

In hawq init command, default bucket number is calculated and the updated 
hawq-site.xml is copied to all the other nodes, if there is a segment host down 
it will fail to syncup the fail. 
Including the option to skip bad hosts as ambari will override this file 
whenever the segment host is brought online and started.


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 b929430 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 5f7f085 

Diff: https://reviews.apache.org/r/46283/diff/


Testing
---

yes.
bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-15 15:16:22,910 
- Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to 
false or is not available in hawq-env.xml
ok
test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
test_install_default (test_hawqsegment.TestHawqSegment) ... ok
test_start_default (test_hawqsegment.TestHawqSegment) ... ok
test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
test_install_default (test_hawqstandby.TestHawqStandby) ... ok
test_start_default (test_hawqstandby.TestHawqStandby) ... ok
test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok

--
Ran 35 tests in 0.148s

OK


Thanks,

bhuvnesh chaudhary

Re: Review Request 46280: Client START while suspended puts entire host to wrong version

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46280/#review129190
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 (lines 2257 - 2262)


OMG, seriously. How many places do we need to intercept commands and inject 
stuff. Nice find!


- Jonathan Hurley


On April 15, 2016, 5:13 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46280/
> ---
> 
> (Updated April 15, 2016, 5:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-15929
> https://issues.apache.org/jira/browse/AMBARI-15929
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When an upgrade is suspended, the user can initiate any command.  Issuing a 
> START against a client actually invokes INSTALL.  When that happens, the 
> after-INSTALL hook will execute {{hdp-select set all}}.  Check to see if the 
> upgrade is suspended before making that call.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  9870441 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
>  7b51fc0 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
>  b188c16 
>   
> ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
>  955404c 
> 
> Diff: https://reviews.apache.org/r/46280/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> --
> Total run:960
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 46263: AMBARI-15906 Make Ranger TagSync to be installed as a slave and optional

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46263/#review129181
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-tagsync-site.xml
 (line 27)


Does this value need to be changed during RU/EU from HDP 2.4 to 2.5?


- Alejandro Fernandez


On April 15, 2016, 6:51 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46263/
> ---
> 
> (Updated April 15, 2016, 6:51 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, Mahadev Konar, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-15906
> https://issues.apache.org/jira/browse/AMBARI-15906
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In order to support blueprints to setup cluster without TagSync, make stack 
> level changes to allow Ranger TagSync to be installed as a optional slave 
> component.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  4466671 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-tagsync-site.xml
>  cfcab3b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
> 9f96308 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
> 
> Diff: https://reviews.apache.org/r/46263/diff/
> 
> 
> Testing
> ---
> 
> Tested on Centos 6 with and without RANGER_TAGSYNC.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 46199: AMBARI-15866: Renew kerberos keyfile rest API returnes a malformed /request URL

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46199/#review129180
---


Ship it!




Ship It!

- Alejandro Fernandez


On April 14, 2016, 12:56 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46199/
> ---
> 
> (Updated April 14, 2016, 12:56 p.m.)
> 
> 
> Review request for Ambari and Alejandro Fernandez.
> 
> 
> Bugs: AMBARI-15866
> https://issues.apache.org/jira/browse/AMBARI-15866
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> curl -u admin:admin 
> 'http://myserver.mydomain.com:8080/api/v1/clusters/mycluster?regenerate_keytabs=all'
>  -X PUT -H 'X-Requested-By: ambari' --data 
> '{"Clusters":{"security_type":"KERBEROS"}}'
> {
> "href" : 
> "http://myserver.mydomain.com:8080/api/v1/clusters/mycluster?regenerate_keytabs=all/requests/15;,
> "Requests" :
> { "id" : 15, "status" : "Accepted" }
> 
> }
> 
> Notice the "regenerate_keytabs=all" piece in the URL returned by the Ambari 
> server. If user pulls request status against that URL, he will get the 
> following err.
> 
> curlit GET 
> http://myserver.mydomain.com:8080/api/v1/clusters/mycluster?regenerate_keytabs=all/requests/15
> { "status" : 400, "message" : "The properties [regenerate_keytabs] specified 
> in the request or predicate are not supported for the resource type Cluster." 
> }
> 
> The correct url should be 
> http://myserver.mydomain.com:8080/api/v1/clusters/mycluster/requests/15
> 
> This does not effect Ambari web UI as Ambari UI pulls status of all requests 
> by /requests url, instead of using individual request URLs.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/resources/RequestResourceDefinition.java
>  cf3f267 
> 
> Diff: https://reviews.apache.org/r/46199/diff/
> 
> 
> Testing
> ---
> 
> The issue is general, regardless whether the cluster is kerberized, as it's 
> about how the href url is parsed.
> 
> kerberized a trunk cluster, patch the cluster with fix. 
> 1. submit REST API call to renew kerberos keytabs.
> Verify the /requests URL returned is valid and can be used to pull request 
> status.
> 2. submit a custom action API (like the host check one).
> Verify the existing href behavior is still correct.
> 3. Submit a rest api that does not need to return /requests url, like get 
> stack details REST API
> Veriy the existing href behavior is still correct.
> 
> On a non-kerberized trunk cluster, patch the cluster with fix, 
> Repeat the 3 tests mentioned above.
> 
> 
> Thanks,
> 
> Di Li
> 
>

Re: Review Request 46278: Ambari falsely reporting Hiveserver2 up

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46278/#review129175
---




ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
 (line 219)


This may be problematic during tasks that are meant to fail quickly. 60 
second sleep is extremely long


- Alejandro Fernandez


On April 15, 2016, 6:42 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46278/
> ---
> 
> (Updated April 15, 2016, 6:42 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15928
> https://issues.apache.org/jira/browse/AMBARI-15928
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> PROBLEM:
> After adding a custom whitelist.append value (incorrect value), and using 
> Ambari server to complete a 'clean' stop/start of all Hive components, it 
> fails to bring up Hiveserver2 properly (but reports successful ambari-agent 
> task).
> Update:
> After 30 attempts (30mins of looping) to start Hiveserver2, the log finally 
> ended at this point
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  7cbe4e0 
> 
> Diff: https://reviews.apache.org/r/46278/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 46278: Ambari falsely reporting Hiveserver2 up

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46278/
---

Review request for Ambari, Andrew Onischuk, Dmitro Lisnichenko, and Sumit 
Mohanty.


Bugs: AMBARI-15928
https://issues.apache.org/jira/browse/AMBARI-15928


Repository: ambari


Description
---

PROBLEM:
After adding a custom whitelist.append value (incorrect value), and using 
Ambari server to complete a 'clean' stop/start of all Hive components, it fails 
to bring up Hiveserver2 properly (but reports successful ambari-agent task).
Update:
After 30 attempts (30mins of looping) to start Hiveserver2, the log finally 
ended at this point


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
 7cbe4e0 

Diff: https://reviews.apache.org/r/46278/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and disabling LLAP queue when Interactive query is turned on and off respectively.

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/#review129170
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
24)


Can we do this in the constructor?


- Alejandro Fernandez


On April 15, 2016, 9:24 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46183/
> ---
> 
> (Updated April 15, 2016, 9:24 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15876
> https://issues.apache.org/jira/browse/AMBARI-15876
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> -- When Hive Server Interactive is turned ON:
>   Ambari creates OR "modifies 'llap' queue 'state and/or capacity' " based on 
> below conditions:
>- if only 1 queue exists at root level and is 'default' queue and has 
> 100% cap -> Creates 'llap' queue,  OR
>- if 2 queues exists at root level ('llap' and 'default') :
>- 'llap' queue state is STOPPED -> Modifies 'llap' queue state to 
> RUNNING, adjusts capacity, OR
>- 'llap' queue state is RUNNING and 'llap_queue_capacity' prop != 
> 'llap' queue current running capacity ->
>   Modifies 'llap' queue capacity to 'llap_queue_capacity'.
>   
> -- When Hive Server Interactive is turned OFF:
>-  Checks and sees (1). If only two leaf queues exist at root level, 
> namely: 'default' and 'llap',
>   and (2). 'llap' is in RUNNING state.
> 
>   If yes, performs the following actions:   (1). 'llap' queue state set 
> to STOPPED,
> (2). 'llap' queue capacity 
> set to 0 %,
> (3). 'default' queue capacity 
> set to 100 %
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  d518c8b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ba46f00 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46183/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Manual testing done. Various scenarios tested.
> - Test cases added to test the creation and disabling logic.
> 
> - Screenshots attached.
> 
> 
> File Attachments
> 
> 
> LLAP queue created
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
> HSI On and llap and default queues shown in dropdown
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
> YARn queue manager with llap and default queues 
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 46253: Add ignore-bad-hosts flag to hawq remove standby custom action to avoid failures when a segment host is down

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46253/#review129167
---


Ship it!




Ship It!

- Matt


On April 14, 2016, 10:59 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46253/
> ---
> 
> (Updated April 14, 2016, 10:59 p.m.)
> 
> 
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, jun aoki, 
> Jayush Luniya, and Matt.
> 
> 
> Bugs: AMBARI-15905
> https://issues.apache.org/jira/browse/AMBARI-15905
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add ignore-bad-hosts flag to hawq remove standby custom action to avoid 
> failures when a segment host is down
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
>  f2ec914 
> 
> Diff: https://reviews.apache.org/r/46253/diff/
> 
> 
> Testing
> ---
> 
> yes.
> 
> bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_exception_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
> handlers could be found for logger "ambari_alerts"
> ok
> test_missing_configs 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_slave_file 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_successful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_empty_db_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_unsuccessful_registration_status_plural 
> (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-14 
> 23:01:10,891 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys 
> is either set to false or is not available in hawq-env.xml
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> 
> --
> Ran 34 tests in 0.147s
> 
> OK
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 46277: Deploy and test security cluster on suse12

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46277/#review129166
---


Ship it!




Ship It!

- Robert Levas


On April 15, 2016, 2:16 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46277/
> ---
> 
> (Updated April 15, 2016, 2:16 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Robert Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15927
> https://issues.apache.org/jira/browse/AMBARI-15927
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> .
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/krb5-conf.xml
>  55f977d 
>   
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/templates/krb5_conf.j2
>  d473ede 
> 
> Diff: https://reviews.apache.org/r/46277/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 46269: SQL constraints: Inline constraints and name them in CREATE table

[Nate Cole]

> On April 15, 2016, 10:02 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql, line 
> > 1572
> > 
> >
> > Quartz tables are not ours and I don't know the implications of 
> > changing them. Let's stick to our own.
> 
> Balázs Bence Sári wrote:
> Ok. I'll revert all changes to them (including inlining constraints if 
> any). Also, is there anything else not owned by us (e.g.alert fwk, kerberos 
> workflow) ?

I think it would be a good time to remove those pesky RCA tables.  They're an 
old artifact that no one's using.  (Test a full deployment just to be sure)


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46269/#review129118
---


On April 15, 2016, 9:41 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46269/
> ---
> 
> (Updated April 15, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, Jonathan Robie, Nate Cole, and Nahappan 
> Somasundaram.
> 
> 
> Bugs: AMBARI-15915
> https://issues.apache.org/jira/browse/AMBARI-15915
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Inlined alter table statements (except one to overcome a circular FK 
> dependency)
> 2. Reorganized create table statemens to avoid forward references
> 3. Named all unnamed constraints, shortened names to fit Oracle's 30 
> character limit
> 4. Unified constraint names across database flavors
> 5. Wrote a JUnit test that checks DDL's for consistency (all constraints are 
> named, the same tables and constraints are defined for all database flavors)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql f90ac96 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 2b214c4 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql fc93372 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 870a8e8 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 71d4813 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 6e600c7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql a3ea10d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTestUtils.java 
> PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTests.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46269/diff/
> 
> 
> Testing
> ---
> 
> 1. Run all java unit tests in ambari-server. All passed, except 
> ViewDirectoryWatcherTest.testDirectoryExtractionOnFileAdd(). Failure occured 
> to a hardcoded timeout value in the test being to low for my environment (I 
> raised the timeout from 7 secs to 20 and the test passed)
> 2. Run DDL's with their respective DB environment. In cases where the DDL 
> contained additional things other than table/index cration and insertion 
> (e.g. creating databases and users) I only run the table/index creation part.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Review Request 46277: Deploy and test security cluster on suse12

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46277/
---

Review request for Ambari, Andrew Onischuk, Robert Levas, and Sumit Mohanty.


Bugs: AMBARI-15927
https://issues.apache.org/jira/browse/AMBARI-15927


Repository: ambari


Description
---

.


Diffs
-

  
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/krb5-conf.xml
 55f977d 
  
ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/templates/krb5_conf.j2
 d473ede 

Diff: https://reviews.apache.org/r/46277/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 46269: SQL constraints: Inline constraints and name them in CREATE table

[Nate Cole]

> On April 15, 2016, 9:52 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql, lines 181-184
> > 
> >
> > Let's be consistent.  Either:
> > pk_sc_desired_state
> > fk_scds_...
> > unq_scdesired...
> > 
> > or
> > PK_sc_desired
> > FK_scds...
> > UNQ_scdesired...
> > 
> > I seem recall issues of case between the sql files and UpgradeCatalog* 
> > classes.  Jonathan?
> 
> Jonathan Hurley wrote:
> I'm all for consistency. You're right that we used to have problems with 
> the case of the contraints. For PK's, we added some utility methods which 
> don't require us to know the name anymore:
> 
> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessor.java#L569-L581
> 
> We could do something similar for FKs as well.
> 
> Balázs Bence Sári wrote:
> I've got a couple of questions:
> - The method in DBAccessor only works with 3 databases. Can we rely on 
> that?
> - I just learned that the name of the constraints are important during 
> upgrade. So I guess I'd need to revert any constraint renamings I've done and 
> only name constraints that had been unnamed so far. (pls. confirm)
> - Existing constraints are mostly in PK_the_rest_is_lowecase format, but 
> some are in all lowercase format (also unique is sometimes noted as UQ and 
> sometimes UNQ). I always used the PK/FK/UQ_the_rest_is_lowercase format 
> because that seemed to be the most frequent. Are you ok to follow that 
> convenions? (Also, what should happen with existing all-lowercase 
> constraints? Would renaming them break upgrade scripts?)

I don't mind the PK/FK/UNQ_lower_name syntax myself, it's easy to visually see 
"type" from "name" if you will.  These scripts are create-only, so changing 
them shouldn't break existing deployments.  The concern is if you've added new 
constraints, then yes, those need to be reflected in UpgradeCatalog, but 
otherwise it seems safe.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46269/#review129114
---


On April 15, 2016, 9:41 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46269/
> ---
> 
> (Updated April 15, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, Jonathan Robie, Nate Cole, and Nahappan 
> Somasundaram.
> 
> 
> Bugs: AMBARI-15915
> https://issues.apache.org/jira/browse/AMBARI-15915
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Inlined alter table statements (except one to overcome a circular FK 
> dependency)
> 2. Reorganized create table statemens to avoid forward references
> 3. Named all unnamed constraints, shortened names to fit Oracle's 30 
> character limit
> 4. Unified constraint names across database flavors
> 5. Wrote a JUnit test that checks DDL's for consistency (all constraints are 
> named, the same tables and constraints are defined for all database flavors)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql f90ac96 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 2b214c4 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql fc93372 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 870a8e8 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 71d4813 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 6e600c7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql a3ea10d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTestUtils.java 
> PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTests.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46269/diff/
> 
> 
> Testing
> ---
> 
> 1. Run all java unit tests in ambari-server. All passed, except 
> ViewDirectoryWatcherTest.testDirectoryExtractionOnFileAdd(). Failure occured 
> to a hardcoded timeout value in the test being to low for my environment (I 
> raised the timeout from 7 secs to 20 and the test passed)
> 2. Run DDL's with their respective DB environment. In cases where the DDL 
> contained additional things other than table/index cration and insertion 
> (e.g. creating databases and users) I only run the table/index creation part.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 46248: YARN work preserving restart should be recommended and validated to be enabled if Hive interactive mode is turned ON

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46248/#review129161
---


Ship it!




Fix it and Ship it.

- Swapan Shridhar


On April 15, 2016, 2:47 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46248/
> ---
> 
> (Updated April 15, 2016, 2:47 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Swapan Shridhar.
> 
> 
> Bugs: AMBARI-15904
> https://issues.apache.org/jira/browse/AMBARI-15904
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> YARN work preserving restart should be recommended and validated to be 
> enabled if Hive interactive mode is turned ON
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46248/diff/
> 
> 
> Testing
> ---
> 
> Unit tests ran successfully and verified by applying on a running cluster. 
> Image attached to the JIRA.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 46248: YARN work preserving restart should be recommended and validated to be enabled if Hive interactive mode is turned ON

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46248/#review129159
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py (line 
62)


You can use the helper code for getting Hosts on which a given service 
component is hosted. Should be in as part of AMBARI-15876 (Stack advisor work 
for adding and disabling LLAP queue when Interactive query is turned on and off 
respectively.)


- Swapan Shridhar


On April 15, 2016, 2:47 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46248/
> ---
> 
> (Updated April 15, 2016, 2:47 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Swapan Shridhar.
> 
> 
> Bugs: AMBARI-15904
> https://issues.apache.org/jira/browse/AMBARI-15904
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> YARN work preserving restart should be recommended and validated to be 
> enabled if Hive interactive mode is turned ON
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46248/diff/
> 
> 
> Testing
> ---
> 
> Unit tests ran successfully and verified by applying on a running cluster. 
> Image attached to the JIRA.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 46274: Metrics Unit Tests Failing on b.a.o

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46274/#review129152
---


Ship it!




Ship It!

- Sumit Mohanty


On April 15, 2016, 4:59 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46274/
> ---
> 
> (Updated April 15, 2016, 4:59 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15923
> https://issues.apache.org/jira/browse/AMBARI-15923
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> https://builds.apache.org/job/Ambari-trunk-Commit/4662/#showFailuresLink
> {code}
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.availability.TimelineMetricHAControllerTest.testHAControllerDistributedAggregation
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.availability.TimelineMetricHAControllerTest.testClusterOK
> {code}
> 
> https://builds.apache.org/job/Ambari-branch-2.2/636/
> {code}
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.discovery.TestMetadataManager.testSaveMetricsMetadata
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.discovery.TestMetadataManager.testClusterOK
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
>  a39fcb6 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java
>  7410e9d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
>  e6a360a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/TimelineMetricHAControllerTest.java
>  04e8909 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  2b33cb95 
> 
> Diff: https://reviews.apache.org/r/46274/diff/
> 
> 
> Testing
> ---
> 
> Unit tests pass.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 46274: Metrics Unit Tests Failing on b.a.o

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46274/#review129148
---


Ship it!




Ship It!

- Jonathan Hurley


On April 15, 2016, 12:59 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46274/
> ---
> 
> (Updated April 15, 2016, 12:59 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jonathan Hurley, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15923
> https://issues.apache.org/jira/browse/AMBARI-15923
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> https://builds.apache.org/job/Ambari-trunk-Commit/4662/#showFailuresLink
> {code}
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.availability.TimelineMetricHAControllerTest.testHAControllerDistributedAggregation
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.availability.TimelineMetricHAControllerTest.testClusterOK
> {code}
> 
> https://builds.apache.org/job/Ambari-branch-2.2/636/
> {code}
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.discovery.TestMetadataManager.testSaveMetricsMetadata
> org.apache.hadoop.yarn.server.applicationhistoryservice.metrics.timeline.discovery.TestMetadataManager.testClusterOK
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
>  a39fcb6 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/AbstractMiniHBaseClusterTest.java
>  7410e9d 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
>  e6a360a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/availability/TimelineMetricHAControllerTest.java
>  04e8909 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  2b33cb95 
> 
> Diff: https://reviews.apache.org/r/46274/diff/
> 
> 
> Testing
> ---
> 
> Unit tests pass.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 46263: AMBARI-15906 Make Ranger TagSync to be installed as a slave and optional

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46263/#review129146
---


Ship it!




Have you validated that TagSync installations in existing clusters still work 
after upgrading to this version of Ambari where it changes from a master to a 
slave?

- Jonathan Hurley


On April 15, 2016, 2:51 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46263/
> ---
> 
> (Updated April 15, 2016, 2:51 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, Mahadev Konar, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-15906
> https://issues.apache.org/jira/browse/AMBARI-15906
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In order to support blueprints to setup cluster without TagSync, make stack 
> level changes to allow Ranger TagSync to be installed as a optional slave 
> component.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  4466671 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-tagsync-site.xml
>  cfcab3b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
> 9f96308 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
> 
> Diff: https://reviews.apache.org/r/46263/diff/
> 
> 
> Testing
> ---
> 
> Tested on Centos 6 with and without RANGER_TAGSYNC.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 46238: Refactor Metadata manager for supporting distributed collector

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46238/#review129138
---


Ship it!




Ship It!

- Dmytro Sen


On Апрель 14, 2016, 11:54 п.п., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46238/
> ---
> 
> (Updated Апрель 14, 2016, 11:54 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15902
> https://issues.apache.org/jira/browse/AMBARI-15902
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Presently metadata is served from a write through cache, we need a bounded 
> cache that gets refreshed periodically to allow sharded writes to succeed.
> 
> Additionally the metadata cache needs ability to skip certain metrics, 
> example: YARN Container metrics with too many unique names using a config.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  5b517ef 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
>  8e58203 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataSync.java
>  54ea200 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  bf887be 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataSync.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  ddaff84 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  e5758bf 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
>  6f32000 
> 
> Diff: https://reviews.apache.org/r/46238/diff/
> 
> 
> Testing
> ---
> 
> All metrics Unit tests passed.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 46055: AMBARI-15815: Direct Ambari Upgrade from 2.0.1 to 2.4.x failed with error

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46055/#review129140
---


Ship it!




Ship It!

- Jonathan Hurley


On April 12, 2016, 11:38 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46055/
> ---
> 
> (Updated April 12, 2016, 11:38 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Jonathan Hurley, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-15815
> https://issues.apache.org/jira/browse/AMBARI-15815
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-15815: Direct Ambari Upgrade from 2.0.1 to 2.4.x  failed with error
> 
> ** Issue **:
> When upgrading from ambari 2.0.1 to 2.4.0, upgrade fails with 
> NullPointerException.
> 
> This is because in alert_definition table, *ambari_agent_disk_usage* type is 
> missing the following parameters in the alert source:
> ** percent.free.space.critical.threshold **
> ** percent.used.space.warning.threshold **
> 
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.upgrade.UpgradeCatalog240.updateAlerts(UpgradeCatalog240.java:497)
> at 
> org.apache.ambari.server.upgrade.UpgradeCatalog240.executeDMLUpdates(UpgradeCatalog240.java:180)
> 
> ** Fix **:
> Verify if float value is not available, before trying to convert it to 
> percentage value.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  31af5e3 
> 
> Diff: https://reviews.apache.org/r/46055/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [7.453s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.039s]
> [INFO] Ambari Web  SUCCESS [24.974s]
> [INFO] Ambari Views .. SUCCESS [1.005s]
> [INFO] Ambari Admin View . SUCCESS [7.226s]
> [INFO] ambari-metrics  SUCCESS [0.355s]
> [INFO] Ambari Metrics Common . SUCCESS [0.449s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.006s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.553s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.589s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.459s]
> [INFO] Ambari Metrics Collector .. SUCCESS [6.242s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.683s]
> [INFO] Ambari Metrics Grafana  SUCCESS [0.934s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:17.665s]
> [INFO] Ambari Server . SUCCESS [2:26.545s]
> [INFO] Ambari Functional Tests ... SUCCESS [1.125s]
> [INFO] Ambari Agent .. SUCCESS [22.799s]
> [INFO] Ambari Client . SUCCESS [0.057s]
> [INFO] Ambari Python Client .. SUCCESS [0.860s]
> [INFO] Ambari Groovy Client .. SUCCESS [1.996s]
> [INFO] Ambari Shell .. SUCCESS [0.046s]
> [INFO] Ambari Python Shell ... SUCCESS [0.659s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.851s]
> [INFO] ambari-logsearch .. SUCCESS [0.086s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [35.525s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [29.153s]
> [INFO] Ambari Logsearch Appender . SUCCESS [1.606s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.075s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:13.709s
> [INFO] Finished at: Mon Apr 11 16:00:34 PDT 2016
> [INFO] Final Memory: 272M/1173M
> [INFO] 
> 
> 
> ** 2. mvn test -Dtest=UpgradeCatalog240Test -DskipPythonTests **
> 
> ---
>  T E S T S
> ---
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m 
> -Djava.awt.headless=true
> Running org.apache.ambari.server.upgrade.UpgradeCatalog240Test
> 

Re: Review Request 46196: Logsearch: reading config values from external property files

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46196/
---

(Updated April 15, 2016, 4:03 p.m.)


Review request for Ambari, Jayush Luniya, Robert Nettleton, and Sumit Mohanty.


Changes
---

- added xml support (xml validtion)


Bugs: AMBARI-15842
https://issues.apache.org/jira/browse/AMBARI-15842


Repository: ambari


Description
---

Reading config values (like xml files or *.sh scripts) from external property 
files.

I did not added xml support for reading external property files. Reason is 
because these xml files are jinja templates. So they can act as text files.

also added: Solr Instance renamed to LogSearch Solr Instance


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
 84842e2 
  ambari-server/src/main/java/org/apache/ambari/server/utils/XmlUtils.java 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
 94ed27d 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-log4j.xml
 91fc0ce 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-audit_logs-solrconfig.xml
 4da16b1 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
 ef6bebf 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-log4j.xml
 6741f41 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-service_logs-solrconfig.xml
 60363c8 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
 855f0e5 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-log4j.xml
 e99cb81 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-xml.xml
 d31d0b3 
  ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml 
018398f 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-env.sh.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-log4j.xml.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-env.sh.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-log4j.xml.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-solr-env.sh.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr-log4j.properties.j2
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr.xml.j2
 PRE-CREATION 

Diff: https://reviews.apache.org/r/46196/diff/


Testing
---

Total run:959
Total errors:0
Total failures:0


Thanks,

Oliver Szabo

Re: Review Request 46238: Refactor Metadata manager for supporting distributed collector

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46238/#review129135
---


Ship it!




Ship It!

- Aravindan Vijayan


On April 14, 2016, 11:54 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46238/
> ---
> 
> (Updated April 14, 2016, 11:54 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15902
> https://issues.apache.org/jira/browse/AMBARI-15902
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Presently metadata is served from a write through cache, we need a bounded 
> cache that gets refreshed periodically to allow sharded writes to succeed.
> 
> Additionally the metadata cache needs ability to skip certain metrics, 
> example: YARN Container metrics with too many unique names using a config.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  5b517ef 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
>  8e58203 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataSync.java
>  54ea200 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  bf887be 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataSync.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  ddaff84 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  e5758bf 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
>  6f32000 
> 
> Diff: https://reviews.apache.org/r/46238/diff/
> 
> 
> Testing
> ---
> 
> All metrics Unit tests passed.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 46238: Refactor Metadata manager for supporting distributed collector

[Sid Wagle]

> On April 15, 2016, 8:19 a.m., Dmytro Sen wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2,
> >  line 104
> > 
> >
> > It'll increase number of metrics sent from sinks to ams collector, so 
> > increase the collector load

The ContainerMetrics will be syphoned off by HadoopMetricsSync into separate 
tables using API. This is currently scheduled for drop into 2.4.0, hence this 
is related change that we need to make so that the ContainerMetrics are not 
dropped.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46238/#review129079
---


On April 14, 2016, 11:54 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46238/
> ---
> 
> (Updated April 14, 2016, 11:54 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15902
> https://issues.apache.org/jira/browse/AMBARI-15902
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Presently metadata is served from a write through cache, we need a bounded 
> cache that gets refreshed periodically to allow sharded writes to succeed.
> 
> Additionally the metadata cache needs ability to skip certain metrics, 
> example: YARN Container metrics with too many unique names using a config.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  5b517ef 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
>  8e58203 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataSync.java
>  54ea200 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  bf887be 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataSync.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  ddaff84 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  e5758bf 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
>  6f32000 
> 
> Diff: https://reviews.apache.org/r/46238/diff/
> 
> 
> Testing
> ---
> 
> All metrics Unit tests passed.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 46273: By default, YARN does not emit audit logs into a separate log file

[Vitalyi Brodetskyi]

> On Квітень 15, 2016, 3:17 після полудня, Vitalyi Brodetskyi wrote:
> > Ship It!

Please check default log4j from Yarn.


- Vitalyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46273/#review129129
---


On Квітень 15, 2016, 3:04 після полудня, Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46273/
> ---
> 
> (Updated Квітень 15, 2016, 3:04 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-15921
> https://issues.apache.org/jira/browse/AMBARI-15921
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *This is not a regression*.
> 
> Current set of logs:
> 
> {code}
> $ls /var/log/hadoop-yarn/yarn
> hadoop-mapreduce.jobsummary.log
> yarn-yarn-resourcemanager.log
> yarn-yarn-resourcemanager.out
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-log4j.xml
>  6402375 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-log4j.xml
>  b226754 
> 
> Diff: https://reviews.apache.org/r/46273/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46271: Migrate instance specific data from one version to another.

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46271/
---

(Updated April 15, 2016, 3:24 p.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, and Rohit Choudhary.


Bugs: AMBARI-15919
https://issues.apache.org/jira/browse/AMBARI-15919


Repository: ambari


Description
---

Added ability to migrate data using rest API.
Added data-migrator-class, data-version tags in view.xml . View needs to 
implement ViewDataMigrator class that contains logic to migrate data from older 
version.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewDataMigrationService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewInstanceService.java
 1eea8db 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrivilegeDAO.java 
772d538 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ViewInstanceEntity.java
 2555f93 
  
ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationContextImpl.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java 
7379f37 
  
ambari-server/src/main/java/org/apache/ambari/server/view/configuration/ViewConfig.java
 728f620 
  
ambari-server/src/test/java/org/apache/ambari/server/api/services/ViewDataMigrationServiceTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/ViewInstanceEntityTest.java
 c8c15da 
  
ambari-server/src/test/java/org/apache/ambari/server/view/ViewDataMigrationContextImplTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/view/configuration/ViewConfigTest.java
 14e5d3c 
  ambari-views/examples/README.md 650f650 
  ambari-views/examples/calculator-view/pom.xml d5e0e8e 
  ambari-views/examples/cluster-view/pom.xml 99582a6 
  ambari-views/examples/favorite-view/pom.xml ba196f5 
  ambari-views/examples/hello-servlet-view/pom.xml ed6fc25 
  ambari-views/examples/hello-spring-view/pom.xml 3475ef4 
  ambari-views/examples/helloworld-view/pom.xml aa89778 
  ambari-views/examples/phone-list-upgrade-view/docs/index.md PRE-CREATION 
  ambari-views/examples/phone-list-upgrade-view/pom.xml PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/java/org/apache/ambari/view/phonelist/DataMigrator.java
 PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/java/org/apache/ambari/view/phonelist/PhoneListServlet.java
 PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/java/org/apache/ambari/view/phonelist/PhoneUser.java
 PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/resources/WEB-INF/web.xml
 PRE-CREATION 
  ambari-views/examples/phone-list-upgrade-view/src/main/resources/view.xml 
PRE-CREATION 
  ambari-views/examples/phone-list-view/pom.xml ead9095 
  ambari-views/examples/pom.xml de55ded 
  ambari-views/examples/property-validator-view/pom.xml 045a277 
  ambari-views/examples/property-view/pom.xml ad4630b 
  ambari-views/examples/restricted-view/pom.xml f7fa3d6 
  ambari-views/examples/simple-view/pom.xml 015f192 
  ambari-views/examples/weather-view/pom.xml a018258 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/EntityConverter.java
 PRE-CREATION 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/ViewDataMigrationContext.java
 PRE-CREATION 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/ViewDataMigrationException.java
 PRE-CREATION 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/ViewDataMigrator.java
 PRE-CREATION 
  ambari-views/src/main/java/org/apache/ambari/view/validation/Validator.java 
ee029a8 
  ambari-views/src/main/resources/view.xsd c3ad711 
  pom.xml f540099 

Diff: https://reviews.apache.org/r/46271/diff/


Testing (updated)
---

Manual testing.


Thanks,

Gaurav Nagar

Review Request 46271: Migrate instance specific data from one version to another.

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46271/
---

Review request for Ambari, DIPAYAN BHOWMICK, Nitiraj Rathore, Pallav 
Kulshreshtha, and Rohit Choudhary.


Bugs: AMBARI-15919
https://issues.apache.org/jira/browse/AMBARI-15919


Repository: ambari


Description
---

Added ability to migrate data using rest API.
Added data-migrator-class, data-version tags in view.xml . View needs to 
implement ViewDataMigrator class that contains logic to migrate data from older 
version.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewDataMigrationService.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewInstanceService.java
 1eea8db 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrivilegeDAO.java 
772d538 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ViewInstanceEntity.java
 2555f93 
  
ambari-server/src/main/java/org/apache/ambari/server/view/ViewDataMigrationContextImpl.java
 PRE-CREATION 
  ambari-server/src/main/java/org/apache/ambari/server/view/ViewRegistry.java 
7379f37 
  
ambari-server/src/main/java/org/apache/ambari/server/view/configuration/ViewConfig.java
 728f620 
  
ambari-server/src/test/java/org/apache/ambari/server/api/services/ViewDataMigrationServiceTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/ViewInstanceEntityTest.java
 c8c15da 
  
ambari-server/src/test/java/org/apache/ambari/server/view/ViewDataMigrationContextImplTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/view/configuration/ViewConfigTest.java
 14e5d3c 
  ambari-views/examples/README.md 650f650 
  ambari-views/examples/calculator-view/pom.xml d5e0e8e 
  ambari-views/examples/cluster-view/pom.xml 99582a6 
  ambari-views/examples/favorite-view/pom.xml ba196f5 
  ambari-views/examples/hello-servlet-view/pom.xml ed6fc25 
  ambari-views/examples/hello-spring-view/pom.xml 3475ef4 
  ambari-views/examples/helloworld-view/pom.xml aa89778 
  ambari-views/examples/phone-list-upgrade-view/docs/index.md PRE-CREATION 
  ambari-views/examples/phone-list-upgrade-view/pom.xml PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/java/org/apache/ambari/view/phonelist/DataMigrator.java
 PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/java/org/apache/ambari/view/phonelist/PhoneListServlet.java
 PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/java/org/apache/ambari/view/phonelist/PhoneUser.java
 PRE-CREATION 
  
ambari-views/examples/phone-list-upgrade-view/src/main/resources/WEB-INF/web.xml
 PRE-CREATION 
  ambari-views/examples/phone-list-upgrade-view/src/main/resources/view.xml 
PRE-CREATION 
  ambari-views/examples/phone-list-view/pom.xml ead9095 
  ambari-views/examples/pom.xml de55ded 
  ambari-views/examples/property-validator-view/pom.xml 045a277 
  ambari-views/examples/property-view/pom.xml ad4630b 
  ambari-views/examples/restricted-view/pom.xml f7fa3d6 
  ambari-views/examples/simple-view/pom.xml 015f192 
  ambari-views/examples/weather-view/pom.xml a018258 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/EntityConverter.java
 PRE-CREATION 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/ViewDataMigrationContext.java
 PRE-CREATION 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/ViewDataMigrationException.java
 PRE-CREATION 
  
ambari-views/src/main/java/org/apache/ambari/view/migration/ViewDataMigrator.java
 PRE-CREATION 
  ambari-views/src/main/java/org/apache/ambari/view/validation/Validator.java 
ee029a8 
  ambari-views/src/main/resources/view.xsd c3ad711 
  pom.xml f540099 

Diff: https://reviews.apache.org/r/46271/diff/


Testing
---


Thanks,

Gaurav Nagar

Re: Review Request 46273: By default, YARN does not emit audit logs into a separate log file

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46273/#review129129
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Квітень 15, 2016, 3:04 після полудня, Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46273/
> ---
> 
> (Updated Квітень 15, 2016, 3:04 після полудня)
> 
> 
> Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-15921
> https://issues.apache.org/jira/browse/AMBARI-15921
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *This is not a regression*.
> 
> Current set of logs:
> 
> {code}
> $ls /var/log/hadoop-yarn/yarn
> hadoop-mapreduce.jobsummary.log
> yarn-yarn-resourcemanager.log
> yarn-yarn-resourcemanager.out
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-log4j.xml
>  6402375 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-log4j.xml
>  b226754 
> 
> Diff: https://reviews.apache.org/r/46273/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Review Request 46273: By default, YARN does not emit audit logs into a separate log file

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46273/
---

Review request for Ambari, Andrew Onischuk and Vitalyi Brodetskyi.


Bugs: AMBARI-15921
https://issues.apache.org/jira/browse/AMBARI-15921


Repository: ambari


Description
---

*This is not a regression*.

Current set of logs:

{code}
$ls /var/log/hadoop-yarn/yarn
hadoop-mapreduce.jobsummary.log
yarn-yarn-resourcemanager.log
yarn-yarn-resourcemanager.out
{code}


Diffs
-

  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-log4j.xml
 6402375 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration/yarn-log4j.xml
 b226754 

Diff: https://reviews.apache.org/r/46273/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 46196: Logsearch: reading config values from external property files

[Oliver Szabo]

> On April 15, 2016, 9:06 a.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2,
> >  line 1
> > 
> >
> > I don't think this will work as the format of the document is really 
> > XML and embedding an XML markup data as an XML value wouldnt work. You will 
> > need to either add CDATA to the document or encode it if you want to use it 
> > as plain text. Better to add support for XML_PROPERTY_FILE_TYPE so that we 
> > can add logic to encode the property xml contents.
> > 
> > You can verify this by starting ambari server and noticing that this 
> > config type will fail to show up in the STACKS API.
> 
> Oliver Szabo wrote:
> I tried that, did not failed and i could install a cluster with my 
> configurations. CDATA is not the best option, because there are CDATA tags in 
> that xml too, so its closing tag closed the initial CDATA tag (i mean if I 
> tried that in the *-site.xml )
> 
> Oliver Szabo wrote:
> of course that format can cause issues with e.g. blueprint export so i 
> should notice that

I checked multiple times now, so its looking good with export blueprints and 
with the stacks API too. But of course i can add XML support just for 
validation (same like with JSON type)


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46196/#review129084
---


On April 14, 2016, 11:47 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46196/
> ---
> 
> (Updated April 14, 2016, 11:47 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15842
> https://issues.apache.org/jira/browse/AMBARI-15842
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Reading config values (like xml files or *.sh scripts) from external property 
> files.
> 
> I did not added xml support for reading external property files. Reason is 
> because these xml files are jinja templates. So they can act as text files.
> 
> also added: Solr Instance renamed to LogSearch Solr Instance
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  94ed27d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-log4j.xml
>  91fc0ce 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-audit_logs-solrconfig.xml
>  4da16b1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  ef6bebf 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-log4j.xml
>  6741f41 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-service_logs-solrconfig.xml
>  60363c8 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  855f0e5 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-log4j.xml
>  e99cb81 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-xml.xml
>  d31d0b3 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml 
> 018398f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-solr-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr.xml.j2
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46196/diff/
> 
> 
> Testing
> ---
> 
> Total run:959
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 46269: SQL constraints: Inline constraints and name them in CREATE table

[Balázs Bence Sári]

> On April 15, 2016, 2:02 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql, line 
> > 1572
> > 
> >
> > Quartz tables are not ours and I don't know the implications of 
> > changing them. Let's stick to our own.

Ok. I'll revert all changes to them (including inlining constraints if any). 
Also, is there anything else not owned by us (e.g.alert fwk, kerberos workflow) 
?


- Balázs Bence


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46269/#review129118
---


On April 15, 2016, 1:41 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46269/
> ---
> 
> (Updated April 15, 2016, 1:41 p.m.)
> 
> 
> Review request for Ambari, Jonathan Robie, Nate Cole, and Nahappan 
> Somasundaram.
> 
> 
> Bugs: AMBARI-15915
> https://issues.apache.org/jira/browse/AMBARI-15915
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Inlined alter table statements (except one to overcome a circular FK 
> dependency)
> 2. Reorganized create table statemens to avoid forward references
> 3. Named all unnamed constraints, shortened names to fit Oracle's 30 
> character limit
> 4. Unified constraint names across database flavors
> 5. Wrote a JUnit test that checks DDL's for consistency (all constraints are 
> named, the same tables and constraints are defined for all database flavors)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql f90ac96 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 2b214c4 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql fc93372 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 870a8e8 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 71d4813 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 6e600c7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql a3ea10d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTestUtils.java 
> PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTests.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46269/diff/
> 
> 
> Testing
> ---
> 
> 1. Run all java unit tests in ambari-server. All passed, except 
> ViewDirectoryWatcherTest.testDirectoryExtractionOnFileAdd(). Failure occured 
> to a hardcoded timeout value in the test being to low for my environment (I 
> raised the timeout from 7 secs to 20 and the test passed)
> 2. Run DDL's with their respective DB environment. In cases where the DDL 
> contained additional things other than table/index cration and insertion 
> (e.g. creating databases and users) I only run the table/index creation part.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Review Request 46272: AMBARI-15914 Ranger stack changes to remove audit to db config params

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46272/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Jonathan Hurley, 
Jayush Luniya, and Velmurugan Periasamy.


Bugs: AMBARI-15914
https://issues.apache.org/jira/browse/AMBARI-15914


Repository: ambari


Description
---

In order to remove feature of Audit to DB in Ranger, need to remove Audit to DB 
related configs from Ambari stacks.
For fresh installs, audit to db configs should not be shown.

Upgrade scenario will be handle in AMBARI-15916.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/constants.py
 86dbc48 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
 7201c36 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py
 c66a0e9 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 6949ced 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 03fd40c 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/params.py
 8d483dc 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/params_linux.py
 5c65a1e 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 c0ab3e8 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 fc722da 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/params_linux.py
 101ca46 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 5c37e4b 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
 03b7e01 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/configuration/ranger-hbase-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/configuration/ranger-hdfs-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/ranger-hive-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/KAFKA/configuration/ranger-kafka-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/configuration/ranger-knox-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/admin-properties.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-admin-site.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/configuration/ranger-kms-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/ranger-storm-audit.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/ranger-yarn-audit.xml
 PRE-CREATION 

Diff: https://reviews.apache.org/r/46272/diff/


Testing
---

Tested installation of Ranger on centos6 and checked audit to db properties are 
not getting reflected under Ranger Configs and components supporting Ranger 
plugin.


Thanks,

Mugdha Varadkar

Re: Review Request 46210: Services status check labels and progress bar changes for Hive, Pig and Files view

[Pallav Kulshreshtha]

> On April 14, 2016, 7:21 p.m., DIPAYAN BHOWMICK wrote:
> > contrib/views/files/src/main/resources/ui/app/controllers/splash.js, line 98
> > 
> >
> > allTestsCompleted property and modelhdfsTestDone are exactly same. What 
> > is the requirement of introducing another computed property?

so, as of now there only one check for file browser view but going forward it 
may have multiple checks similar to hive and ping views. that's when we do not 
need to change the code on HTML.


- Pallav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46210/#review128958
---


On April 14, 2016, 3:23 p.m., Pallav Kulshreshtha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46210/
> ---
> 
> (Updated April 14, 2016, 3:23 p.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, and Nitiraj 
> Rathore.
> 
> 
> Bugs: AMBARI-15886
> https://issues.apache.org/jira/browse/AMBARI-15886
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - add a function for checking if all the serivce checks are completed.
> - using this function on UI to show the correct label and progress bar status.
> 
> 
> Diffs
> -
> 
>   contrib/views/files/src/main/resources/ui/app/controllers/splash.js 84a404a 
>   contrib/views/files/src/main/resources/ui/app/routes/splash.js 6005937 
>   contrib/views/files/src/main/resources/ui/app/templates/splash.hbs 72406a0 
>   contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/splash.js 
> eec1f17 
>   contrib/views/hive/src/main/resources/ui/hive-web/app/routes/splash.js 
> 1ff155c 
>   contrib/views/hive/src/main/resources/ui/hive-web/app/templates/splash.hbs 
> 3e81a9c 
>   contrib/views/pig/src/main/resources/ui/pig-web/app/controllers/splash.js 
> 959d6d0 
>   contrib/views/pig/src/main/resources/ui/pig-web/app/routes/splash.js 
> 5223a5a 
>   contrib/views/pig/src/main/resources/ui/pig-web/app/templates/splash.hbs 
> 7455a4f 
> 
> Diff: https://reviews.apache.org/r/46210/diff/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> 
> Thanks,
> 
> Pallav Kulshreshtha
> 
>

Re: Review Request 46269: SQL constraints: Inline constraints and name them in CREATE table

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46269/#review129118
---




ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql (line 1326)


Quartz tables are not ours and I don't know the implications of changing 
them. Let's stick to our own.


- Jonathan Hurley


On April 15, 2016, 9:41 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46269/
> ---
> 
> (Updated April 15, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, Jonathan Robie, Nate Cole, and Nahappan 
> Somasundaram.
> 
> 
> Bugs: AMBARI-15915
> https://issues.apache.org/jira/browse/AMBARI-15915
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Inlined alter table statements (except one to overcome a circular FK 
> dependency)
> 2. Reorganized create table statemens to avoid forward references
> 3. Named all unnamed constraints, shortened names to fit Oracle's 30 
> character limit
> 4. Unified constraint names across database flavors
> 5. Wrote a JUnit test that checks DDL's for consistency (all constraints are 
> named, the same tables and constraints are defined for all database flavors)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql f90ac96 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 2b214c4 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql fc93372 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 870a8e8 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 71d4813 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 6e600c7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql a3ea10d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTestUtils.java 
> PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTests.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46269/diff/
> 
> 
> Testing
> ---
> 
> 1. Run all java unit tests in ambari-server. All passed, except 
> ViewDirectoryWatcherTest.testDirectoryExtractionOnFileAdd(). Failure occured 
> to a hardcoded timeout value in the test being to low for my environment (I 
> raised the timeout from 7 secs to 20 and the test passed)
> 2. Run DDL's with their respective DB environment. In cases where the DDL 
> contained additional things other than table/index cration and insertion 
> (e.g. creating databases and users) I only run the table/index creation part.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 46269: SQL constraints: Inline constraints and name them in CREATE table

[Jonathan Hurley]

> On April 15, 2016, 9:52 a.m., Nate Cole wrote:
> > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql, lines 181-184
> > 
> >
> > Let's be consistent.  Either:
> > pk_sc_desired_state
> > fk_scds_...
> > unq_scdesired...
> > 
> > or
> > PK_sc_desired
> > FK_scds...
> > UNQ_scdesired...
> > 
> > I seem recall issues of case between the sql files and UpgradeCatalog* 
> > classes.  Jonathan?

I'm all for consistency. You're right that we used to have problems with the 
case of the contraints. For PK's, we added some utility methods which don't 
require us to know the name anymore:
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessor.java#L569-L581

We could do something similar for FKs as well.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46269/#review129114
---


On April 15, 2016, 9:41 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46269/
> ---
> 
> (Updated April 15, 2016, 9:41 a.m.)
> 
> 
> Review request for Ambari, Jonathan Robie, Nate Cole, and Nahappan 
> Somasundaram.
> 
> 
> Bugs: AMBARI-15915
> https://issues.apache.org/jira/browse/AMBARI-15915
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Inlined alter table statements (except one to overcome a circular FK 
> dependency)
> 2. Reorganized create table statemens to avoid forward references
> 3. Named all unnamed constraints, shortened names to fit Oracle's 30 
> character limit
> 4. Unified constraint names across database flavors
> 5. Wrote a JUnit test that checks DDL's for consistency (all constraints are 
> named, the same tables and constraints are defined for all database flavors)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql f90ac96 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 2b214c4 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql fc93372 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 870a8e8 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 71d4813 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 6e600c7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql a3ea10d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTestUtils.java 
> PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTests.java 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46269/diff/
> 
> 
> Testing
> ---
> 
> 1. Run all java unit tests in ambari-server. All passed, except 
> ViewDirectoryWatcherTest.testDirectoryExtractionOnFileAdd(). Failure occured 
> to a hardcoded timeout value in the test being to low for my environment (I 
> raised the timeout from 7 secs to 20 and the test passed)
> 2. Run DDL's with their respective DB environment. In cases where the DDL 
> contained additional things other than table/index cration and insertion 
> (e.g. creating databases and users) I only run the table/index creation part.
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 46270: AMBARI-15917 : Add ability to specify a custom trust store for ranger admin via ambari config

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46270/#review129116
---



Does anything need to happen for existing HDP-2.3 deployments to get this value 
(should Ambari upgrade set them)?  Any RU/EU considerations?

- Nate Cole


On April 15, 2016, 9:51 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46270/
> ---
> 
> (Updated April 15, 2016, 9:51 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Mahadev Konar, 
> Nate Cole, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-15917
> https://issues.apache.org/jira/browse/AMBARI-15917
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Expose custom trust store related configs for Ranger on Ambari UI.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/configuration/ranger-admin-site.xml
>  b358f27 
> 
> Diff: https://reviews.apache.org/r/46270/diff/
> 
> 
> Testing
> ---
> 
> Verified addition of : ranger.truststore.file and ranger.truststore.password 
> in ranger-admin-site.xml.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Review Request 46270: AMBARI-15917 : Add ability to specify a custom trust store for ranger admin via ambari config

[Gautam Borad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46270/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Mahadev Konar, 
Nate Cole, Sumit Mohanty, and Velmurugan Periasamy.


Bugs: AMBARI-15917
https://issues.apache.org/jira/browse/AMBARI-15917


Repository: ambari


Description
---

Expose custom trust store related configs for Ranger on Ambari UI.


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/configuration/ranger-admin-site.xml
 b358f27 

Diff: https://reviews.apache.org/r/46270/diff/


Testing
---

Verified addition of : ranger.truststore.file and ranger.truststore.password in 
ranger-admin-site.xml.


Thanks,

Gautam Borad

Review Request 46269: SQL constraints: Inline constraints and name them in CREATE table

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46269/
---

Review request for Ambari, Jonathan Robie, Nate Cole, and Nahappan Somasundaram.


Bugs: AMBARI-15915
https://issues.apache.org/jira/browse/AMBARI-15915


Repository: ambari


Description
---

1. Inlined alter table statements (except one to overcome a circular FK 
dependency)
2. Reorganized create table statemens to avoid forward references
3. Named all unnamed constraints, shortened names to fit Oracle's 30 character 
limit
4. Unified constraint names across database flavors
5. Wrote a JUnit test that checks DDL's for consistency (all constraints are 
named, the same tables and constraints are defined for all database flavors)


Diffs
-

  ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql f90ac96 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 2b214c4 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql fc93372 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 870a8e8 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
71d4813 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 6e600c7 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql a3ea10d 
  ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTestUtils.java 
PRE-CREATION 
  ambari-server/src/test/java/org/apache/ambari/server/orm/db/DDLTests.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/46269/diff/


Testing
---

1. Run all java unit tests in ambari-server. All passed, except 
ViewDirectoryWatcherTest.testDirectoryExtractionOnFileAdd(). Failure occured to 
a hardcoded timeout value in the test being to low for my environment (I raised 
the timeout from 7 secs to 20 and the test passed)
2. Run DDL's with their respective DB environment. In cases where the DDL 
contained additional things other than table/index cration and insertion (e.g. 
creating databases and users) I only run the table/index creation part.


Thanks,

Balázs Bence Sári

Re: Review Request 46247: AMBARI-15903 Script + Server alert display, descriptions and ui

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46247/#review129112
---


Ship it!




Ship It!

- Jonathan Hurley


On April 14, 2016, 10:31 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46247/
> ---
> 
> (Updated April 14, 2016, 10:31 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Jonathan Hurley, 
> Oleg Nechiporenko, Richard Zang, Srimanth Gunturi, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15903
> https://issues.apache.org/jira/browse/AMBARI-15903
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1) Some are showing params that should be hidden
> 2) Some need new display labels
> 3) Some need hover tooltip description text
> 4) Some need alert description tweaks
> 5) In the hive alerts, some of those params, missing thresholds
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/alerts.json 0bbd004 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
> 4dc52a2 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/alerts.json 
> 1f86e57 
>   ambari-web/app/controllers/main/alerts/definition_configs_controller.js 
> 9dc4f92 
>   ambari-web/app/models/alerts/alert_definition.js 245771e 
> 
> Diff: https://reviews.apache.org/r/46247/diff/
> 
> 
> Testing
> ---
> 
> ambari-server:
> All alert unit test passed.
> mvn clean test -Dtest=Alert* -Drat.ignoreErrors -DfailIfNoTests=false
> ambari-web:
> All UT passed
> Manual testing done.
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Re: Review Request 46226: AMBARI-15899 Alert Check Retries: UX edits v1

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46226/#review129109
---


Ship it!





ambari-web/app/messages.js (line 1013)


99 :) ... you'll never see any alerts


- Jonathan Hurley


On April 14, 2016, 8:14 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46226/
> ---
> 
> (Updated April 14, 2016, 8:14 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Jonathan Hurley, 
> Oleg Nechiporenko, Richard Zang, Srimanth Gunturi, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15899
> https://issues.apache.org/jira/browse/AMBARI-15899
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Change the way to present the relation between global and individual alert 
> check.
> Other UX edits.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/config.js 06afa49 
>   ambari-web/app/controllers/global/cluster_controller.js 327bd45 
>   ambari-web/app/controllers/global/update_controller.js e13fe41 
>   
> ambari-web/app/controllers/main/alerts/alert_definitions_actions_controller.js
>  1712b6f 
>   ambari-web/app/controllers/main/alerts/definition_details_controller.js 
> 8e716bc 
>   ambari-web/app/messages.js 14c957c 
>   ambari-web/app/styles/alerts.less 24b29b2 
>   ambari-web/app/styles/application.less 61ccbfa 
>   ambari-web/app/templates/common/modal_popups/prompt_popup.hbs e1e03fb 
>   ambari-web/app/templates/main/alerts/definition_details.hbs fab41c7 
>   ambari-web/app/views/main/alerts/definition_details_view.js 606c955 
>   ambari-web/test/controllers/global/update_controller_test.js 2c679c6 
>   ambari-web/test/views/main/alerts/definition_details_view_test.js e6b88b6 
> 
> Diff: https://reviews.apache.org/r/46226/diff/
> 
> 
> Testing
> ---
> 
> Modified unit tests.
> Local ambari-web test passed.
> 25683 tests complete (24 seconds)
> 154 tests pending
> Manual testing done.
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Re: Review Request 46221: RU/EU: Unable to start upgrade with host in MM

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46221/#review129107
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/checks/ServicesUpCheck.java
 (line 158)


I think SLAVE_THRESHOLD should be configurable in the UP.



ambari-server/src/main/java/org/apache/ambari/server/checks/ServicesUpCheck.java
 (line 206)


isConsideredDown?


- Nate Cole


On April 14, 2016, 4:13 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46221/
> ---
> 
> (Updated April 14, 2016, 4:13 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-15896
> https://issues.apache.org/jira/browse/AMBARI-15896
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> # Install cluster (e.g HDP  2.3.0.0-2557). One of the hosts should consists 
> from only slaves and clients components (including Phoenix query server)
> # Register and install new stack version (e.g 2.4.2.0-195)
> # Stop ambari-agent on host without masters 
> # Turn on Maintenance Mode for this host
> # Try to start upgrade.
> Result: PHOENIX_QUERY_SERVER is in UNKNOWN state on Maintenance Mode host is 
> preventing upgrade.
> 
> The problem here is that the ServicesUpCheck does not take MM into account.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/ServicesUpCheck.java
>  09ad55d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
>  2153113 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/models/HostComponentSummary.java
>  c79ec8a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  caab7df 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesUpCheckTest.java
>  88826a0 
> 
> Diff: https://reviews.apache.org/r/46221/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 46267: Handle database connection properties upgrade for Hive and Oozie.

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46267/#review129104
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On April 15, 2016, 11:53 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46267/
> ---
> 
> (Updated April 15, 2016, 11:53 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-15912
> https://issues.apache.org/jira/browse/AMBARI-15912
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Removed properties:
> 
>   * hive_hostname
>   * oozie_hostname
>   * oozie_derby_database
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  1433a1b 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  ddaff84 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  834953b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  cfb57ab 
> 
> Diff: https://reviews.apache.org/r/46267/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 46267: Handle database connection properties upgrade for Hive and Oozie.

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46267/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-15912
https://issues.apache.org/jira/browse/AMBARI-15912


Repository: ambari


Description
---

Removed properties:

  * hive_hostname
  * oozie_hostname
  * oozie_derby_database


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 1433a1b 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 ddaff84 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 834953b 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 cfb57ab 

Diff: https://reviews.apache.org/r/46267/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 46196: Logsearch: reading config values from external property files

[Oliver Szabo]

> On April 15, 2016, 9:06 a.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2,
> >  line 1
> > 
> >
> > I don't think this will work as the format of the document is really 
> > XML and embedding an XML markup data as an XML value wouldnt work. You will 
> > need to either add CDATA to the document or encode it if you want to use it 
> > as plain text. Better to add support for XML_PROPERTY_FILE_TYPE so that we 
> > can add logic to encode the property xml contents.
> > 
> > You can verify this by starting ambari server and noticing that this 
> > config type will fail to show up in the STACKS API.
> 
> Oliver Szabo wrote:
> I tried that, did not failed and i could install a cluster with my 
> configurations. CDATA is not the best option, because there are CDATA tags in 
> that xml too, so its closing tag closed the initial CDATA tag (i mean if I 
> tried that in the *-site.xml )

of course that format can cause issues with e.g. blueprint export so i should 
notice that


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46196/#review129084
---


On April 14, 2016, 11:47 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46196/
> ---
> 
> (Updated April 14, 2016, 11:47 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15842
> https://issues.apache.org/jira/browse/AMBARI-15842
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Reading config values (like xml files or *.sh scripts) from external property 
> files.
> 
> I did not added xml support for reading external property files. Reason is 
> because these xml files are jinja templates. So they can act as text files.
> 
> also added: Solr Instance renamed to LogSearch Solr Instance
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  94ed27d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-log4j.xml
>  91fc0ce 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-audit_logs-solrconfig.xml
>  4da16b1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  ef6bebf 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-log4j.xml
>  6741f41 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-service_logs-solrconfig.xml
>  60363c8 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  855f0e5 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-log4j.xml
>  e99cb81 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-xml.xml
>  d31d0b3 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml 
> 018398f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-solr-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr.xml.j2
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46196/diff/
> 
> 
> Testing
> ---
> 
> Total run:959
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 46196: Logsearch: reading config values from external property files

[Oliver Szabo]

> On April 15, 2016, 9:06 a.m., Jayush Luniya wrote:
> > ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2,
> >  line 1
> > 
> >
> > I don't think this will work as the format of the document is really 
> > XML and embedding an XML markup data as an XML value wouldnt work. You will 
> > need to either add CDATA to the document or encode it if you want to use it 
> > as plain text. Better to add support for XML_PROPERTY_FILE_TYPE so that we 
> > can add logic to encode the property xml contents.
> > 
> > You can verify this by starting ambari server and noticing that this 
> > config type will fail to show up in the STACKS API.

I tried that, did not failed and i could install a cluster with my 
configurations. CDATA is not the best option, because there are CDATA tags in 
that xml too, so its closing tag closed the initial CDATA tag (i mean if I 
tried that in the *-site.xml )


- Oliver


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46196/#review129084
---


On April 14, 2016, 11:47 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46196/
> ---
> 
> (Updated April 14, 2016, 11:47 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15842
> https://issues.apache.org/jira/browse/AMBARI-15842
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Reading config values (like xml files or *.sh scripts) from external property 
> files.
> 
> I did not added xml support for reading external property files. Reason is 
> because these xml files are jinja templates. So they can act as text files.
> 
> also added: Solr Instance renamed to LogSearch Solr Instance
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  94ed27d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-log4j.xml
>  91fc0ce 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-audit_logs-solrconfig.xml
>  4da16b1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  ef6bebf 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-log4j.xml
>  6741f41 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-service_logs-solrconfig.xml
>  60363c8 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  855f0e5 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-log4j.xml
>  e99cb81 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-xml.xml
>  d31d0b3 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml 
> 018398f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-solr-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr.xml.j2
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46196/diff/
> 
> 
> Testing
> ---
> 
> Total run:959
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 46196: Logsearch: reading config values from external property files

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46196/#review129084
---




ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
 (line 1)


I don't think this will work as the format of the document is really XML 
and embedding an XML markup data as an XML value wouldnt work. You will need to 
either add CDATA to the document or encode it if you want to use it as plain 
text. Better to add support for XML_PROPERTY_FILE_TYPE so that we can add logic 
to encode the property xml contents.

You can verify this by starting ambari server and noticing that this config 
type will fail to show up in the STACKS API.


- Jayush Luniya


On April 14, 2016, 11:47 a.m., Oliver Szabo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46196/
> ---
> 
> (Updated April 14, 2016, 11:47 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15842
> https://issues.apache.org/jira/browse/AMBARI-15842
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Reading config values (like xml files or *.sh scripts) from external property 
> files.
> 
> I did not added xml support for reading external property files. Reason is 
> because these xml files are jinja templates. So they can act as text files.
> 
> also added: Solr Instance renamed to LogSearch Solr Instance
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-env.xml
>  94ed27d 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logfeeder-log4j.xml
>  91fc0ce 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-audit_logs-solrconfig.xml
>  4da16b1 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-env.xml
>  ef6bebf 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-log4j.xml
>  6741f41 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-service_logs-solrconfig.xml
>  60363c8 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-env.xml
>  855f0e5 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-log4j.xml
>  e99cb81 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/configuration/logsearch-solr-xml.xml
>  d31d0b3 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/metainfo.xml 
> 018398f 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/audit_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logfeeder-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-log4j.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/logsearch-solr-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/service_logs-solrconfig.xml.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr-log4j.properties.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/properties/solr.xml.j2
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46196/diff/
> 
> 
> Testing
> ---
> 
> Total run:959
> Total errors:0
> Total failures:0
> 
> 
> Thanks,
> 
> Oliver Szabo
> 
>

Re: Review Request 46238: Refactor Metadata manager for supporting distributed collector

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46238/#review129079
---




ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
 


It'll increase number of metrics sent from sinks to ams collector, so 
increase the collector load


- Dmytro Sen


On Апрель 14, 2016, 11:54 п.п., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46238/
> ---
> 
> (Updated Апрель 14, 2016, 11:54 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15902
> https://issues.apache.org/jira/browse/AMBARI-15902
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Presently metadata is served from a write through cache, we need a bounded 
> cache that gets refreshed periodically to allow sharded writes to succeed.
> 
> Additionally the metadata cache needs ability to skip certain metrics, 
> example: YARN Container metrics with too many unique names using a config.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  5b517ef 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataManager.java
>  8e58203 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TimelineMetricMetadataSync.java
>  54ea200 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  bf887be 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataSync.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  ddaff84 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  e5758bf 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
>  6f32000 
> 
> Diff: https://reviews.apache.org/r/46238/diff/
> 
> 
> Testing
> ---
> 
> All metrics Unit tests passed.
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 46183: AMBARI-15876. Stack advisor work for adding and removing LLAP queue when Interactive query is turned on and off respectively.

[Swapan Shridhar]

> On April 14, 2016, 6:03 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml,
> >  line 360
> > 
> >
> > Is this correct, cluster_name?

Yes. Suggested by Sid.


> On April 14, 2016, 6:03 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml,
> >  line 28
> > 
> >
> > This means that ambari upgrade will not add this property, but because 
> > it is new, then stack upgrade from HDP 2.4 to 2.5 will add it. Is that 
> > correct?
> > DOes it need a value?

It doesn't need a value. Used for the reaosn that initially capacity-scheduler 
reads props from its xml and later in lifecycle Ambari UI and backend talks in 
terms of all properties coalesced as a bag of string 
(key=value\nkey=value\nkey=value\n). So, this becomes a placeholder for 
setting values at backend and be passed back to UI.


- Swapan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46183/#review128936
---


On April 14, 2016, 9:01 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46183/
> ---
> 
> (Updated April 14, 2016, 9:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-15876
> https://issues.apache.org/jira/browse/AMBARI-15876
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - When Hive Server Interactive is turned ON, Ambari will create 'llap' named 
> YARN queue if only 'default' queue exists at root level and is consuming 100% 
> capacity.
> - When Hive Server Interactive is turned OFF, Ambari will delete 'llap' queue 
> only if 2 queues exist at root level namely, 'llap' and 'default'.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-env.xml
>  d518c8b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-interactive-site.xml
>  ba46f00 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/configuration/capacity-scheduler.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 7407da3 
>   ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 
> c21bad7 
> 
> Diff: https://reviews.apache.org/r/46183/diff/
> 
> 
> Testing
> ---
> 
> - Python UT passes.
> - Manual testing done. Various scenarios tested.
> - Test cases added to test the creation and deletion logic.
> 
> - Screenshots attached.
> 
> 
> File Attachments
> 
> 
> LLAP queue created
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/4304fa89-fcfa-47b2-9240-d4ffa221cb76__Screen_Shot_2016-04-13_at_4.51.35_PM.png
> HSI On and llap and default queues shown in dropdown
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/622db04f-ec39-4a63-920d-dffc3a269917__Screen_Shot_2016-04-13_at_4.51.03_PM.png
> YARn queue manager with llap and default queues 
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/14/29ee3e5c-9c49-4e73-be39-46e6ec786d21__Screen_Shot_2016-04-13_at_4.48.46_PM.png
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 46217: After adding support for real jdbc jar names, verify the post Ambari upgrade nothing breaks

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46217/#review129073
---


Ship it!




Ship It!

- Dmytro Sen


On Апрель 14, 2016, 6:40 п.п., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46217/
> ---
> 
> (Updated Апрель 14, 2016, 6:40 п.п.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-15894
> https://issues.apache.org/jira/browse/AMBARI-15894
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Lets think through the upgrade scenario again:
> Prior to upgrade we know of hard-coded names and the system is in working 
> state
> During upgrade, lets pre-populate the ambari.properties file with the old 
> hard-coded values (think of them as the real names as thats what we know)
> If user wants to use real name, they can call ambari-server setup ...
> Net goal is to not have user do anything post Ambari upgrade.
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/ambari.properties 1830d18 
> 
> Diff: https://reviews.apache.org/r/46217/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 46263: AMBARI-15906 Make Ranger TagSync to be installed as a slave and optional

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46263/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Jonathan Hurley, 
Jayush Luniya, Mahadev Konar, and Velmurugan Periasamy.


Bugs: AMBARI-15906
https://issues.apache.org/jira/browse/AMBARI-15906


Repository: ambari


Description
---

In order to support blueprints to setup cluster without TagSync, make stack 
level changes to allow Ranger TagSync to be installed as a optional slave 
component.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
 4466671 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/ranger-tagsync-site.xml
 cfcab3b 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
9f96308 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
7407da3 

Diff: https://reviews.apache.org/r/46263/diff/


Testing
---

Tested on Centos 6 with and without RANGER_TAGSYNC.


Thanks,

Mugdha Varadkar