Re: Review Request 45391: HAWQ - Add option to enable or disabling exchanging keys during HAWQ start.

2016-03-28 Thread bhuvnesh chaudhary 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45391/
---

(Updated March 29, 2016, 5:54 a.m.)


Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and 
Oleksandr Diachenko.


Changes
---

Applied feedback


Bugs: AMBARI-15595
https://issues.apache.org/jira/browse/AMBARI-15595


Repository: ambari


Description
---

In HAWQ, SSH Keys are exchanged currently during every start of HAWQ Master. 
This patch introduces a flag to enable / disable exchanging keys during start.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
 fc5385f 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 d634fb2 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
 7c3a832 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
 9d8b32d 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 4dc3cc8 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py e98b253 
  ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json 99d43cb 
  ambari-web/app/data/HDP2.3/site_properties.js e706d40 

Diff: https://reviews.apache.org/r/45391/diff/


Testing
---

yes
bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-28 10:42:36,732 
- Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to 
false or is not available in hawq-env.xml
ok
test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
test_install_default (test_hawqsegment.TestHawqSegment) ... ok
test_start_default (test_hawqsegment.TestHawqSegment) ... ok
test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
test_install_default (test_hawqstandby.TestHawqStandby) ... ok
test_start_default (test_hawqstandby.TestHawqStandby) ... ok
test_stop_default (test_hawqstandby.TestHawqStandby) ... ok

--
Ran 27 tests in 0.175s

OK


Thanks,

bhuvnesh chaudhary

Re: Review Request 45391: HAWQ - Add option to enable or disabling exchanging keys during HAWQ start.

2016-03-28 Thread Matt 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45391/#review125766
---


Ship it!





ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
 (line 36)


Since this would appear as a checkbox on the configs, the description 
wouldn't show. The last time I checked, checkboxes do not show descriptions 
while hovering over the checkbox



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
 (line 39)


Enable/Disable (with no space before or after slash)



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
 (line 41)


or when HAWQ **components** are added to new hosts?

exchanged every time **during**  HAWQ Master start?



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
 (line 43)


The hawq_hosts_file is probably required only on HAWQMASTER before running 
custom action HAWQ Check.

Should we create/overwrite the file right before running HAWQ Check so that 
it has the most recent host list before running HAWQ Check.



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
 (line 46)


setup_passwordless_ssh is used only by HAWQMASTER.

You may move this function to hawqmaster.py if you are keen.



ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
 (line 60)


Do you think it would be better to provide the user the list of hosts on 
which ssh key exchange failed?


- Matt


On March 28, 2016, 10:42 a.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45391/
> ---
> 
> (Updated March 28, 2016, 10:42 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and 
> Oleksandr Diachenko.
> 
> 
> Bugs: AMBARI-15595
> https://issues.apache.org/jira/browse/AMBARI-15595
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In HAWQ, SSH Keys are exchanged currently during every start of HAWQ Master. 
> This patch introduces a flag to enable / disable exchanging keys during start.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
>  fc5385f 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d634fb2 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
>  7c3a832 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9d8b32d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  d3c9009 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 4dc3cc8 
>   ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json 99d43cb 
>   ambari-web/app/data/HDP2.3/site_properties.js e706d40 
> 
> Diff: https://reviews.apache.org/r/45391/diff/
> 
> 
> Testing
> ---
> 
> yes
> bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestH

Re: Review Request 45391: HAWQ - Add option to enable or disabling exchanging keys during HAWQ start.

2016-03-28 Thread bhuvnesh chaudhary 


> On March 28, 2016, 6:03 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py,
> >  line 313
> > 
> >
> > can use default("/configuration/hdfs-site/dfs.allow.truncate", None)

Thanks for the review Alejandro, will make the change.


- bhuvnesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45391/#review125690
---


On March 28, 2016, 5:42 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45391/
> ---
> 
> (Updated March 28, 2016, 5:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and 
> Oleksandr Diachenko.
> 
> 
> Bugs: AMBARI-15595
> https://issues.apache.org/jira/browse/AMBARI-15595
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In HAWQ, SSH Keys are exchanged currently during every start of HAWQ Master. 
> This patch introduces a flag to enable / disable exchanging keys during start.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
>  fc5385f 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d634fb2 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
>  7c3a832 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9d8b32d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  d3c9009 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 4dc3cc8 
>   ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json 99d43cb 
>   ambari-web/app/data/HDP2.3/site_properties.js e706d40 
> 
> Diff: https://reviews.apache.org/r/45391/diff/
> 
> 
> Testing
> ---
> 
> yes
> bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-28 
> 10:42:36,732 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys 
> is either set to false or is not available in hawq-env.xml
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> 
> --
> Ran 27 tests in 0.175s
> 
> OK
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Re: Review Request 45391: HAWQ - Add option to enable or disabling exchanging keys during HAWQ start.

2016-03-28 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45391/#review125690
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 (line 313)


can use default("/configuration/hdfs-site/dfs.allow.truncate", None)


- Alejandro Fernandez


On March 28, 2016, 5:42 p.m., bhuvnesh chaudhary wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45391/
> ---
> 
> (Updated March 28, 2016, 5:42 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and 
> Oleksandr Diachenko.
> 
> 
> Bugs: AMBARI-15595
> https://issues.apache.org/jira/browse/AMBARI-15595
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In HAWQ, SSH Keys are exchanged currently during every start of HAWQ Master. 
> This patch introduces a flag to enable / disable exchanging keys during start.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
>  fc5385f 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
>  d634fb2 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
>  7c3a832 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
>  9d8b32d 
>   
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
>  d3c9009 
>   ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 4dc3cc8 
>   ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json 99d43cb 
>   ambari-web/app/data/HDP2.3/site_properties.js e706d40 
> 
> Diff: https://reviews.apache.org/r/45391/diff/
> 
> 
> Testing
> ---
> 
> yes
> bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
> test_hawq_master_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_segment_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_hawq_standby_critical 
> (test_alert_component_status.TestAlertComponentStatus) ... ok
> test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_component_status.TestAlertComponentStatus) 
> ... ok
> test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... 
> ok
> test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
> test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
> test_install_default (test_hawqmaster.TestHawqMaster) ... ok
> test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-28 
> 10:42:36,732 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys 
> is either set to false or is not available in hawq-env.xml
> ok
> test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
> test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
> test_install_default (test_hawqsegment.TestHawqSegment) ... ok
> test_start_default (test_hawqsegment.TestHawqSegment) ... ok
> test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
> test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
> test_install_default (test_hawqstandby.TestHawqStandby) ... ok
> test_start_default (test_hawqstandby.TestHawqStandby) ... ok
> test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
> 
> --
> Ran 27 tests in 0.175s
> 
> OK
> 
> 
> Thanks,
> 
> bhuvnesh chaudhary
> 
>

Review Request 45391: HAWQ - Add option to enable or disabling exchanging keys during HAWQ start.

2016-03-28 Thread bhuvnesh chaudhary 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45391/
---

Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and 
Oleksandr Diachenko.


Bugs: AMBARI-15595
https://issues.apache.org/jira/browse/AMBARI-15595


Repository: ambari


Description
---

In HAWQ, SSH Keys are exchanged currently during every start of HAWQ Master. 
This patch introduces a flag to enable / disable exchanging keys during start.


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
 fc5385f 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py
 d634fb2 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py
 7c3a832 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py
 9d8b32d 
  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py
 d3c9009 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 4dc3cc8 
  ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json 99d43cb 
  ambari-web/app/data/HDP2.3/site_properties.js e706d40 

Diff: https://reviews.apache.org/r/45391/diff/


Testing
---

yes
bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-03-28 10:42:36,732 
- Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to 
false or is not available in hawq-env.xml
ok
test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
test_install_default (test_hawqsegment.TestHawqSegment) ... ok
test_start_default (test_hawqsegment.TestHawqSegment) ... ok
test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
test_install_default (test_hawqstandby.TestHawqStandby) ... ok
test_start_default (test_hawqstandby.TestHawqStandby) ... ok
test_stop_default (test_hawqstandby.TestHawqStandby) ... ok

--
Ran 27 tests in 0.175s

OK


Thanks,

bhuvnesh chaudhary