Re: Review Request 44528: Align PXF Alert description with other services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44528/#review122611 --- Ship it! Ship It! - Matt On March 8, 2016, 1:46 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44528/ > --- > > (Updated March 8, 2016, 1:46 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-15339 > https://issues.apache.org/jira/browse/AMBARI-15339 > > > Repository: ambari > > > Description > --- > > Align PXF Alert description with other services > > > Diffs > - > > ambari-server/src/main/resources/common-services/PXF/3.0.0/alerts.json > d32fc97 > > Diff: https://reviews.apache.org/r/44528/diff/ > > > Testing > --- > > Manually Done > > > Thanks, > > jun aoki > >
Re: Review Request 44576: Update sysctl value for HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44576/#review122754 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-check-env.xml (line 33) <https://reviews.apache.org/r/44576/#comment184849> Description says increase it to 5 Update the sysctl values for the below parameters as recommended by HAWQ configuration. kernel.shmmax - 5 net.ipv4.ip_local_port_range - 1025 65535 ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml (line 21) <https://reviews.apache.org/r/44576/#comment184850> Description says increase it to 5 Update the sysctl values for the below parameters as recommended by HAWQ configuration. kernel.shmmax - 5 net.ipv4.ip_local_port_range - 1025 65535 - Matt On March 9, 2016, 9:23 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44576/ > --- > > (Updated March 9, 2016, 9:23 a.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, > Matt, and Oleksandr Diachenko. > > > Bugs: AMBARI-15359 > https://issues.apache.org/jira/browse/AMBARI-15359 > > > Repository: ambari > > > Description > --- > > Update the sysctl values for the below parameters as recommended by HAWQ > configuration. > kernel.shmmax - 5 > net.ipv4.ip_local_port_range - 1025 65535 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-check-env.xml > 9501a6a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml > 3ca9e87 > > Diff: https://reviews.apache.org/r/44576/diff/ > > > Testing > --- > > yes, manually. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 44575: Enable bulk operation for HAWQ Segments and PXF on Hosts page
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44575/#review122755 --- Ship it! Ship It! - Matt On March 9, 2016, 9:52 a.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44575/ > --- > > (Updated March 9, 2016, 9:52 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Lav Jain, > and Matt. > > > Bugs: AMBARI-15358 > https://issues.apache.org/jira/browse/AMBARI-15358 > > > Repository: ambari > > > Description > --- > > Enable bulk operation for HAWQ Segments and PXF on Hosts page > > > Diffs > - > > ambari-web/app/messages.js 8e69dd0 > ambari-web/app/views/main/host/hosts_table_menu_view.js 5670814 > > Diff: https://reviews.apache.org/r/44575/diff/ > > > Testing > --- > > Yes, done > > > Thanks, > > Goutam Tadi > >
Re: Review Request 44576: Update sysctl value for HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44576/#review122762 --- Ship it! Ship It! - Matt On March 9, 2016, 10:09 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44576/ > --- > > (Updated March 9, 2016, 10:09 a.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, > Matt, and Oleksandr Diachenko. > > > Bugs: AMBARI-15359 > https://issues.apache.org/jira/browse/AMBARI-15359 > > > Repository: ambari > > > Description > --- > > Update the sysctl values for the below parameters as recommended by HAWQ > configuration. > kernel.shmmax - 10 > net.ipv4.ip_local_port_range - 1025 65535 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-check-env.xml > 9501a6a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml > 3ca9e87 > > Diff: https://reviews.apache.org/r/44576/diff/ > > > Testing > --- > > yes, manually. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 44542: Use pxf-service restart method when Restart action is called via Ambari
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44542/#review122791 --- Fix it, then Ship it! ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/pxf.py (line 62) <https://reviews.apache.org/r/44542/#comment184883> What happens now if PXF is in started state? Ideally when you do a restart: - if PXF is already in started state, do stop and start - if PXF is already in stopped state, do start - Matt On March 8, 2016, 8:32 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44542/ > --- > > (Updated March 8, 2016, 8:32 p.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, > Matt, and Oleksandr Diachenko. > > > Bugs: AMBARI-15341 > https://issues.apache.org/jira/browse/AMBARI-15341 > > > Repository: ambari > > > Description > --- > > This patch updates the method used to restart PXF. > We should use pxf-service restart method when Restart action is called via > Ambari. > Currently we use pxf-service stop followed by pxf-service start, due to which > if pxf-service is already stopped, stop command fails. > pxf-service restart takes care of not exiting with exit code 1 if pxf-service > is already stopped, and finishes successfully. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/pxf.py > 9a1eaf3 > > Diff: https://reviews.apache.org/r/44542/diff/ > > > Testing > --- > > yes. manually > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 44574: Rename Service Action 'HAWQ Check' to 'HAWQ Config Check'
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44574/#review122789 --- Ship it! Ship It! - Matt On March 9, 2016, 9:16 a.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44574/ > --- > > (Updated March 9, 2016, 9:16 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Lav Jain, > and Matt. > > > Bugs: AMBARI-15357 > https://issues.apache.org/jira/browse/AMBARI-15357 > > > Repository: ambari > > > Description > --- > > Rename Service Action 'HAWQ Check' to 'HAWQ Config Check' > > > Diffs > - > > ambari-web/app/messages.js 8e69dd0 > > Diff: https://reviews.apache.org/r/44574/diff/ > > > Testing > --- > > Yeah done. > > > Thanks, > > Goutam Tadi > >
Re: Review Request 44577: Update descriptions for configs - HAWQ and PXF
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44577/#review122882
---
Fix it, then Ship it!
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
(line 26)
<https://reviews.apache.org/r/44577/#comment184966>
From Ambari perspective, there is no default.
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
(line 53)
<https://reviews.apache.org/r/44577/#comment184967>
Same here. No default.
The UI logic sets the property based on host component.
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
(line 150)
<https://reviews.apache.org/r/44577/#comment184970>
Please ensure that you mention **Native** and **YARN** in the description
(refer the old description).
These are the values that are shown in the dropdown on the theme
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
(line 175)
<https://reviews.apache.org/r/44577/#comment184968>
Mention Native ('none')
Also mention **Resource Manager (hawq_global_rm_type)**. the display name
on the theme is **Resource Manager**
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
(line 188)
<https://reviews.apache.org/r/44577/#comment184969>
Mention Native ('none')
Also mention **Resource Manager (hawq_global_rm_type)**. the display name
on the theme is **Resource Manager**
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
(line 204)
<https://reviews.apache.org/r/44577/#comment184971>
Mention YARN ('yarn')
Also mention **Resource Manager (hawq_global_rm_type)**. the display name
on the theme is **Resource Manager**
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
(line 224)
<https://reviews.apache.org/r/44577/#comment184972>
Mention YARN ('yarn')
Also mention **Resource Manager (hawq_global_rm_type)**. the display name
on the theme is **Resource Manager**
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml
(line 232)
<https://reviews.apache.org/r/44577/#comment184974>
Some changes going on for this one.
Refer: https://reviews.apache.org/r/44576/diff/1#index_header
- Matt
On March 9, 2016, 9:24 a.m., Goutam Tadi wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44577/
> ---
>
> (Updated March 9, 2016, 9:24 a.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Lav Jain,
> and Matt.
>
>
> Bugs: AMBARI-15311
> https://issues.apache.org/jira/browse/AMBARI-15311
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Update descriptions for configs - HAWQ and PXF
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-check-env.xml
> 9501a6a
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
> e40caca
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-limits-env.xml
> cd3d329
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
> 474f509
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml
> 3ca9e87
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml
> 7518c6b
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/yarn-client.xml
> cd7e6be
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/configuration/pxf-profiles.xml
> c87ce17
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/configuration/pxf-public-classpath.xml
> 7b7e6c4
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/configuration/pxf-site.xml
> 20ca06b
>
> Diff: https://reviews.apache.org/r/44577/diff/
>
>
> Testing
> ---
>
> Yes, done
>
>
> Thanks,
>
> Goutam Tadi
>
>
Re: Review Request 44576: Update sysctl value for HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44576/#review122896 --- Since the changes proposed on this review are minor, it can be incorporated [here](https://reviews.apache.org/r/44577/), to avoid merge conflicts. - Matt On March 9, 2016, 10:09 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44576/ > --- > > (Updated March 9, 2016, 10:09 a.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, > Matt, and Oleksandr Diachenko. > > > Bugs: AMBARI-15359 > https://issues.apache.org/jira/browse/AMBARI-15359 > > > Repository: ambari > > > Description > --- > > Update the sysctl values for the below parameters as recommended by HAWQ > configuration. > kernel.shmmax - 10 > net.ipv4.ip_local_port_range - 1025 65535 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-check-env.xml > 9501a6a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml > 3ca9e87 > > Diff: https://reviews.apache.org/r/44576/diff/ > > > Testing > --- > > yes, manually. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 45124: Update logic for HAWQ 'output.replace-datanode-on-failure' property with threshold of 3
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45124/#review124632 --- Ship it! Ship It! - Matt On March 21, 2016, 12:29 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45124/ > --- > > (Updated March 21, 2016, 12:29 p.m.) > > > Review request for Ambari, bhuvnesh chaudhary, Goutam Tadi, and Lav Jain. > > > Bugs: AMBARI-15494 > https://issues.apache.org/jira/browse/AMBARI-15494 > > > Repository: ambari > > > Description > --- > > Update logic for HAWQ 'output.replace-datanode-on-failure' property with > threshold of 3 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hdfs-client.xml > 6754fcd > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > 171a6b6 > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > b5cf1a6 > > Diff: https://reviews.apache.org/r/45124/diff/ > > > Testing > --- > > Manual. > > -- > Ran 246 tests in 7.028s > OK > -- > Total run:911 > Total errors:0 > Total failures:0 > > > Thanks, > > Alexander Denissov > >
Re: Review Request 45188: update pxf-profile template with new fields and description
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45188/#review124949 --- Ship it! Ship It! - Matt On March 22, 2016, 4:43 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45188/ > --- > > (Updated March 22, 2016, 4:43 p.m.) > > > Review request for Ambari, bhuvnesh chaudhary, Goutam Tadi, Lav Jain, and > Matt. > > > Bugs: AMBARI-15522 > https://issues.apache.org/jira/browse/AMBARI-15522 > > > Repository: ambari > > > Description > --- > > update pxf-profile template with new fields and description > > > Diffs > - > > > ambari-server/src/main/resources/common-services/PXF/3.0.0/configuration/pxf-profiles.xml > 7ce0f71 > > Diff: https://reviews.apache.org/r/45188/diff/ > > > Testing > --- > > manual > > > Thanks, > > Alexander Denissov > >
Re: Review Request 45191: HAWQ - exchange keys should be done only from HAWQMASTER
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45191/#review125154 --- Ship it! Ship It! - Matt On March 23, 2016, 10:11 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45191/ > --- > > (Updated March 23, 2016, 10:11 a.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and > Oleksandr Diachenko. > > > Bugs: AMBARI-15524 > https://issues.apache.org/jira/browse/AMBARI-15524 > > > Repository: ambari > > > Description > --- > > HAWQ - exchange keys should be done only from HAWQMASTER. Currently, both > standby and master does exchange keys however, its not required be done twice > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 0631144 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > 2c3493a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py > 1891ede > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py > 0f52b9e > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 330b6c0 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 3907ad9 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 8049821 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 039d109 > > Diff: https://reviews.apache.org/r/45191/diff/ > > > Testing > --- > > yes. manually > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 45252: PXF service checks fails on secured cluster without YARN
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125170
---
ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
(line 154)
<https://reviews.apache.org/r/45252/#comment187958>
Does YARN have to be functional (green status), or just installed so that
libraries are available?
- Matt
On March 23, 2016, 3:23 p.m., Lav Jain wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> ---
>
> (Updated March 23, 2016, 3:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh
> chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton
> Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> ---
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True,
> 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs':
> 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local':
> '/usr/bin/kinit', 'principal_name': 'hdfs-...@ambari.apache.org', 'user':
> 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test:
> PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path
> [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos
> principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at
> org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at
> org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py
> eed2d18
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
> 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> ---
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45255: PXF should use hostname instead of localhost in pxf urls
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45255/#review125212 --- Ship it! ambari-server/src/main/resources/common-services/PXF/3.0.0/package/alerts/api_status.py (line 134) <https://reviews.apache.org/r/45255/#comment188000> Did anything break when it was localhost? Would be surprised if it caused any issue. - Matt On March 23, 2016, 5:24 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45255/ > --- > > (Updated March 23, 2016, 5:24 p.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jonathan Hurley, > Jayush Luniya, and Oleksandr Diachenko. > > > Bugs: AMBARI-15550 > https://issues.apache.org/jira/browse/AMBARI-15550 > > > Repository: ambari > > > Description > --- > > PXF should use actual hostname instead of localhost in pxf urls. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/alerts/api_status.py > 3134f41 > > Diff: https://reviews.apache.org/r/45255/diff/ > > > Testing > --- > > yes. manually > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 45252: PXF service checks fails on secured cluster without YARN
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45252/#review125388
---
Ship it!
Ship It!
- Matt
On March 23, 2016, 3:23 p.m., Lav Jain wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45252/
> ---
>
> (Updated March 23, 2016, 3:23 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, Alejandro Fernandez, bhuvnesh
> chaudhary, Goutam Tadi, Jaimin Jetly, jun aoki, Jayush Luniya, Matt, Newton
> Alex, and Oleksandr Diachenko.
>
>
> Bugs: AMBARI-15542
> https://issues.apache.org/jira/browse/AMBARI-15542
>
>
> Repository: ambari
>
>
> Description
> ---
>
> PXF service checks fails on secured cluster
> ```
> 2016-03-11 23:37:35,588 - HdfsResource[None] {'security_enabled': True,
> 'keytab': '/etc/security/keytabs/hdfs.headless.keytab', 'default_fs':
> 'hdfs://c6401.ambari.apache.org:8020', 'hdfs_site': ..., 'kinit_path_local':
> '/usr/bin/kinit', 'principal_name': 'hdfs-...@ambari.apache.org', 'user':
> 'hdfs', 'action': ['execute']}
> 2016-03-11 23:37:35,588 - Testing PXF HDFS read
> 2016-03-11 23:37:35,675 - Error code: 500
> 2016-03-11 23:37:35,675 - HDFS test Failed: Exception occurred in HDFS test:
> PXF data read failed: HTTP Error 500: Internal Server Error
> ```
>
> Pxf log says
> ```
> SEVERE: Servlet.service() for servlet [PXF REST Service] in context with path
> [/pxf] threw exception [java.io.IOException: Can't get Master Kerberos
> principal for use as renewer] with root cause
> java.io.IOException: Can't get Master Kerberos principal for use as renewer
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:116)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
> at
> org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:206)
> at
> org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:315)
> at
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getSplits(HdfsDataFragmenter.java:108)
> at
> org.apache.hawq.pxf.plugins.hdfs.HdfsDataFragmenter.getFragments(HdfsDataFragmenter.java:69)
> at
> org.apache.hawq.pxf.service.rest.FragmenterResource.getFragments(FragmenterResource.java:84)
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py
> eed2d18
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py
> 40e6d6f
>
> Diff: https://reviews.apache.org/r/45252/diff/
>
>
> Testing
> ---
>
> Tested manually with and without YARN, with and without security.
>
>
> Thanks,
>
> Lav Jain
>
>
Review Request 45343: Stack Advisor Recommendation overrides installed slave components topology
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45343/ --- Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Mahadev Konar, and Srimanth Gunturi. Bugs: AMBARI-15589 https://issues.apache.org/jira/browse/AMBARI-15589 Repository: ambari Description --- Stack Advisor recommendation overrides installed slaves topology of installed services when a new service is added. Steps to reproduce: - Install a cluster with HDFS and HAWQ select HAWQSEGMENTs on hosts different from the recommended set of hosts - Install any other service (HBASE). Error: On the assign slaves page, the HAWQSEGMENTs are checked according to recommendations and not according to the current host topology. The patch attached to this JIRA would fix the following issues: - Recommendation given during Add Service wizard should be a combination of current topology for installed services and recommendation for service that is to be installed. - Adding a new service after adding HAWQ or PXF would checkbox the HAWQSEGMENTs and PXFs correctly even if the user has placed these components different from the layout provided by the recommendation. Diffs - ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py cbcc4cb ambari-server/src/main/resources/stacks/stack_advisor.py 539bd25 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 6dd0b44 ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 8ae32a1 Diff: https://reviews.apache.org/r/45343/diff/ Testing --- Manually Tested. Ran unit tests on all stack advisors. Thanks, Matt
Re: Review Request 45391: HAWQ - Add option to enable or disabling exchanging keys during HAWQ start.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45391/#review125766 --- Ship it! ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml (line 36) <https://reviews.apache.org/r/45391/#comment188627> Since this would appear as a checkbox on the configs, the description wouldn't show. The last time I checked, checkboxes do not show descriptions while hovering over the checkbox ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml (line 39) <https://reviews.apache.org/r/45391/#comment188621> Enable/Disable (with no space before or after slash) ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml (line 41) <https://reviews.apache.org/r/45391/#comment188620> or when HAWQ **components** are added to new hosts? exchanged every time **during** HAWQ Master start? ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py (line 43) <https://reviews.apache.org/r/45391/#comment188628> The hawq_hosts_file is probably required only on HAWQMASTER before running custom action HAWQ Check. Should we create/overwrite the file right before running HAWQ Check so that it has the most recent host list before running HAWQ Check. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py (line 46) <https://reviews.apache.org/r/45391/#comment188626> setup_passwordless_ssh is used only by HAWQMASTER. You may move this function to hawqmaster.py if you are keen. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py (line 60) <https://reviews.apache.org/r/45391/#comment188625> Do you think it would be better to provide the user the list of hosts on which ssh key exchange failed? - Matt On March 28, 2016, 10:42 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45391/ > --- > > (Updated March 28, 2016, 10:42 a.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and > Oleksandr Diachenko. > > > Bugs: AMBARI-15595 > https://issues.apache.org/jira/browse/AMBARI-15595 > > > Repository: ambari > > > Description > --- > > In HAWQ, SSH Keys are exchanged currently during every start of HAWQ Master. > This patch introduces a flag to enable / disable exchanging keys during start. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml > fc5385f > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > d634fb2 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > 7c3a832 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 9d8b32d > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py > d3c9009 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 4dc3cc8 > ambari-server/src/test/python/stacks/2.3/configs/hawq_default.json 99d43cb > ambari-web/app/data/HDP2.3/site_properties.js e706d40 > > Diff: https://reviews.apache.org/r/45391/diff/ > > > Testing > --- > > yes > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncSta
Re: Review Request 45566: HAWQ config should not allow multiple Master/Segment directories
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45566/#review126532
---
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line
972)
<https://reviews.apache.org/r/45566/#comment189554>
Is trailing space valid or invalid?
If it is valid, you may want to use strip() before splitting
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py (line
2045)
<https://reviews.apache.org/r/45566/#comment189557>
This would never fail:
problems_dict is assigned {} and is never changed.
expected_warnings is assigned {} and is never changed.
Is this required?
ambari-web/app/views/common/configs/service_config_layout_tab_view.js (line 56)
<https://reviews.apache.org/r/45566/#comment189559>
Can you please confirm if the help text is displayed when you hover over
the directory box?
- Matt
On March 31, 2016, 3:09 p.m., Lav Jain wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45566/
> ---
>
> (Updated March 31, 2016, 3:09 p.m.)
>
>
> Review request for Ambari, Andriy Babiichuk, Alexander Denissov, Alexandr
> Antonenko, Andrii Tkach, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Matt,
> Newton Alex, Oleksandr Diachenko, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-15657
> https://issues.apache.org/jira/browse/AMBARI-15657
>
>
> Repository: ambari
>
>
> Description
> ---
>
> User can add multiple space delimited directories, but after installation, it
> shows a comma in between. however, only first directory goes into effect,
> that too with a comma in the end.
> ```
> [pivotal@ip-10-32-36-213 etc]$ cat hawq-site.xml
>
> hawq_master_directory
> /data/hawq/master,/data/hawq/master2
>
>
>
> hawq_segment_directory
> /data/hawq/segment,/data/hawq/segment2
>
> [pivotal@ip-10-32-36-213 etc]$ ls -l /data/hawq
> drwxr-xr-x 3 root root 4096 Mar 12 01:00 master,
> drwxr-xr-x 3 root root 4096 Mar 12 01:00 segment,
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
> ec7275f
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json
> a23afe6
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py
> 741011c
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py
> d415b6f
> ambari-web/app/messages.js ea06bf3
> ambari-web/app/mixins/common/serverValidator.js 99a5921
> ambari-web/app/models/configs/objects/service_config_property.js a437c98
> ambari-web/app/styles/application.less a6b79ce
>
> ambari-web/app/templates/common/modal_popups/config_recommendation_popup.hbs
> f58e086
> ambari-web/app/views/common/configs/service_config_layout_tab_view.js
> 694e883
>
> Diff: https://reviews.apache.org/r/45566/diff/
>
>
> Testing
> ---
>
> Manually tested the UI.
>
> Unit test added for stack advisor.
>
>
> Thanks,
>
> Lav Jain
>
>
Re: Review Request 45566: HAWQ config should not allow multiple Master/Segment directories
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45566/#review126669 --- Ship it! Ship It! - Matt On April 1, 2016, 2:39 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45566/ > --- > > (Updated April 1, 2016, 2:39 p.m.) > > > Review request for Ambari, Andriy Babiichuk, Alexander Denissov, Alexandr > Antonenko, Andrii Tkach, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Matt, > Newton Alex, Oleksandr Diachenko, and Srimanth Gunturi. > > > Bugs: AMBARI-15657 > https://issues.apache.org/jira/browse/AMBARI-15657 > > > Repository: ambari > > > Description > --- > > User can add multiple space delimited directories, but after installation, it > shows a comma in between. however, only first directory goes into effect, > that too with a comma in the end. > ``` > [pivotal@ip-10-32-36-213 etc]$ cat hawq-site.xml > > hawq_master_directory > /data/hawq/master,/data/hawq/master2 > > > > hawq_segment_directory > /data/hawq/segment,/data/hawq/segment2 > > [pivotal@ip-10-32-36-213 etc]$ ls -l /data/hawq > drwxr-xr-x 3 root root 4096 Mar 12 01:00 master, > drwxr-xr-x 3 root root 4096 Mar 12 01:00 segment, > ``` > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > ec7275f > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/themes/theme.json > a23afe6 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > 741011c > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > d415b6f > ambari-web/app/messages.js ea06bf3 > ambari-web/app/mixins/common/serverValidator.js 99a5921 > ambari-web/app/models/configs/objects/service_config_property.js a437c98 > ambari-web/app/styles/application.less a6b79ce > > ambari-web/app/templates/common/modal_popups/config_recommendation_popup.hbs > f58e086 > ambari-web/app/views/common/configs/service_config_layout_tab_view.js > 694e883 > > Diff: https://reviews.apache.org/r/45566/diff/ > > > Testing > --- > > Manually tested the UI. > > Unit test added for stack advisor. > > > Thanks, > > Lav Jain > >
Re: Review Request 45705: HAWQ segment percentage alert critical threshold should be 25%
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45705/#review127001 --- Ship it! Ship It! - Matt On April 4, 2016, 2:48 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45705/ > --- > > (Updated April 4, 2016, 2:48 p.m.) > > > Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam > Tadi, Lav Jain, Matt, Sumit Mohanty, and Srimanth Gunturi. > > > Bugs: AMBARI-15693 > https://issues.apache.org/jira/browse/AMBARI-15693 > > > Repository: ambari > > > Description > --- > > HAWQ segment percentage alert critical threshold should be 25% > > > Diffs > - > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/alerts.json > 8da5beb > ambari-web/app/views/main/dashboard/widgets.js 04be7bb > ambari-web/app/views/main/dashboard/widgets/hawqsegment_live.js c35d2b2 > > Diff: https://reviews.apache.org/r/45705/diff/ > > > Testing > --- > > manual, ran python tests: > > -- > Total run:933 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Alexander Denissov > >
Re: Review Request 45719: Alert to notify if a HAWQ Segment is unregistered with HAWQ Master
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45719/#review127002 --- Fix it, then Ship it! ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_segment_registration_status.py (line 58) <https://reviews.apache.org/r/45719/#comment190133> You may call it [Alert HAWQ] Would be great help for **grep**-pers out there. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_segment_registration_status.py (line 74) <https://reviews.apache.org/r/45719/#comment190135> else is not necessary, since the previous line has a return statement ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_segment_registration_status.py (line 77) <https://reviews.apache.org/r/45719/#comment190136> This message would be too long for the user to see on the Ambari UI. If I were adding this message, I would point them to gp_segment_configuration table since that is the originial source of information. Plus, the table has more information than the log message we dump into the alerts log. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_segment_registration_status.py (line 101) <https://reviews.apache.org/r/45719/#comment190134> You do not have to use strip() again - you have used strip in the previous line to remove trailing whitespaces. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_segment_registration_status.py (line 110) <https://reviews.apache.org/r/45719/#comment190132> Would the file be closed after execution of this line? Better to use 'with open(HAWQ_SLAVES_FILE) as f', so that you can ensure that the file is closed and frees up system resources. - Matt On April 4, 2016, 5:41 p.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45719/ > --- > > (Updated April 4, 2016, 5:41 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Lav Jain, > and Matt. > > > Bugs: AMBARI-15704 > https://issues.apache.org/jira/browse/AMBARI-15704 > > > Repository: ambari > > > Description > --- > > Include an alert informing the number of segments marked down in > gp_segment_configuration table > > > Diffs > - > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/alerts.json > 8da5beb > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_segment_registration_status.py > PRE-CREATION > > ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_segment_registration_status.py > PRE-CREATION > > Diff: https://reviews.apache.org/r/45719/diff/ > > > Testing > --- > > Test cases added. > > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > > > Thanks, > > Goutam Tadi > >
Re: Review Request 45705: HAWQ segment percentage alert critical threshold should be 25%
> On April 4, 2016, 7:17 p.m., Jayush Luniya wrote: > > ambari-web/app/views/main/dashboard/widgets.js, line 481 > > <https://reviews.apache.org/r/45705/diff/1/?file=1325063#file1325063line481> > > > > Surprised that this needs UI changes too. Why isnt alerts.json change > > sufficient for this? > > > > Also these values need documentation. Change to the alert.json is sufficient for the alert. We have a main dashboard widget that shows no. of HAWQ Segments Live (similar to Datanodes Live). The dashboard widget is independent of the alert. The change to the UI is to keep the thresholds of the dashboard widget similar to that of the alert thresholds so that users see the same color coding (Alert - CRITICAL RED when 25% or more segments are down AND Dashboard widget shows RED when less than 75% of segments are up) - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45705/#review127007 --- On April 4, 2016, 2:48 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45705/ > --- > > (Updated April 4, 2016, 2:48 p.m.) > > > Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam > Tadi, Lav Jain, Matt, Sumit Mohanty, and Srimanth Gunturi. > > > Bugs: AMBARI-15693 > https://issues.apache.org/jira/browse/AMBARI-15693 > > > Repository: ambari > > > Description > --- > > HAWQ segment percentage alert critical threshold should be 25% > > > Diffs > - > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/alerts.json > 8da5beb > ambari-web/app/views/main/dashboard/widgets.js 04be7bb > ambari-web/app/views/main/dashboard/widgets/hawqsegment_live.js c35d2b2 > > Diff: https://reviews.apache.org/r/45705/diff/ > > > Testing > --- > > manual, ran python tests: > > -- > Total run:933 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Alexander Denissov > >
Review Request 45724: Activate Standby option should not be shown on HAWQ Standby component on host summary page
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45724/ --- Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam Tadi, and Lav Jain. Bugs: AMBARI-15705 https://issues.apache.org/jira/browse/AMBARI-15705 Repository: ambari Description --- Activate Standby option should not be shown on HAWQ Standby component on host summary page Diffs - ambari-web/app/models/host_component.js 7a5805c Diff: https://reviews.apache.org/r/45724/diff/ Testing --- Manually Tested. Ran unit tests: 25609 tests complete (22 seconds) 154 tests pending [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 55.352 s [INFO] Finished at: 2016-04-04T19:35:54-07:00 [INFO] Final Memory: 12M/311M [INFO] Thanks, Matt
Re: Review Request 45826: Descriptions for HAWQ sysctl params mention incorrect default values
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45826/#review127390 --- Ship it! Ship It! - Matt On April 6, 2016, 11:04 a.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45826/ > --- > > (Updated April 6, 2016, 11:04 a.m.) > > > Review request for Ambari, bhuvnesh chaudhary, Goutam Tadi, Lav Jain, and > Matt. > > > Bugs: AMBARI-15738 > https://issues.apache.org/jira/browse/AMBARI-15738 > > > Repository: ambari > > > Description > --- > > Descriptions for HAWQ sysctl params mention incorrect default values > > kernel.shmmax and net.ipv4.ip_local_port_range have incorrect default values > in their descriptions > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-sysctl-env.xml > 365fa86 > > Diff: https://reviews.apache.org/r/45826/diff/ > > > Testing > --- > > Manual, tests ran OK > > -- > Ran 246 tests in 7.005s > > OK > -- > Total run:921 > Total errors:0 > Total failures:0 > > > Thanks, > > Alexander Denissov > >
Re: Review Request 45832: Add ignore-bad-hosts flag to hawq activate standby custom action
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45832/#review127562 --- Ship it! ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py (line 59) <https://reviews.apache.org/r/45832/#comment190934> Would a service restart be required afer bad hosts (which will get ignored) become 'good'? - Matt On April 6, 2016, 12:57 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45832/ > --- > > (Updated April 6, 2016, 12:57 p.m.) > > > Review request for Ambari, jun aoki, Matt, and Oleksandr Diachenko. > > > Bugs: AMBARI-15743 > https://issues.apache.org/jira/browse/AMBARI-15743 > > > Repository: ambari > > > Description > --- > > If a HAWQ Component host is down, HAWQ fails during activate standby > operation syncing configuration file to down hosts. > We should use the flag ignore-bad-hosts to exclude the hosts which are > currently marked down so that hawq activate commands does not syncs the file > on bad node. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py > 2e5dccf > > Diff: https://reviews.apache.org/r/45832/diff/ > > > Testing > --- > > yes manually. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 45878: Remove PHD references from HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45878/#review127717 --- Ship it! Ship It! - Matt On April 7, 2016, 9:48 a.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45878/ > --- > > (Updated April 7, 2016, 9:48 a.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-15741 > https://issues.apache.org/jira/browse/AMBARI-15741 > > > Repository: ambari > > > Description > --- > > Remove any reference to PHD (pivotal) stack in HAWQ codebase as it has been > obsoleted. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > 4edf64f > > Diff: https://reviews.apache.org/r/45878/diff/ > > > Testing > --- > > Python test result > --- > Ran 253 tests in 6.818s > > OK > -- > Total run:944 > Total errors:0 > Total failures:0 > OK > > > Thanks, > > Lav Jain > >
Re: Review Request 45899: After enabling NN HA with HAWQ installed, duplicate properties are seen on HAWQ config tab
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45899/#review127761 --- Ship it! Ship It! - Matt On April 7, 2016, 3:26 p.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45899/ > --- > > (Updated April 7, 2016, 3:26 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Lav Jain, > and Matt. > > > Bugs: AMBARI-15771 > https://issues.apache.org/jira/browse/AMBARI-15771 > > > Repository: ambari > > > Description > --- > > After enabling NN HA with HAWQ installed, duplicate properties are seen on > HAWQ config tab > > > Diffs > - > > > ambari-web/app/controllers/main/admin/highAvailability/nameNode/step9_controller.js > d004ba1 > > Diff: https://reviews.apache.org/r/45899/diff/ > > > Testing > --- > > Yes. > 10491 tests complete (11 seconds) > 121 tests pending > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 41.245 s > [INFO] Finished at: 2016-04-07T14:59:25-07:00 > [INFO] Final Memory: 15M/422M > [INFO] -- > > > Thanks, > > Goutam Tadi > >
Re: Review Request 45906: HAWQ fails to connect to YARN in HA+SEC mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45906/#review127824 --- Ship it! Ship It! - Matt On April 7, 2016, 4 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45906/ > --- > > (Updated April 7, 2016, 4 p.m.) > > > Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam > Tadi, Jayush Luniya, Lav Jain, Matt, Sumit Mohanty, and Srimanth Gunturi. > > > Bugs: AMBARI-15770 > https://issues.apache.org/jira/browse/AMBARI-15770 > > > Repository: ambari > > > Description > --- > > When HAWQ is in the cluster with YARN HA and Kerberos enabled, the segments > are down as they cannot connect to YARN RM. > > The following updates are required: > 1. add hadoop.security.authentication = kerberos in yarn-client.xml > > 2. add 2 properties to core-site.xml: > > hadoop.proxyuser.postgres.hosts > * > > > hadoop.proxyuser.postgres.groups > hadoop > > > 3. on every HAWQ node: > sudo useradd postgres > sudo usermod -a -G hadoop postgres > > > Diffs > - > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/kerberos.json > da11986 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 2ec4903 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py > 2a671da > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 5a5ee8d > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py c212b7d > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 1eb335e > > Diff: https://reviews.apache.org/r/45906/diff/ > > > Testing > --- > > Tested manually and updates python tests: > -- > Ran 246 tests in 7.061s > OK > -- > Total run:923 > Total errors:0 > Total failures:0 > > > Thanks, > > Alexander Denissov > >
Review Request 45990: Assign Slaves and Clients Page: Incorrect validation error shown
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45990/ --- Review request for Ambari, Jaimin Jetly and Mahadev Konar. Bugs: AMBARI-15783 https://issues.apache.org/jira/browse/AMBARI-15783 Repository: ambari Description --- A component with cardinality "ALL" should be recommended on all hosts by component recommendation API irrespective of the data posted with the API. Diffs - ambari-server/src/main/resources/stacks/stack_advisor.py 636de01 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py c577538 Diff: https://reviews.apache.org/r/45990/diff/ Testing --- Passed unit tests. Deploying a cluster to test manually... Will update with results soon... Thanks, Matt
Re: Review Request 45990: Assign Slaves and Clients Page: Incorrect validation error shown
> On April 11, 2016, 1:32 p.m., Mahadev Konar wrote: > > Matt - any update on the testing? I just deployed the new build with my code. The issue has been fixed. I was just about to update the review. - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45990/#review128242 --- On April 10, 2016, 1:37 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45990/ > --- > > (Updated April 10, 2016, 1:37 p.m.) > > > Review request for Ambari, Jaimin Jetly and Mahadev Konar. > > > Bugs: AMBARI-15783 > https://issues.apache.org/jira/browse/AMBARI-15783 > > > Repository: ambari > > > Description > --- > > A component with cardinality "ALL" should be recommended on all hosts by > component recommendation API irrespective of the data posted with the API. > > > Diffs > - > > ambari-server/src/main/resources/stacks/stack_advisor.py 636de01 > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > c577538 > > Diff: https://reviews.apache.org/r/45990/diff/ > > > Testing > --- > > Passed unit tests. > Deploying a cluster to test manually... Will update with results soon... > > > Thanks, > > Matt > >
Re: Review Request 45990: Assign Slaves and Clients Page: Incorrect validation error shown
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45990/ --- (Updated April 11, 2016, 2:17 p.m.) Review request for Ambari, Jaimin Jetly and Mahadev Konar. Changes --- Testing complete. Bugs: AMBARI-15783 https://issues.apache.org/jira/browse/AMBARI-15783 Repository: ambari Description --- A component with cardinality "ALL" should be recommended on all hosts by component recommendation API irrespective of the data posted with the API. Diffs - ambari-server/src/main/resources/stacks/stack_advisor.py 636de01 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py c577538 Diff: https://reviews.apache.org/r/45990/diff/ Testing (updated) --- Passed unit tests. Deploying a cluster for manual testing. There is no validation error regarding METRICS MONITOR after following the **steps to reproduce** as mentioned on the JIRA. Thanks, Matt
Re: Review Request 46062: PXF service check must occur only after PXF is started
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46062/#review128284 --- Ship it! Ship It! - Matt On April 11, 2016, 4:23 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46062/ > --- > > (Updated April 11, 2016, 4:23 p.m.) > > > Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, Jayush Luniya, Lav Jain, Matt, Sumit Mohanty, and Srimanth > Gunturi. > > > Bugs: AMBARI-15817 > https://issues.apache.org/jira/browse/AMBARI-15817 > > > Repository: ambari > > > Description > --- > > PXF service check must occur only after PXF is started > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json > b179d1c > > Diff: https://reviews.apache.org/r/46062/diff/ > > > Testing > --- > > Tested manually, ran tests: > -- > Ran 246 tests in 7.222s > OK > -- > Total run:925 > Total errors:0 > Total failures:0 > > > Thanks, > > Alexander Denissov > >
Re: Review Request 46072: HAWQ HDFS directory should be owned by postgres user if secured
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46072/#review128363 --- ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py <https://reviews.apache.org/r/46072/#comment191799> This was dead code? I don't see any other reference for **__setup_hdfs_dirs** on the left side ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py (line 190) <https://reviews.apache.org/r/46072/#comment191798> Configure calls HdfsResource twice (create_on_execute and execute). Start calls configure on line 188. Lines 190-217: Assert again HdfsResource is called twice(create_on_execute and execute). So in total 4 times for start and 2 times for configure? - Matt On April 11, 2016, 11:27 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46072/ > --- > > (Updated April 11, 2016, 11:27 p.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and > Oleksandr Diachenko. > > > Bugs: AMBARI-15820 > https://issues.apache.org/jira/browse/AMBARI-15820 > > > Repository: ambari > > > Description > --- > > HAWQ HDFS directory should be owned by postgres user if secured. Currently, > it is owned by gpadmin even when the cluster is secured. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > 98dedf1 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 87d3c8a > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 9017d24 > > Diff: https://reviews.apache.org/r/46072/diff/ > > > Testing > --- > > yes > > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-11 > 23:28:50,081 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys > is either set to false or is not available in hawq-env.xml > ok > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > test_configure_default (test_hawqse
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On March 2, 2016, 1:37 p.m., jun aoki wrote: > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py, > > line 32 > > <https://reviews.apache.org/r/44210/diff/1/?file=1275516#file1275516line32> > > > > Could this inheritate a brand new DefaultServiceAdvisor instead of the > > stack advisor? service and stack are different hierarchy and I feel somehow > > wrong if a service inheritate a stack. > > Tim Thorpe wrote: > Point well taken. I'll look at refactoring it that way but really would > prefer to get the code in the current way first. There are code changes to > the stack advisor files on a daily basis and the longer it waits the more > merges will be needed. I will already be refactoring the code to split the > test scripts to be on the service level rather than the stack level. I will > attempt to do both of these tasks at the same time. May be this should inherit from service_advisor for that service under common-services? - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review121720 --- On April 12, 2016, 10:33 a.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated April 12, 2016, 10:33 a.m.) > > > Review request for Ambari, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-agent/pom.xml c2c993f > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/scripts/stack_advisor.py cdd9acb > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/GANGLIA/service_advisor_BIGTOP08GANGLIA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HBASE/service_advisor_BIGTOP08HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HDFS/service_advisor_BIGTOP08HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/HIVE/service_advisor_BIGTOP08HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/OOZIE/service_advisor_BIGTOP08OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/service_advisor_BIGTOP08YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/ZOOKEEPER/service_advisor_BIGTOP08ZOOKEEPER.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/stack_advisor.py > 53591cd > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS/service_advisor_HDP206AMBARI_METRICS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/service_advisor_HDP206GANGLIA.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/service_advisor_HDP206HBASE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/service_advisor_HDP206HDFS.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/service_advisor_HDP206HIVE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/OOZIE/service_advisor_HDP206OOZIE.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/service_advisor_HDP206YARN.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/service_advisor_HDP206ZOOKEEPER.py > PRE-CREATION > ambari-server/src/main/resources/stack
Re: Review Request 46072: HAWQ HDFS directory should be owned by postgres user if secured
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46072/#review128584 --- Ship it! Ship It! - Matt On April 12, 2016, 12:12 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46072/ > --- > > (Updated April 12, 2016, 12:12 p.m.) > > > Review request for Ambari, Alejandro Fernandez, jun aoki, Jayush Luniya, and > Oleksandr Diachenko. > > > Bugs: AMBARI-15820 > https://issues.apache.org/jira/browse/AMBARI-15820 > > > Repository: ambari > > > Description > --- > > HAWQ HDFS directory should be owned by postgres user if secured. Currently, > it is owned by gpadmin even when the cluster is secured. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 665d18e > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 87d3c8a > > Diff: https://reviews.apache.org/r/46072/diff/ > > > Testing > --- > > yes > > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-11 > 23:28:50,081 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys > is either set to false or is not available in hawq-env.xml > ok > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > test_configure_default (test_hawqsegment.TestHawqSegment) ... ok > test_install_default (test_hawqsegment.TestHawqSegment) ... ok > test_start_default (test_hawqsegment.TestHawqSegment) ... ok > test_stop_default (test_hawqsegment.TestHawqSegment) ... ok > test_configure_default (test_hawqstandby.TestHawqStandby) ... ok > test_install_default (test_hawqstandby.TestHawqStandby) ... ok > test_start_default (test_hawqstandby.TestHawqStandby) ... ok > test_stop_default (test_hawqstandby.TestHawqStandby) ... ok > > -- > Ran 34 tests in 0.140s > > OK > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 46170: HAWQ Segment Registration Status Alert shows false alert
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46170/#review128771 --- Ship it! Ship It! - Matt On April 13, 2016, 2:55 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46170/ > --- > > (Updated April 13, 2016, 2:55 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, Matt, Newton Alex, Oleksandr Diachenko, and Srimanth Gunturi. > > > Bugs: AMBARI-15794 > https://issues.apache.org/jira/browse/AMBARI-15794 > > > Repository: ambari > > > Description > --- > > Steps to reproduce: > > Install HAWQ with HAWQSEGMENT on the same host as HAWQMASTER > Segment registration would show an alert saying that localhost is not > registered. > > localhost is returned by gp_segment_configuration for that HAWQSEGMENT, but > in the slaves file, there is no entry 'localhost' > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_segment_registration_status.py > 4d09763 > > Diff: https://reviews.apache.org/r/46170/diff/ > > > Testing > --- > > Manually tested. > > > Thanks, > > Lav Jain > >
Re: Review Request 46176: Update group for postgres proxy user to * from hadoop
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46176/#review128782 --- Ship it! Ship It! - Matt On April 13, 2016, 3:41 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46176/ > --- > > (Updated April 13, 2016, 3:41 p.m.) > > > Review request for Ambari, Alejandro Fernandez, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, Jayush Luniya, Lav Jain, Matt, Sumit Mohanty, and Srimanth > Gunturi. > > > Bugs: AMBARI-15770 > https://issues.apache.org/jira/browse/AMBARI-15770 > > > Repository: ambari > > > Description > --- > > Update group for postgres proxy user to * from hadoop > > > Diffs > - > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/kerberos.json > 088a244 > > Diff: https://reviews.apache.org/r/46176/diff/ > > > Testing > --- > > Manual. > > -- > Ran 260 tests in 7.220s > OK > -- > Total run:959 > Total errors:0 > Total failures:0 > > > Thanks, > > Alexander Denissov > >
Review Request 46181: Activate HAWQ standby wizard fails to delete previous hawq standby master
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46181/ --- Review request for Ambari and Alexander Denissov. Bugs: AMBARI-15849 https://issues.apache.org/jira/browse/AMBARI-15849 Repository: ambari Description --- Activate HAWQ standby wizard fails to delete previous hawq standby master. Currently, HAWQ standby status check is done based on the pid available in a directory. This directory is same for HAWQ master and HAWQ standby, thus if standby has been activated to active master, status of HAWQ Standby component still relies on the file and states it as running, and when an attempt is made to remove the standby component, it fails. Fixed the hawq component status derivation logic Diffs - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py 2432df6 ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py 28946f1 ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py c38e0de ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstatus.py a1b6dd9 Diff: https://reviews.apache.org/r/46181/diff/ Testing --- Manually Tested. Ran unit tests: test_hawq_master_critical (test_alert_component_status.TestAlertComponentStatus) ... ok test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... ok test_hawq_segment_critical (test_alert_component_status.TestAlertComponentStatus) ... ok test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... ok test_hawq_standby_critical (test_alert_component_status.TestAlertComponentStatus) ... ok test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... ok test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... ok test_exception_registration_status (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No handlers could be found for logger "ambari_alerts" ok test_missing_configs (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok test_missing_slave_file (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok test_successful_registration_status (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok test_unsuccessful_empty_db_registration_status (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok test_unsuccessful_registration_status (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok test_unsuccessful_registration_status_plural (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok test_configure_default (test_hawqmaster.TestHawqMaster) ... ok test_install_default (test_hawqmaster.TestHawqMaster) ... ok test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-13 16:36:49,436 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to false or is not available in hawq-env.xml ok test_stop_default (test_hawqmaster.TestHawqMaster) ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok -- Ran 34 tests in 0.357s OK Thanks, Matt
Re: Review Request 46186: Changing HAWQ Ports through Ambari prevents HAWQ service from restarting
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46186/#review128823 --- Ship it! ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py (line 92) <https://reviews.apache.org/r/46186/#comment192275> Are the expected return values just 0 and 1? If yes, the function can have a name similar to __is_process_running ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py (line 96) <https://reviews.apache.org/r/46186/#comment192277> You may use the component attribute map to get program_name or process_name. ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstatus.py (line 25) <https://reviews.apache.org/r/46186/#comment192276> You may move this attribute map to hawq_constants if you think that is the right place. - Matt On April 13, 2016, 7:19 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46186/ > --- > > (Updated April 13, 2016, 7:19 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-15852 > https://issues.apache.org/jira/browse/AMBARI-15852 > > > Repository: ambari > > > Description > --- > > HAWQ restart after port change will fail because Ambari looks for a currently > running process with a new port number. > This change is to make Ambari to look for an older port value. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 413cf1a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > f2ec914 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqsegment.py > 636dcad > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py > 67d1cb0 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstatus.py > 92ac0af > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/utils.py > 0baa4c5 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py c18941c > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 17d7473 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 44565b3 > > Diff: https://reviews.apache.org/r/46186/diff/ > > > Testing > --- > > Unit test has passed. > Manual Test in progress. > > > Thanks, > > jun aoki > >
Re: Review Request 46253: Add ignore-bad-hosts flag to hawq remove standby custom action to avoid failures when a segment host is down
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46253/#review129167 --- Ship it! Ship It! - Matt On April 14, 2016, 10:59 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46253/ > --- > > (Updated April 14, 2016, 10:59 p.m.) > > > Review request for Ambari, Alexander Denissov, Alejandro Fernandez, jun aoki, > Jayush Luniya, and Matt. > > > Bugs: AMBARI-15905 > https://issues.apache.org/jira/browse/AMBARI-15905 > > > Repository: ambari > > > Description > --- > > Add ignore-bad-hosts flag to hawq remove standby custom action to avoid > failures when a segment host is down > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > f2ec914 > > Diff: https://reviews.apache.org/r/46253/diff/ > > > Testing > --- > > yes. > > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-14 > 23:01:10,891 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys > is either set to false or is not available in hawq-env.xml > ok > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > test_configure_default (test_hawqsegment.TestHawqSegment) ... ok > test_install_default (test_hawqsegment.TestHawqSegment) ... ok > test_start_default (test_hawqsegment.TestHawqSegment) ... ok > test_stop_default (test_hawqsegment.TestHawqSegment) ... ok > test_configure_default (test_hawqstandby.TestHawqStandby) ... ok > test_install_default (test_hawqstandby.TestHawqStandby) ... ok > test_start_default (test_hawqstandby.TestHawqStandby) ... ok > test_stop_default (test_hawqstandby.TestHawqStandby) ... ok > > -- > Ran 34 tests in 0.147s > > OK > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 46283: hawq init master fails to syncup hawq-site xml if there is a segment host down
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46283/#review129235 --- Ship it! Ship It! - Matt On April 15, 2016, 3:15 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46283/ > --- > > (Updated April 15, 2016, 3:15 p.m.) > > > Review request for Ambari, Alexander Denissov, Alejandro Fernandez, jun aoki, > Jayush Luniya, Matt, Oleksandr Diachenko, and Srimanth Gunturi. > > > Bugs: AMBARI-15930 > https://issues.apache.org/jira/browse/AMBARI-15930 > > > Repository: ambari > > > Description > --- > > In hawq init command, default bucket number is calculated and the updated > hawq-site.xml is copied to all the other nodes, if there is a segment host > down it will fail to syncup the fail. > Including the option to skip bad hosts as ambari will override this file > whenever the segment host is brought online and started. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > b929430 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 5f7f085 > > Diff: https://reviews.apache.org/r/46283/diff/ > > > Testing > --- > > yes. > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-15 > 15:16:22,910 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys > is either set to false or is not available in hawq-env.xml > ok > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > test_configure_default (test_hawqsegment.TestHawqSegment) ... ok > test_install_default (test_hawqsegment.TestHawqSegment) ... ok > test_start_default (test_hawqsegment.TestHawqSegment) ... ok > test_stop_default (test_hawqsegment.TestHawqSegment) ... ok > test_configure_default (test_hawqstandby.TestHawqStandby) ... ok > test_install_default (test_hawqstandby.TestHawqStandby) ... ok > test_start_default (test_hawqstandby.TestHawqStandby) ... ok > test_stop_default (test_hawqstandby.TestHawqStandby) ... ok > test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok > > -- > Ran 35 tests in 0.148s > > OK > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 46294: HAWQ activate standby wizard fails after port number change but before restart.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46294/#review129408 --- Ship it! Ship It! - Matt On April 15, 2016, 5:56 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46294/ > --- > > (Updated April 15, 2016, 5:56 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-15926 > https://issues.apache.org/jira/browse/AMBARI-15926 > > > Repository: ambari > > > Description > --- > > Activating standby could start a master with an old port, and the following > operations in the Activate Standby wizard could be mislead. > The fix is to stop the newly become master so that the wizard will only have > to deal with the current value. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py > e2114d8 > > Diff: https://reviews.apache.org/r/46294/diff/ > > > Testing > --- > > Unit test passed. > Manually tested. > > > Thanks, > > jun aoki > >
Review Request 46359: HAWQ Immediate Stop cluster fails with key error
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46359/ --- Review request for Ambari. Bugs: AMBARI-15960 https://issues.apache.org/jira/browse/AMBARI-15960 Repository: ambari Description --- Error log from running the wizard: stderr: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py", line 98, in HawqMaster().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py", line 69, in immediate_stop_hawq_service common.stop_component(hawq_constants.CLUSTER, hawq_constants.IMMEDIATE) File "/var/lib/ambari-agent/cache/common-services/HAWQ/2.0.0/package/scripts/common.py", line 309, in stop_component port_property_name = hawq_constants.COMPONENT_ATTRIBUTES_MAP[component_name]['port_property'] KeyError: 'cluster' stdout: None Diffs - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py 4ce0c94 Diff: https://reviews.apache.org/r/46359/diff/ Testing --- Manually Tested. Unit Tests passed. Thanks, Matt
Re: Review Request 46350: PXF Check fails on Kerberized cluster without a YARN component
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46350/#review129631 --- Ship it! Ship It! - Matt On April 19, 2016, 2:18 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46350/ > --- > > (Updated April 19, 2016, 2:18 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, Matt, Newton Alex, Oleksandr Diachenko, and Srimanth Gunturi. > > > Bugs: AMBARI-15954 > https://issues.apache.org/jira/browse/AMBARI-15954 > > > Repository: ambari > > > Description > --- > > If the PXF component is not co-located with a YARN component, PXF service > check would fail. This is because it requires yarn-site on that host. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/params.py > ae0c80c > > ambari-server/src/main/resources/common-services/PXF/3.0.0/package/scripts/service_check.py > fe3c17e > > Diff: https://reviews.apache.org/r/46350/diff/ > > > Testing > --- > > Tested manually > > > Thanks, > > Lav Jain > >
Re: Review Request 46509: Remove performing service check during "Remove Standby Wizard"
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46509/#review129946 --- Ship it! Better run unit tests on ambari-web instead of stacks/HAWQ - Matt On April 21, 2016, 12:12 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46509/ > --- > > (Updated April 21, 2016, 12:12 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16024 > https://issues.apache.org/jira/browse/AMBARI-16024 > > > Repository: ambari > > > Description > --- > > Users will need to remove the HAWQ Standby Master using "Remove Standby > Wizard" after enabling HDFS HA. > So the service check may fail during the wizard as the HAWQ catalog might be > still pointing to the old filespace. > > > Diffs > - > > > ambari-web/app/controllers/main/admin/highAvailability/hawq/removeStandby/step3_controller.js > 5bcf34f > > Diff: https://reviews.apache.org/r/46509/diff/ > > > Testing > --- > > yes. > ~/github/ambari-vagrant/centos6.4/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ > ~/github/ambari-vagrant/centos6.4/ambari > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-21 > 12:15:00,349 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys > is either set to false or is not available in hawq-env.xml > ok > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > test_configure_default (test_hawqsegment.TestHawqSegment) ... ok > test_install_default (test_hawqsegment.TestHawqSegment) ... ok > test_start_default (test_hawqsegment.TestHawqSegment) ... ok > test_stop_default (test_hawqsegment.TestHawqSegment) ... ok > test_configure_default (test_hawqstandby.TestHawqStandby) ... ok > test_install_default (test_hawqstandby.TestHawqStandby) ... ok > test_start_default (test_hawqstandby.TestHawqStandby) ... ok > test_stop_default (test_hawqstandby.TestHawqStandby) ... ok > test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok > > -- > Ran 35 tests in 0.178s > > OK > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review130191
---
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
(line 44)
<https://reviews.apache.org/r/44210/#comment193880>
*return componentName in ('HAWQMASTER', 'HAWQSTANDBY')*
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
(line 118)
<https://reviews.apache.org/r/44210/#comment193923>
There is some additional logic for HAWQ mentioned in
recommendHDFSConfigurations:
https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py#L368-#L370
I believe this would still work from the HDP 2.3 stack advisor. Would you
like to move it here?
ambari-server/src/main/resources/stacks/service_advisor.py (line 48)
<https://reviews.apache.org/r/44210/#comment193918>
stackadvisor is being passed to many of the service advisor functions for
using some utility functions like isLocalHost()
These utility functions should ideally reside outside both stack advisor
and service advisor.
ambari-server/src/main/resources/stacks/stack_advisor.py (line 419)
<https://reviews.apache.org/r/44210/#comment193899>
If component A of service SA is co-located based on how component B of
service SB is laid out, and services list is ['SA', 'SB'], then host assignment
for A comes before B, which would be incorrect.
SB's components should be recommended first, then SA's components, because
A depends on host assignment of B.
ambari-server/src/main/resources/stacks/stack_advisor.py (line 576)
<https://reviews.apache.org/r/44210/#comment193912>
createServiceAdvisor is called multiple times, in
componentLayoutValidations, recommendComponentLayout, etc.
Is it not possible to call it once, and cache it and use it again?
ambari-server/src/main/resources/stacks/stack_advisor.py (line 654)
<https://reviews.apache.org/r/44210/#comment193926>
How is this different from the function defined on line 586 (right side)?
- Matt
On April 22, 2016, 11:27 a.m., Tim Thorpe wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> ---
>
> (Updated April 22, 2016, 11:27 a.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush
> Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Currently the stack advisor is defined under each stack version such as
> HDP/2.3. The problem with this is that it restricts the services that can be
> added to the stack. If a custom service is to be added, they would need to
> modify the stack advisor. If the configuration recommendation and validation
> can be done at the service level then the custom service could just include
> their own recommendations and validations separately.
>
>
> Diffs
> -
>
> ambari-server/src/main/assemblies/server.xml e1a4919
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> df65010
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java
> 00c8696
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java
> ca1968e
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java
> 6c6fa91
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
> 636de37
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
> d574d60
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java
> b7e09a9
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
> d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java
> db95fec
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py
> PRE-CREATION
>
> ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py
> PRE-CREATION
> ambari-server/src/main/resources/properties.json eac0dbd
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services
Review Request 46585: Show number of PXFs Live widget on main dashboard
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46585/ --- Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam Tadi, jun aoki, and Lav Jain. Bugs: AMBARI-16074 https://issues.apache.org/jira/browse/AMBARI-16074 Repository: ambari Description --- Show number of PXFs Live widget on main dashboard Diffs - ambari-web/app/assets/test/tests.js 33e7f1c ambari-web/app/mappers/components_state_mapper.js 755c0f5 ambari-web/app/messages.js 907d9ae ambari-web/app/views.js 6bb2774 ambari-web/app/views/main/dashboard/widgets.js 067607f ambari-web/app/views/main/dashboard/widgets/hawqsegment_live.js bec98c6 ambari-web/app/views/main/dashboard/widgets/pxf_live.js PRE-CREATION ambari-web/test/views/main/dashboard/widgets/pxf_live_test.js PRE-CREATION Diff: https://reviews.apache.org/r/46585/diff/ Testing --- Manually Tested Added Unit test. Unit test Results: 27538 tests complete (22 seconds) 154 tests pending [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:11 min [INFO] Finished at: 2016-04-22T16:50:37-07:00 [INFO] Final Memory: 12M/310M [INFO] Thanks, Matt
Re: Review Request 46585: Show number of PXFs Live widget on main dashboard
> On April 25, 2016, 5:12 p.m., jun aoki wrote: > > ambari-web/app/views/main/dashboard/widgets/pxf_live.js, line 87 > > <https://reviews.apache.org/r/46585/diff/1/?file=1358236#file1358236line87> > > > > How about ```isAnyMetricsNA()``` ? Will keep it as is to follow same format as DATANODE and HAWQSEGMENT https://github.com/apache/ambari/blob/trunk/ambari-web/app/views/main/dashboard/widgets/datanode_live.js#L81 - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46585/#review130523 --- On April 22, 2016, 4:51 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46585/ > --- > > (Updated April 22, 2016, 4:51 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-16074 > https://issues.apache.org/jira/browse/AMBARI-16074 > > > Repository: ambari > > > Description > --- > > Show number of PXFs Live widget on main dashboard > > > Diffs > - > > ambari-web/app/assets/test/tests.js 33e7f1c > ambari-web/app/mappers/components_state_mapper.js 755c0f5 > ambari-web/app/messages.js 907d9ae > ambari-web/app/views.js 6bb2774 > ambari-web/app/views/main/dashboard/widgets.js 067607f > ambari-web/app/views/main/dashboard/widgets/hawqsegment_live.js bec98c6 > ambari-web/app/views/main/dashboard/widgets/pxf_live.js PRE-CREATION > ambari-web/test/views/main/dashboard/widgets/pxf_live_test.js PRE-CREATION > > Diff: https://reviews.apache.org/r/46585/diff/ > > > Testing > --- > > Manually Tested > > Added Unit test. > > Unit test Results: > > 27538 tests complete (22 seconds) > 154 tests pending > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 01:11 min > [INFO] Finished at: 2016-04-22T16:50:37-07:00 > [INFO] Final Memory: 12M/310M > [INFO] > > > > Thanks, > > Matt > >
Review Request 46742: Use additional flags for HAWQ Check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46742/ --- Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam Tadi, jun aoki, and Lav Jain. Bugs: AMBARI-16106 https://issues.apache.org/jira/browse/AMBARI-16106 Repository: ambari Description --- Make changes to HAWQ Check implementation based on HAWQ-668 Diffs - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py 15ceaa2 ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py 76a5885 Diff: https://reviews.apache.org/r/46742/diff/ Testing --- Manually tested Ran unit tests: test_stop_default (test_hawqmaster.TestHawqMaster) ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok -- Ran 35 tests in 0.126s OK Thanks, Matt
Re: Review Request 46742: Use additional flags for HAWQ Check
> On April 27, 2016, 10:45 a.m., Alexander Denissov wrote: > > Please add unit tests. Working on it - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46742/#review130816 --- On April 27, 2016, 10:29 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46742/ > --- > > (Updated April 27, 2016, 10:29 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-16106 > https://issues.apache.org/jira/browse/AMBARI-16106 > > > Repository: ambari > > > Description > --- > > Make changes to HAWQ Check implementation based on HAWQ-668 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqmaster.py > 15ceaa2 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/params.py > 76a5885 > > Diff: https://reviews.apache.org/r/46742/diff/ > > > Testing > --- > > Manually tested > > Ran unit tests: > test_stop_default (test_hawqmaster.TestHawqMaster) ... ok > test_configure_default (test_hawqsegment.TestHawqSegment) ... ok > test_install_default (test_hawqsegment.TestHawqSegment) ... ok > test_start_default (test_hawqsegment.TestHawqSegment) ... ok > test_stop_default (test_hawqsegment.TestHawqSegment) ... ok > test_configure_default (test_hawqstandby.TestHawqStandby) ... ok > test_install_default (test_hawqstandby.TestHawqStandby) ... ok > test_start_default (test_hawqstandby.TestHawqStandby) ... ok > test_stop_default (test_hawqstandby.TestHawqStandby) ... ok > test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok > > -- > Ran 35 tests in 0.126s > > OK > > > Thanks, > > Matt > >
Re: Review Request 46800: HAWQ Configuration should be updated whenever Namenode is being moved.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46800/#review131000 --- Fix it, then Ship it! ambari-web/test/controllers/main/service/reassign/step4_controller_test.js (line 805) <https://reviews.apache.org/r/46800/#comment194918> Missing expect statement for hdfs-client? ambari-web/test/controllers/main/service/reassign/step4_controller_test.js (line 827) <https://reviews.apache.org/r/46800/#comment194919> Missing expect statement? - Matt On April 28, 2016, 2:57 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46800/ > --- > > (Updated April 28, 2016, 2:57 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, Oleksandr > Diachenko, and Richard Zang. > > > Bugs: AMBARI-16172 > https://issues.apache.org/jira/browse/AMBARI-16172 > > > Repository: ambari > > > Description > --- > > HAWQ Configuration should be updated whenever Namenode is being moved using > the move namenode wizard. > Parameters which should be updated > Case 1: HDFS HA > hdfs-client parameters under HAWQ service > dfs.namenode.rpc-address.. > dfs.namenode.http-address.. > Case 2: Non HA HDFS > hawq-site parmeters under HAWQ service > hawq_dfs_url > > > Diffs > - > > ambari-web/app/controllers/main/service/reassign/step4_controller.js > 840bda3 > ambari-web/test/controllers/main/service/reassign/step4_controller_test.js > 1e137ee > > Diff: https://reviews.apache.org/r/46800/diff/ > > > Testing > --- > > yes. > > 10505 tests complete (13 seconds) > 121 tests pending > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 29.607 s > [INFO] Finished at: 2016-04-28T14:53:42-07:00 > [INFO] Final Memory: 11M/213M > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 46742: Use additional flags for HAWQ Check
heck_case5 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 5: Non HDFS-HA, YARN Resource Management YARN_HA, Not Kerberized ... 2016-04-28 18:33:41,435 - Executing HAWQ Check ... ok test_run_hawq_check_case6 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 6: Non HDFS-HA, YARN Resource Management YARN_HA, Kerberized ... 2016-04-28 18:33:41,444 - Executing HAWQ Check ... ok test_run_hawq_check_case7 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 7: HDFS-HA, Standalone Resource Management, Not Kerberized ... 2016-04-28 18:33:41,451 - Executing HAWQ Check ... ok test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, Kerberized ... 2016-04-28 18:33:41,459 - Executing HAWQ Check ... ok test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not Kerberized ... 2016-04-28 18:33:41,466 - Executing HAWQ Check ... ok test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-28 18:33:41,473 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to false or is not available in hawq-env.xml ok test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster) Run Stop HAWQ Cluster Immediate Mode ... ok test_stop_default (test_hawqmaster.TestHawqMaster) Run Stop HAWQMASTER ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok -- Ran 50 tests in 0.347s OK Thanks, Matt
Re: Review Request 46791: Capitalize HAWQ and PXF on service check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46791/#review131032 --- Ship it! Ship It! - Matt On April 28, 2016, 10:03 a.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46791/ > --- > > (Updated April 28, 2016, 10:03 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, > Lav Jain, and Matt. > > > Bugs: AMBARI-16168 > https://issues.apache.org/jira/browse/AMBARI-16168 > > > Repository: ambari > > > Description > --- > > Capitalize HAWQ and PXF on service check > > > Diffs > - > > ambari-web/app/utils/helper.js 3cffc09 > > Diff: https://reviews.apache.org/r/46791/diff/ > > > Testing > --- > > Yes, Tested manually. Tests passed. > > > Thanks, > > Goutam Tadi > >
Re: Review Request 46742: Use additional flags for HAWQ Check
eck ... ok test_run_hawq_check_case5 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 5: Non HDFS-HA, YARN Resource Management YARN_HA, Not Kerberized ... 2016-04-29 11:48:07,261 - Executing HAWQ Check ... ok test_run_hawq_check_case6 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 6: Non HDFS-HA, YARN Resource Management YARN_HA, Kerberized ... 2016-04-29 11:48:07,268 - Executing HAWQ Check ... ok test_run_hawq_check_case7 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 7: HDFS-HA, Standalone Resource Management, Not Kerberized ... 2016-04-29 11:48:07,275 - Executing HAWQ Check ... ok test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, Kerberized ... 2016-04-29 11:48:07,283 - Executing HAWQ Check ... ok test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not Kerberized ... 2016-04-29 11:48:07,292 - Executing HAWQ Check ... ok test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-29 11:48:07,307 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to false or is not available in hawq-env.xml ok test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster) Run Stop HAWQ Cluster Immediate Mode ... ok test_stop_default (test_hawqmaster.TestHawqMaster) Run Stop HAWQMASTER ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok -- Ran 50 tests in 0.374s OK Thanks, Matt
Re: Review Request 46857: 'Restart All' for HAWQ should follow role_command_order
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46857/#review131234 --- Ship it! Ship It! - Matt On April 29, 2016, 3:20 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46857/ > --- > > (Updated April 29, 2016, 3:20 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, Jayush Luniya, Matt, and Srimanth Gunturi. > > > Bugs: AMBARI-16192 > https://issues.apache.org/jira/browse/AMBARI-16192 > > > Repository: ambari > > > Description > --- > > The current implementation in Ambari, 'Restart All' service action does not > follow role_command_order. > For example, all HAWQSEGMENTs start before HAWQMASTER and HAWQSTANDBY starts. > > > Diffs > - > > ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json > da4146a > > Diff: https://reviews.apache.org/r/46857/diff/ > > > Testing > --- > > Tested manually. > > > Thanks, > > Lav Jain > >
Re: Review Request 46860: Move RM wizard should update HAWQ related parameters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46860/#review131391 --- ambari-web/app/controllers/main/service/reassign/step4_controller.js (line 85) <https://reviews.apache.org/r/46860/#comment195342> If HAWQ is configured to work on Standalone mode, should HAWQ be in the list of services to stop/start when some operation is done with YARN service? Can this be appended to this list based on what type of resource management HAWQ is relying on? I would configure HAWQ in standalone mode, and move YARN Resource Manager - I would not expect HAWQ to be restarted in this case. ambari-web/app/utils/configs/move_rm_config_initializer.js (line 120) <https://reviews.apache.org/r/46860/#comment195347> The line numbers on the link does not stick when the file in that repository gets updated. - Matt On May 2, 2016, 2:47 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46860/ > --- > > (Updated May 2, 2016, 2:47 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-16174 > https://issues.apache.org/jira/browse/AMBARI-16174 > > > Repository: ambari > > > Description > --- > > when RM HA is not enabled, following properties should be updated in > hawq-site.xml should be updated. > hawq_rm_yarn_address > hawq_rm_yarn_scheduler_address > When RM HA Is enabled and then a RM is moved, following properties in > yarn-client.xml should be updated > yarn.resourcemanager.ha > yarn.resourcemanager.scheduler.ha > > > Diffs > - > > ambari-web/app/controllers/main/service/reassign/step4_controller.js > 736f467 > ambari-web/app/utils/configs/move_rm_config_initializer.js f8800de > ambari-web/test/controllers/main/service/reassign/step4_controller_test.js > 329249c > > Diff: https://reviews.apache.org/r/46860/diff/ > > > Testing > --- > > Unit test passed > Calling set on destroyed view > > 27428 tests complete (35 seconds) > 154 tests pending > > > Manually tested > > > Thanks, > > jun aoki > >
Review Request 46919: Enable addingNewRepository on trunk by default
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46919/ --- Review request for Ambari and Alexandr Antonenko. Bugs: AMBARI-16208 https://issues.apache.org/jira/browse/AMBARI-16208 Repository: ambari Description --- Enable the experimental feature so it can be tested on trunk. Diffs - ambari-web/app/config.js fe970b1 Diff: https://reviews.apache.org/r/46919/diff/ Testing --- Ran mvn clean test in ambari-web: 27523 tests complete (22 seconds) 154 tests pending [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 1409 licence. [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 54.087 s [INFO] Finished at: 2016-05-02T17:23:30-07:00 [INFO] Final Memory: 12M/310M [INFO] Ran npm test in ambari-admin/src/main/resources/ui/admin-web/: INFO [karma]: Karma v0.12.16 server started at http://localhost:9876/ INFO [launcher]: Starting browser PhantomJS INFO [PhantomJS 1.9.7 (Mac OS X)]: Connected on socket dT-CAywe-Rzo6pm9AFWY with id 18022707 PhantomJS 1.9.7 (Mac OS X): Executed 64 of 64 SUCCESS (0.263 secs / 0.263 secs) Thanks, Matt
Re: Review Request 46860: Move RM wizard should update HAWQ related parameters
> On April 29, 2016, 5:21 p.m., bhuvnesh chaudhary wrote:
> > ambari-web/app/controllers/main/service/reassign/step4_controller.js, line
> > 85
> > <https://reviews.apache.org/r/46860/diff/1/?file=1367779#file1367779line85>
> >
> > Unless HAWQ is using YARN as the scheduler, HAWQ may still be
> > functional and we may need not restart HAWQ with the move of RM.
> > So do you think should we restart it only when HAWQ is using
> > hawq_global_rm_type as yarn ?
>
> jun aoki wrote:
> I thought it'd be a good idea but data.Cluters won't return enoguh
> information to decide if HAWQ is on YARN.
> ```
> data.Clusters.desired_configs["hawq-site"]
> Object {tag: "TOPOLOGY_RESOLVED", user: "admin", version: 2}
> ```
> Let me know if you have a better idea to find it out.
>
> bhuvnesh chaudhary wrote:
> You can get the configuration as soon you reach onLoadConfigs, and then
> you can use it which is before the start service ?
That sounds better to me. Modify the array after receiving the config.
If HAWQ is using YARN, append value 'HAWQ' to RESOURCEMANAGER's array value.
- Matt
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46860/#review131231
---
On May 2, 2016, 2:47 p.m., jun aoki wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46860/
> ---
>
> (Updated May 2, 2016, 2:47 p.m.)
>
>
> Review request for Ambari.
>
>
> Bugs: AMBARI-16174
> https://issues.apache.org/jira/browse/AMBARI-16174
>
>
> Repository: ambari
>
>
> Description
> ---
>
> when RM HA is not enabled, following properties should be updated in
> hawq-site.xml should be updated.
> hawq_rm_yarn_address
> hawq_rm_yarn_scheduler_address
> When RM HA Is enabled and then a RM is moved, following properties in
> yarn-client.xml should be updated
> yarn.resourcemanager.ha
> yarn.resourcemanager.scheduler.ha
>
>
> Diffs
> -
>
> ambari-web/app/controllers/main/service/reassign/step4_controller.js
> 736f467
> ambari-web/app/utils/configs/move_rm_config_initializer.js f8800de
> ambari-web/test/controllers/main/service/reassign/step4_controller_test.js
> 329249c
>
> Diff: https://reviews.apache.org/r/46860/diff/
>
>
> Testing
> ---
>
> Unit test passed
> Calling set on destroyed view
>
> 27428 tests complete (35 seconds)
> 154 tests pending
>
>
> Manually tested
>
>
> Thanks,
>
> jun aoki
>
>
Re: Review Request 46853: clean up import * : Accumulo, Ambari-metrics, Atlas, Falcon, Flume and Ganglia
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46853/#review131430 --- Ship it! Ship It! - Matt On April 29, 2016, 5:51 p.m., Juanjo Marron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46853/ > --- > > (Updated April 29, 2016, 5:51 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Jayush Luniya. > > > Bugs: AMBARI-16175 > https://issues.apache.org/jira/browse/AMBARI-16175 > > > Repository: ambari > > > Description > --- > > Python code at at common-services level used generic imports form > resource_management (from resource_management import *) > Ideally, for easier code tracking and performance, these import should be > more specific, such as: > from resource_management.libraries.script.script import Script > from resource_management.core.resources.system import Directory > This subtask cleans up import * from resource_management and replace it for > specific imports for: > Accumulo, Ambari-metrics, Atlas, Falcon, Flume and Ganglia services > > > Diffs > - > > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_configuration.py > 09a3d90 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_service.py > 33d3273 > > ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/service_check.py > a7067cb > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams.py > 3c3d804 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_service.py > eb1b6d7 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase.py > 7fb9715 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase_master.py > b769a0d > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase_regionserver.py > cf0efef > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase_service.py > 4d0d7f3 > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py > a5abe1c > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py > 32b8fdb > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params_linux.py > 21b491d > > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/status_params.py > 3c4bffc > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py > 2bdb4c0 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata.py > 43c1ea6 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py > 4b08cbc > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py > defd6f0 > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/service_check.py > 194a3fb > > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/status_params.py > 4c54214 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py > 64ace1a > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py > a201619 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server_upgrade.py > 6277086 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_windows.py > b213028 > > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/service_check.py > 473b7c3 > > ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume.py > 9df2ba6 > > ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_check.py > 25d9a08 > > ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py > 1333605 > > ambari-server/src/main/resources/common-services/FLUME/1.4.
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131579 --- The functionality of ServiceAdvisor works as expected. However more clarification is needed on how to use inheritance and override service advisor defined in common-services. For example: HAWQ200ServiceAdvisor is defined under commmon-services. I'd like to override the method getComponentLayoutScheme with some new logic for HDP 2.3 stack. Followed the documentation in ambari-server/src/main/resources/stacks/service_advisor.py but it did not work as expected. - Matt On May 3, 2016, 12:04 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated May 3, 2016, 12:04 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > f475798 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 2080c52 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 46949: PXF alert: change the message to make it more meaningful when both namenodes are down on Secured HA cluster.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46949/#review131580 --- Ship it! ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py (line 141) <https://reviews.apache.org/r/46949/#comment195595> Is there a possibility where active_namenodes can be an empty list? - A case where both namenodes are down? ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py (line 143) <https://reviews.apache.org/r/46949/#comment195597> Do you need to log this explicitly using Logger.fail() or does it get logged automatically to std-err log, which shows up the UI? - Matt On May 3, 2016, 4:05 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46949/ > --- > > (Updated May 3, 2016, 4:05 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-16237 > https://issues.apache.org/jira/browse/AMBARI-16237 > > > Repository: ambari > > > Description > --- > > PXF alerts shows "Configuration parameter 'fs_root' was not found in > configurations dictionary!" when both namnodes are down on secured NN HA > environment, which the error message is not too clear. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py > ee16c9b > > Diff: https://reviews.apache.org/r/46949/diff/ > > > Testing > --- > > Manually tested. > > > Thanks, > > jun aoki > >
Re: Review Request 46949: PXF alert: change the message to make it more meaningful when both namenodes are down on Secured HA cluster.
> On May 3, 2016, 4:14 p.m., Matt wrote: > > Discussed this in person: This function is not used anywhere else, besides PXF service. Logging would be taken care of by PXF service component. - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46949/#review131580 --- On May 3, 2016, 4:05 p.m., jun aoki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46949/ > --- > > (Updated May 3, 2016, 4:05 p.m.) > > > Review request for Ambari. > > > Bugs: AMBARI-16237 > https://issues.apache.org/jira/browse/AMBARI-16237 > > > Repository: ambari > > > Description > --- > > PXF alerts shows "Configuration parameter 'fs_root' was not found in > configurations dictionary!" when both namnodes are down on secured NN HA > environment, which the error message is not too clear. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py > ee16c9b > > Diff: https://reviews.apache.org/r/46949/diff/ > > > Testing > --- > > Manually tested. > > > Thanks, > > jun aoki > >
Re: Review Request 42155: AMBARI-14605 '[RAM_DISK]' configured in dfs.datanode.data.dir fails to validate
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42155/#review131621 --- You may close this review as it has been submitted already. - Matt On May 2, 2016, 9:57 p.m., yao lei wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42155/ > --- > > (Updated May 2, 2016, 9:57 p.m.) > > > Review request for Ambari and Matt. > > > Bugs: AMBARI-14605 > https://issues.apache.org/jira/browse/AMBARI-14605 > > > Repository: ambari > > > Description > --- > > HDFS now support four kinds of heterogeneous storage types: ARCHIVE, DISK, > SSD and RAM_DISK. > If we configure dfs.datanode.data.dir as [RAM_DISK]/anydir,it fails to > validate. > please see > https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html#Configuration > A > > > Diffs > - > > ambari-web/app/utils/validator.js 9d11746 > ambari-web/test/utils/validator_test.js ef90561 > > Diff: https://reviews.apache.org/r/42155/diff/ > > > Testing > --- > > Manually Tested. > But Ambari Web Unit Test fails in trunk for other reasons described in > https://builds.apache.org/job/Ambari-trunk-test-patch/4848//artifact/patch-work/testrun_ambari-web.txt > > > Thanks, > > yao lei > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On May 3, 2016, 4:09 p.m., Matt wrote:
> > The functionality of ServiceAdvisor works as expected. However more
> > clarification is needed on how to use inheritance and override service
> > advisor defined in common-services.
> > For example:
> > HAWQ200ServiceAdvisor is defined under commmon-services. I'd like to
> > override the method getComponentLayoutScheme with some new logic for HDP
> > 2.3 stack. Followed the documentation in
> > ambari-server/src/main/resources/stacks/service_advisor.py but it did not
> > work as expected.
>
> Tim Thorpe wrote:
> Thanks Matt, for finding that issue. There were actually 2 problems.
> The first was with the advisor class naming convention for stack service's
> that contained the service_advisor.py rather than inherit it from
> common-services. It missed the stack version number in the advisor class
> name. The second issue was with the HAWQ and PXF service_advisor.py files
> which added to the stacks directory to the system path to load the
> stacks/service_advisor.py (for the default service advisor). I have modified
> that code to use the same dynamic loading I recommended for service advisor
> inheritance. In other words, I changed it to this:
>
> SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
> STACKS_DIR = os.path.join(SCRIPT_DIR, '../../../stacks/')
> PARENT_FILE = os.path.join(STACKS_DIR, 'service_advisor.py')
>
> try:
> with open(PARENT_FILE, 'rb') as fp:
> service_advisor = imp.load_module('service_advisor', fp, PARENT_FILE,
> ('.py', 'rb', imp.PY_SOURCE))
> except Exception as e:
> traceback.print_exc()
> print "Failed to load parent"
>
> class HAWQ200ServiceAdvisor(service_advisor.ServiceAdvisor):
>
> I presume the reason this change is required is due to a naming conflict
> because all the files are called service_advisor.py.
>
> Tim Thorpe wrote:
> I have also added a comment in stacks/service_advisor.py to look at the
> HAWQ and PXF service advisors for examples on how to create your own service
> advisor.
Thank you Tim for making the updates. We are testing it at the moment...
- Matt
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44210/#review131579
---
On May 4, 2016, 7:17 a.m., Tim Thorpe wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44210/
> ---
>
> (Updated May 4, 2016, 7:17 a.m.)
>
>
> Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush
> Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako.
>
>
> Bugs: AMBARI-15226
> https://issues.apache.org/jira/browse/AMBARI-15226
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Currently the stack advisor is defined under each stack version such as
> HDP/2.3. The problem with this is that it restricts the services that can be
> added to the stack. If a custom service is to be added, they would need to
> modify the stack advisor. If the configuration recommendation and validation
> can be done at the service level then the custom service could just include
> their own recommendations and validations separately.
>
>
> Diffs
> -
>
> ambari-server/src/main/assemblies/server.xml e1a4919
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> df65010
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java
> 00c8696
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java
> ca1968e
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java
> 6c6fa91
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java
> 636de37
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java
> 356adb1
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java
> b7e09a9
>
> ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java
> d27e52a
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java
> 5a2bf84
>
>
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131747 --- Ship it! We have tested the following changes: - The logic for stack_advisor which was pulled into service_advisor works well. - Inheritance between stacks works well. We tested inheritance from common-services into stack 2.3, and then inheriting from 2.3 to stack 2.4 with overrides. Enhancements: - Factor out utility functions that are common for both stack_advisor and service_advisor. - Cache service_advisors to avoid loading service_advisor multiple times for the same service. - At the moment, the service advisor class name under the stack should be of the format **ServiceAdvisor**. It would be great if this class can have a different name in the service_advisor.py file and specify the file path and class name in metainfo.xml under that stack, so that the class gets loaded. - Matt On May 4, 2016, 7:17 a.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated May 4, 2016, 7:17 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > f475798 > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > 2080c52 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 46996: HAWQ service can be stopped and checked even when STANDBY MASTER is stopped
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/46996/#review131762 --- Ship it! Ship It! - Matt On May 4, 2016, 4:52 p.m., Alexander Denissov wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/46996/ > --- > > (Updated May 4, 2016, 4:52 p.m.) > > > Review request for Ambari, bhuvnesh chaudhary, Goutam Tadi, jun aoki, Lav > Jain, and Matt. > > > Bugs: AMBARI-16236 > https://issues.apache.org/jira/browse/AMBARI-16236 > > > Repository: ambari > > > Description > --- > > HAWQ service can be stopped and checked even when STANDBY MASTER is stopped > > > Diffs > - > > ambari-web/app/controllers/main/service/item.js bfede8a > ambari-web/test/controllers/main/service/item_test.js 4bac784 > > Diff: https://reviews.apache.org/r/46996/diff/ > > > Testing > --- > > manual. added unit tests: > > 27532 tests complete (29 seconds) > 154 tests pending > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 45.229 s > [INFO] Finished at: 2016-05-04T16:25:14-08:00 > > > Thanks, > > Alexander Denissov > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On May 4, 2016, 2:37 p.m., Matt wrote: > > We have tested the following changes: > > - The logic for stack_advisor which was pulled into service_advisor works > > well. > > - Inheritance between stacks works well. We tested inheritance from > > common-services into stack 2.3, and then inheriting from 2.3 to stack 2.4 > > with overrides. > > > > Enhancements: > > - Factor out utility functions that are common for both stack_advisor and > > service_advisor. > > - Cache service_advisors to avoid loading service_advisor multiple times > > for the same service. > > - At the moment, the service advisor class name under the stack should be > > of the format **ServiceAdvisor**. > > It would be great if this class can have a different name in the > > service_advisor.py file and specify the file path and class name in > > metainfo.xml under that stack, so that the class gets loaded. > > Tim Thorpe wrote: > Hi Matt, are you able to push the patch in? I don't have committer > access. If not can someone else? Thanks I'll look into this within a couple of hours, when I get back to my laptop. - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131747 --- On May 5, 2016, 7:50 a.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated May 5, 2016, 7:50 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > cf0990d > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > fcb5407 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 44210: AMBARI-15226 - The stack advisor should be pushed down to the services
> On May 5, 2016, 3:52 p.m., Jayush Luniya wrote: > > Ship It! Jayush, shall I commit this or are you working on it? - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/44210/#review131962 --- On May 5, 2016, 7:50 a.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/44210/ > --- > > (Updated May 5, 2016, 7:50 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush > Luniya, Oleksandr Diachenko, Sumit Mohanty, Srimanth Gunturi, and Yusaku Sako. > > > Bugs: AMBARI-15226 > https://issues.apache.org/jira/browse/AMBARI-15226 > > > Repository: ambari > > > Description > --- > > Currently the stack advisor is defined under each stack version such as > HDP/2.3. The problem with this is that it restricts the services that can be > added to the stack. If a custom service is to be added, they would need to > modify the stack advisor. If the configuration recommendation and validation > can be done at the service level then the custom service could just include > their own recommendations and validations separately. > > > Diffs > - > > ambari-server/src/main/assemblies/server.xml e1a4919 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java > df65010 > > ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/commands/StackAdvisorCommand.java > 00c8696 > > ambari-server/src/main/java/org/apache/ambari/server/controller/StackServiceResponse.java > ca1968e > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/StackServiceResourceProvider.java > 6c6fa91 > > ambari-server/src/main/java/org/apache/ambari/server/stack/CommonServiceDirectory.java > 636de37 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 356adb1 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > b7e09a9 > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackServiceDirectory.java > d27e52a > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 5a2bf84 > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > PRE-CREATION > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > PRE-CREATION > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/stack_advisor.py > 1680f21 > ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py > cf0990d > ambari-server/src/main/resources/stacks/service_advisor.py PRE-CREATION > ambari-server/src/main/resources/stacks/stack_advisor.py 9979e7e > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > fcb5407 > > Diff: https://reviews.apache.org/r/44210/diff/ > > > Testing > --- > > Ran all the non java unit tests. > > Total run:945 > Total errors:0 > Total failures:0 > > Manually configured HAWQ and PXF as part of the HDP 2.3 stack and made sure > their service advisors were called. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 47134: HAWQ Password handling updated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47134/#review132319 --- Ship it! ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py (line 53) <https://reviews.apache.org/r/47134/#comment196503> json.dumps ensures that quotes and special characters are escaped? Did you consider && and giving an additional command? Or is this a new JIRA? - Matt On May 9, 2016, 12:55 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47134/ > --- > > (Updated May 9, 2016, 12:55 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16386 > https://issues.apache.org/jira/browse/AMBARI-16386 > > > Repository: ambari > > > Description > --- > > Updated salt used while creating HAWQ user, and also ensured that the > password is treated exactly as entered on the HAWQ config ui. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 1bdc85a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 670a58b > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 2671202 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 7963b3d > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 605f49a > > Diff: https://reviews.apache.org/r/47134/diff/ > > > Testing > --- > > yes. > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test_hawqmaster.TestHawqMaster) ... ok > test_remove_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Remove HAWQ Standby ... 2016-05-09 12:54:41,958 - Removing > HAWQ Standby Master ... > ok > test_resync_hawq_standby (test_hawqmaster.TestHawqMaster) > Run custom command Resync HAWQ Standby ... 2016-05-09 12:54:41,963 - HAWQ > Standby Master Re-Sync started in fast mode... > ok > test_run_hawq_check_case1 (test_hawqmaster.TestHawqMaster) > Running HAWQ Check Case 1: Non HDFS-HA, Standalone Resource Management, Not > Kerberized ... 2016-
Re: Review Request 47138: stackadvisor uses getHostsForSlaveComponent with wrong parameter name
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47138/#review132354 --- Ship it! Ship It! - Matt On May 9, 2016, 4:43 p.m., Lav Jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47138/ > --- > > (Updated May 9, 2016, 4:43 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, Matt, and Tim Thorpe. > > > Bugs: AMBARI-16378 > https://issues.apache.org/jira/browse/AMBARI-16378 > > > Repository: ambari > > > Description > --- > > ``` > def getHostsForSlaveComponent(self, services, hosts, component, hostsList, > hostsComponentsMap, freeHosts): > componentName = component["StackServiceComponents"]["component_name"] > > if component["StackServiceComponents"]["cardinality"] == "ALL": > return hostsList > > componentIsPopulated = self.isComponentHostsPopulated(component) > if componentIsPopulated: > return component["StackServiceComponents"]["hostnames"] > > hostsForComponent = [] > > if self.isSlaveComponent(component): > cardinality = str(component["StackServiceComponents"]["cardinality"]) > if self.isComponentUsingCardinalityForLayout(component) and cardinality: > # cardinality types: 1+, 1-2, 1 > ``` > > The correct parameter name is comonentName (instead of component) > > > Diffs > - > > ambari-server/src/main/resources/stacks/stack_advisor.py 00b9d79 > > Diff: https://reviews.apache.org/r/47138/diff/ > > > Testing > --- > > Tested manually > > > Thanks, > > Lav Jain > >
Re: Review Request 47156: More information for Standby sync alert
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47156/#review132384
---
Ship it!
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_sync_status.py
(line 68)
<https://reviews.apache.org/r/47156/#comment196571>
Would be better to have **'HAWQSTANDBY is not in sync with HAWQMASTER.
ERROR: ' + error_message**
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_sync_status.py
(line 102)
<https://reviews.apache.org/r/47156/#comment196572>
I believe you had to do resort to a separate query because you were not
able to parse the error message when the message spanned over more than one
line...
- Matt
On May 9, 2016, 9:27 p.m., jun aoki wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47156/
> ---
>
> (Updated May 9, 2016, 9:27 p.m.)
>
>
> Review request for Ambari.
>
>
> Bugs: AMBARI-16417
> https://issues.apache.org/jira/browse/AMBARI-16417
>
>
> Repository: ambari
>
>
> Description
> ---
>
> When Standby master is down (actually I removed it through API, and it got
> deleted but somehow the configuration has the old IP) the hawq alert pops
> up(, which is good) but only saying "HAWQSTANDBY is not in sync with
> HAWQMASTER".
>
> If you take a close look at gp_master_mirroring table, it has more detailed
> information. (in this case, connection issue)
> {code}
> [gpadmin@ip-10-32-38-104 ~]$ psql -d template1 -c "select * from
> gp_master_mirroring;"
> summary_state | detail_state |log_time|
>error_message
> --+--++--
> Not Synchronized | Connection error | 2016-04-12 21:14:23+00 | error
> received sending data to standby master: server closed the connection
> unexpectedly
> : This
> probably means the server terminated abnormally
> :
> before or while processing the request.
> :
> (1 row)
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_sync_status.py
> c94be9e
>
> Diff: https://reviews.apache.org/r/47156/diff/
>
>
> Testing
> ---
>
>
> Thanks,
>
> jun aoki
>
>
Re: Review Request 47156: More information for Standby sync alert
> On May 10, 2016, 12:10 a.m., Matt wrote:
> >
Please double check if any unit tests have to be updated.
- Matt
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47156/#review132384
---
On May 9, 2016, 9:27 p.m., jun aoki wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47156/
> ---
>
> (Updated May 9, 2016, 9:27 p.m.)
>
>
> Review request for Ambari.
>
>
> Bugs: AMBARI-16417
> https://issues.apache.org/jira/browse/AMBARI-16417
>
>
> Repository: ambari
>
>
> Description
> ---
>
> When Standby master is down (actually I removed it through API, and it got
> deleted but somehow the configuration has the old IP) the hawq alert pops
> up(, which is good) but only saying "HAWQSTANDBY is not in sync with
> HAWQMASTER".
>
> If you take a close look at gp_master_mirroring table, it has more detailed
> information. (in this case, connection issue)
> {code}
> [gpadmin@ip-10-32-38-104 ~]$ psql -d template1 -c "select * from
> gp_master_mirroring;"
> summary_state | detail_state |log_time|
>error_message
> --+--++--
> Not Synchronized | Connection error | 2016-04-12 21:14:23+00 | error
> received sending data to standby master: server closed the connection
> unexpectedly
> : This
> probably means the server terminated abnormally
> :
> before or while processing the request.
> :
> (1 row)
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_sync_status.py
> c94be9e
>
> Diff: https://reviews.apache.org/r/47156/diff/
>
>
> Testing
> ---
>
>
> Thanks,
>
> jun aoki
>
>
Review Request 47160: PXF should be collocated with NAMENODE even if NAMENODE is moved to another host
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47160/ --- Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam Tadi, jun aoki, and Lav Jain. Bugs: AMBARI-16419 https://issues.apache.org/jira/browse/AMBARI-16419 Repository: ambari Description --- After move NAMENODE wizard is executed, - PXF should be installed on the target host. - If the source host does not have a DATANODE, PXF should be removed from the source host. Diffs - ambari-web/app/controllers/main/service/reassign/step6_controller.js f718d46 ambari-web/app/messages.js e388526 ambari-web/test/controllers/main/service/reassign/step6_controller_test.js 89e2325 Diff: https://reviews.apache.org/r/47160/diff/ Testing --- Added unit tests. 27727 tests complete (25 seconds) 154 tests pending [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:14 min [INFO] Finished at: 2016-05-10T00:20:54-07:00 [INFO] Final Memory: 12M/310M [INFO] Thanks, Matt
Re: Review Request 47160: PXF should be collocated with NAMENODE even if NAMENODE is moved to another host
> On May 10, 2016, 12:13 p.m., jun aoki wrote: > > Do we have to Clean PXF from the previous NN node, if the node does no > > longer have any of PXF related ones? (DN, HAWQ etc.?) Did you mean to say, remove the rpms and files? I would be looking into this on a follow-up JIRA. - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47160/#review132518 --- On May 10, 2016, 12:26 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47160/ > --- > > (Updated May 10, 2016, 12:26 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-16419 > https://issues.apache.org/jira/browse/AMBARI-16419 > > > Repository: ambari > > > Description > --- > > After move NAMENODE wizard is executed, > - PXF should be installed on the target host. > - If the source host does not have a DATANODE, PXF should be removed from the > source host. > > > Diffs > - > > ambari-web/app/controllers/main/service/reassign/step6_controller.js > f718d46 > ambari-web/app/messages.js e388526 > ambari-web/test/controllers/main/service/reassign/step6_controller_test.js > 89e2325 > > Diff: https://reviews.apache.org/r/47160/diff/ > > > Testing > --- > > Added unit tests. > > 27727 tests complete (25 seconds) > 154 tests pending > > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 01:14 min > [INFO] Finished at: 2016-05-10T00:20:54-07:00 > [INFO] Final Memory: 12M/310M > [INFO] > > > > Thanks, > > Matt > >
Re: Review Request 47134: HAWQ Password handling updated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47134/#review132812 --- Fix it, then Ship it! ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py (line 30) <https://reviews.apache.org/r/47134/#comment197064> This is not required. ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py (line 49) <https://reviews.apache.org/r/47134/#comment197063> For this implementation, you do not need the config_dict to be loaded as json. This was required in HAWQMASTER test for updating the configs after loading within each unit test for testing different scenarios. This would work: ```password = crypt.crypt(self.getConfig()['configurations']['hawq-env']['hawq_password'], "$1$salt$")``` ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py (line 31) <https://reviews.apache.org/r/47134/#comment197065> This is not required. Refer comment on the test_hawqsegment.py file. - Matt On May 11, 2016, 6:07 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47134/ > --- > > (Updated May 11, 2016, 6:07 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16386 > https://issues.apache.org/jira/browse/AMBARI-16386 > > > Repository: ambari > > > Description > --- > > Updated salt used while creating HAWQ user, and also ensured that the > password is treated exactly as entered on the HAWQ config ui. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py > 1bdc85a > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > 670a58b > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 2671202 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 7963b3d > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py 605f49a > > Diff: https://reviews.apache.org/r/47134/diff/ > > > Testing > --- > > yes. > bhuvneshchaudhary@bhuviMac:HAWQ$ python -m discover -v > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_configure_default (test_hawqmaster.TestHawqMaster) ... ok > test_install_default (test
Re: Review Request 47277: hawq_ssh_exkeys config should be a checkbox on UI
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47277/#review132813
---
Ship it!
:)
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
(line 45)
<https://reviews.apache.org/r/47277/#comment197066>
- Matt
On May 11, 2016, 6:05 p.m., bhuvnesh chaudhary wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47277/
> ---
>
> (Updated May 11, 2016, 6:05 p.m.)
>
>
> Review request for Ambari, Alexander Denissov, jun aoki, and Matt.
>
>
> Bugs: AMBARI-16624
> https://issues.apache.org/jira/browse/AMBARI-16624
>
>
> Repository: ambari
>
>
> Description
> ---
>
> hawq_ssh_exkeys config should be a checkbox on UI. Currently it accepts a
> string.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-env.xml
> d2da5b2
>
> Diff: https://reviews.apache.org/r/47277/diff/
>
>
> Testing
> ---
>
> yes. manual.
>
>
> Thanks,
>
> bhuvnesh chaudhary
>
>
Re: Review Request 47252: PXF Agents Live Widget on Dashboard needs custom widget to display only single threshold i.e., Green/Red only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47252/#review132815 --- Would you like to add tests for this new widget? - Matt On May 11, 2016, 12:04 p.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47252/ > --- > > (Updated May 11, 2016, 12:04 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, > Lav Jain, and Matt. > > > Bugs: AMBARI-16461 > https://issues.apache.org/jira/browse/AMBARI-16461 > > > Repository: ambari > > > Description > --- > > PXF Agents Live Widget on Dashboard needs custom widget to display only > single threshold i.e., Green/Red only > > > Diffs > - > > ambari-web/app/messages.js c5e1145 > ambari-web/app/mixins.js cf8cef6 > ambari-web/app/mixins/main/dashboard/widgets/editable_with_limit.js 97b6330 > ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js > PRE-CREATION > ambari-web/app/styles/modal_popups.less 5e89dbc > > ambari-web/app/templates/main/dashboard/edit_widget_popup_single_threshold.hbs > PRE-CREATION > ambari-web/app/views.js 127a996 > ambari-web/app/views/main/dashboard/widgets.js 38181c0 > ambari-web/app/views/main/dashboard/widgets/pxf_live.js 75f5886 > ambari-web/app/views/main/dashboard/widgets/text_widget_single_threshold.js > PRE-CREATION > > Diff: https://reviews.apache.org/r/47252/diff/ > > > Testing > --- > > Yes, All tests passed. Test was already in place. > > > Thanks, > > Goutam Tadi > >
Re: Review Request 47156: More information for Standby sync alert
> On May 10, 2016, 12:10 a.m., Matt wrote:
> > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_sync_status.py,
> > line 104
> > <https://reviews.apache.org/r/47156/diff/1/?file=1377351#file1377351line104>
> >
> > I believe you had to do resort to a separate query because you were not
> > able to parse the error message when the message spanned over more than one
> > line...
>
> jun aoki wrote:
> I tested it and the error message is shown in UI OK, and sorting is not
> necessary. Let me know if I'm mistaken.
Looks like Lav clarified what I was trying to point out.
Yu may close this review since it has been submitted already.
- Matt
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47156/#review132384
---
On May 10, 2016, 3:22 p.m., jun aoki wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47156/
> ---
>
> (Updated May 10, 2016, 3:22 p.m.)
>
>
> Review request for Ambari.
>
>
> Bugs: AMBARI-16417
> https://issues.apache.org/jira/browse/AMBARI-16417
>
>
> Repository: ambari
>
>
> Description
> ---
>
> When Standby master is down (actually I removed it through API, and it got
> deleted but somehow the configuration has the old IP) the hawq alert pops
> up(, which is good) but only saying "HAWQSTANDBY is not in sync with
> HAWQMASTER".
>
> If you take a close look at gp_master_mirroring table, it has more detailed
> information. (in this case, connection issue)
> {code}
> [gpadmin@ip-10-32-38-104 ~]$ psql -d template1 -c "select * from
> gp_master_mirroring;"
> summary_state | detail_state |log_time|
>error_message
> --+--++--
> Not Synchronized | Connection error | 2016-04-12 21:14:23+00 | error
> received sending data to standby master: server closed the connection
> unexpectedly
> : This
> probably means the server terminated abnormally
> :
> before or while processing the request.
> :
> (1 row)
> {code}
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/alerts/alert_sync_status.py
> c94be9e
> ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_sync_status.py
> 7d030dc
>
> Diff: https://reviews.apache.org/r/47156/diff/
>
>
> Testing
> ---
>
>
> Thanks,
>
> jun aoki
>
>
Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ
q_check_case6 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 6: Non HDFS-HA, YARN Resource Management YARN_HA, Kerberized ... 2016-05-12 17:56:32,055 - Executing HAWQ Check ... ok test_run_hawq_check_case7 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 7: HDFS-HA, Standalone Resource Management, Not Kerberized ... 2016-05-12 17:56:32,061 - Executing HAWQ Check ... ok test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, Kerberized ... 2016-05-12 17:56:32,070 - Executing HAWQ Check ... ok test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not Kerberized ... 2016-05-12 17:56:32,078 - Executing HAWQ Check ... ok test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-05-12 17:56:32,086 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to false or is not available in hawq-env.xml ok test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster) Run Stop HAWQ Cluster Immediate Mode ... ok test_stop_default (test_hawqmaster.TestHawqMaster) Run Stop HAWQMASTER ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok test_getServiceConfigurationRecommendations (test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok -- Ran 51 tests in 0.226s OK ``` Thanks, Matt
Re: Review Request 47252: PXF Agents Live Widget on Dashboard needs custom widget to display only single threshold i.e., Green/Red only
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47252/#review133160 --- Ship it! Ship It! - Matt On May 12, 2016, 2:26 p.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47252/ > --- > > (Updated May 12, 2016, 2:26 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, > Lav Jain, and Matt. > > > Bugs: AMBARI-16461 > https://issues.apache.org/jira/browse/AMBARI-16461 > > > Repository: ambari > > > Description > --- > > PXF Agents Live Widget on Dashboard needs custom widget to display only > single threshold i.e., Green/Red only > > > Diffs > - > > ambari-web/app/assets/test/tests.js eb2edf1 > ambari-web/app/messages.js 910bed7 > ambari-web/app/mixins.js cf8cef6 > ambari-web/app/mixins/main/dashboard/widgets/editable_with_limit.js 97b6330 > ambari-web/app/mixins/main/dashboard/widgets/single_numeric_threshold.js > PRE-CREATION > ambari-web/app/styles/modal_popups.less 5e89dbc > > ambari-web/app/templates/main/dashboard/edit_widget_popup_single_threshold.hbs > PRE-CREATION > ambari-web/app/views.js 127a996 > ambari-web/app/views/main/dashboard/widgets.js 38181c0 > ambari-web/app/views/main/dashboard/widgets/pxf_live.js 75f5886 > ambari-web/app/views/main/dashboard/widgets/text_widget_single_threshold.js > PRE-CREATION > ambari-web/test/aliases/computed/gtProperties.js f3f1def > > ambari-web/test/views/main/dashboard/widgets/text_widget_single_threshold_test.js > PRE-CREATION > > Diff: https://reviews.apache.org/r/47252/diff/ > > > Testing > --- > > Yes, All tests passed. Test was already in place. > > > Thanks, > > Goutam Tadi > >
Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ
> On May 13, 2016, 10:08 a.m., Alexander Denissov wrote: > > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py, line > > 2041 > > <https://reviews.apache.org/r/47328/diff/1/?file=1382038#file1382038line2041> > > > > should HAWQ/PXF tests be now moved to separate service advisor tests > > files, perhaps as a separate effort ? Refer https://issues.apache.org/jira/browse/AMBARI-16647 > On May 13, 2016, 10:08 a.m., Alexander Denissov wrote: > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py, > > line 166 > > <https://reviews.apache.org/r/47328/diff/1/?file=1382035#file1382035line166> > > > > fancy, but this is a constant that we don;t have to calculate every > > time, why not just say > > > > MEM_THRESHOLD = 33554432 # 32GB in KB Will update - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47328/#review133146 --- On May 12, 2016, 5:58 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47328/ > --- > > (Updated May 12, 2016, 5:58 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Goutam > Tadi, jun aoki, and Lav Jain. > > > Bugs: AMBARI-16646 > https://issues.apache.org/jira/browse/AMBARI-16646 > > > Repository: ambari > > > Description > --- > > Consider all HAWQ host machines. > If the lowest memory among all systems is >= 32 GB set vm.overcommit_memory > as 2, else use 1 > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > a26a398 > ambari-server/src/main/resources/stacks/service_advisor.py 86682c5 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > PRE-CREATION > ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py > be0f3e4 > > Diff: https://reviews.apache.org/r/47328/diff/ > > > Testing > --- > > Added unit tests. > > ``` > bash-3.2$ cd ambari-server/src/test/python/stacks/2.3/common/ > bash-3.2$ python -m discover -v > test_createComponentLayoutRecommendations_hawq_1_Host > (test_stack_advisor.TestHDP23StackAdvisor) ... ServiceAdvisor implementation > for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSTANDBY is recommended on a 3-node cluster ... ServiceAdvisor > implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT does not get recommended during Add Service Wizard, > when HAWQ has already been installed ... ServiceAdvisor implementation for > service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, > when HAWQ is selected for installation ... ServiceAdvisor implementation for > service HAWQ was loaded > ServiceAdvisor implementation for service HAWQ was loaded > Se
Re: Review Request 47328: Set vm.overcommit_memory dynamically for HAWQ
YARN_HA, Not Kerberized ... 2016-05-12 17:56:32,049 - Executing HAWQ Check ... ok test_run_hawq_check_case6 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 6: Non HDFS-HA, YARN Resource Management YARN_HA, Kerberized ... 2016-05-12 17:56:32,055 - Executing HAWQ Check ... ok test_run_hawq_check_case7 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 7: HDFS-HA, Standalone Resource Management, Not Kerberized ... 2016-05-12 17:56:32,061 - Executing HAWQ Check ... ok test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, Kerberized ... 2016-05-12 17:56:32,070 - Executing HAWQ Check ... ok test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not Kerberized ... 2016-05-12 17:56:32,078 - Executing HAWQ Check ... ok test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-05-12 17:56:32,086 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to false or is not available in hawq-env.xml ok test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster) Run Stop HAWQ Cluster Immediate Mode ... ok test_stop_default (test_hawqmaster.TestHawqMaster) Run Stop HAWQMASTER ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok test_getServiceConfigurationRecommendations (test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok -- Ran 51 tests in 0.226s OK ``` Thanks, Matt
Re: Review Request 47379: PXF operations considers the agent status
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47379/#review133561 --- Ship it! Ship It! - Matt On May 13, 2016, 5:27 p.m., Goutam Tadi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47379/ > --- > > (Updated May 13, 2016, 5:27 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, jun aoki, > Lav Jain, and Matt. > > > Bugs: AMBARI-16670 > https://issues.apache.org/jira/browse/AMBARI-16670 > > > Repository: ambari > > > Description > --- > > PXF operations considers the agent status > > > Diffs > - > > ambari-web/app/controllers/main/service/item.js 9250bfc > ambari-web/app/models/host_component.js 569879f > ambari-web/test/controllers/main/service/item_test.js 383aa2f > > Diff: https://reviews.apache.org/r/47379/diff/ > > > Testing > --- > > Yes. > > User should be able to do the following ops on PXF: > Start PXF slaves if one or more slaves are down > Stop PXF slaves if one or more slaves are up > Start disabled if all agents are up > Stop disabled if all agents are down > Service Check disabled if one or more slaves are down > Service Check enabled if all agents are up > > > Thanks, > > Goutam Tadi > >
Re: Review Request 47434: Remove unused parameters from hawq-site.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47434/#review133630 --- Ship it! Ship It! - Matt On May 16, 2016, 5:14 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47434/ > --- > > (Updated May 16, 2016, 5:14 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16694 > https://issues.apache.org/jira/browse/AMBARI-16694 > > > Repository: ambari > > > Description > --- > > Remove the below unused parameters from hawq-site.xml > hawq_re_cgroup_hierarchy_name > hawq_re_cgroup_mount_point > hawq_re_cpu_enable > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml > 170e8cf > > Diff: https://reviews.apache.org/r/47434/diff/ > > > Testing > --- > > yes. > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 47475: clean up import * for Hive service scripts in common-services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47475/#review133824 --- Ship it! Ship It! - Matt On May 18, 2016, 9:48 a.m., Juanjo Marron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47475/ > --- > > (Updated May 18, 2016, 9:48 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Matt. > > > Bugs: AMBARI-16189 > https://issues.apache.org/jira/browse/AMBARI-16189 > > > Repository: ambari > > > Description > --- > > Python code at at common-services level used generic imports form > resource_management (from resource_management import *) > Ideally, for easier code tracking and performance, these import should be > more specific, such as: > from resource_management.libraries.script.script import Script > from resource_management.core.resources.system import Directory > This patch cleans up import * from resource_management for Hive service > scripts in common-services > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py > 45dcbe7 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py > b37698e > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_service_check.py > 07b4095 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py > da49450 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_interactive.py > e8532da > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py > d0dd9bb > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_server.py > 851dc02 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_service.py > 8b98ed1 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_users.py > c023548 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_utils.py > 5006b56 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_windows.py > 880fdb5 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py > f7304f7 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py > 3acbc7b > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py > 34687c4 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py > c24db4c > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service_check.py > 8e80d48 > > Diff: https://reviews.apache.org/r/47475/diff/ > > > Testing > --- > > Pass mvn clean test -DskipSurefireTests > HIVE fresh installation and service check > > > https://builds.apache.org/job/Ambari-trunk-test-patch/6869 > > > Hadoop QA added a comment - 8 minutes ago > -1 overall. Here are the results of testing the latest attachment > http://issues.apache.org/jira/secure/attachment/12802696/AMBARI-16189.patch > against trunk revision . > +1 @author. The patch does not contain any @author tags. > -1 tests included. The patch doesn't appear to include any new or modified > tests. > Please justify why no new tests are needed for this patch. > Also please list what manual steps were performed to verify this patch. > +1 javac. The applied patch does not increase the total number of javac > compiler warnings. > +1 release audit. The applied patch does not increase the total number of > release audit warnings. > +1 core tests. The patch passed unit tests in ambari-server. > Test results: > https://builds.apache.org/job/Ambari-trunk-test-patch/6869//testReport/ > Console output: > https://builds.apache.org/job/Ambari-trunk-test-patch/6869//console > This message is automatically generated. > > > Thanks, > > Juanjo Marron > >
Re: Review Request 47541: clean up import * for HBase service scripts in common-services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47541/#review133825 --- Ship it! Ship It! - Matt On May 18, 2016, 9:41 a.m., Juanjo Marron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47541/ > --- > > (Updated May 18, 2016, 9:41 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Matt. > > > Bugs: AMBARI-16408 > https://issues.apache.org/jira/browse/AMBARI-16408 > > > Repository: ambari > > > Description > --- > > Python code at at common-services level used generic imports form > resource_management (from resource_management import *) > Ideally, for easier code tracking and performance, these import should be > more specific, such as: > from resource_management.libraries.script.script import Script > from resource_management.core.resources.system import Directory > This patch cleans up import * from resource_management for Hive service > scripts in common-services > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py > d2f28f5 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py > f35a570 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_decommission.py > 022465a > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_master.py > 840411f > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_regionserver.py > 4001912 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py > a6904f6 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_upgrade.py > 2dc9883 > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_windows.py > 7d634cb > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/service_check.py > 6ecb58e > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py > b1a19e6 > > Diff: https://reviews.apache.org/r/47541/diff/ > > > Testing > --- > > Pass mvn clean test -DskipSurefireTests > Hbase fresh installation and service check > > Hadoop QA added a comment - 2 hours ago > -1 overall. Here are the results of testing the latest attachment > http://issues.apache.org/jira/secure/attachment/12803349/AMBARI-16408.patch > against trunk revision . > +1 @author. The patch does not contain any @author tags. > -1 tests included. The patch doesn't appear to include any new or modified > tests. > Please justify why no new tests are needed for this patch. > Also please list what manual steps were performed to verify this patch. > +1 javac. The applied patch does not increase the total number of javac > compiler warnings. > +1 release audit. The applied patch does not increase the total number of > release audit warnings. > +1 core tests. The patch passed unit tests in ambari-server. > Test results: > https://builds.apache.org/job/Ambari-trunk-test-patch/6878//testReport/ > Console output: > https://builds.apache.org/job/Ambari-trunk-test-patch/6878//console > > > Thanks, > > Juanjo Marron > >
Re: Review Request 47560: clean up import * for Kerberos service scripts in common-services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47560/#review133826 --- Ship it! Ship It! - Matt On May 18, 2016, 3:14 p.m., Juanjo Marron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47560/ > --- > > (Updated May 18, 2016, 3:14 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Matt. > > > Bugs: AMBARI-16645 > https://issues.apache.org/jira/browse/AMBARI-16645 > > > Repository: ambari > > > Description > --- > > Python code at at common-services level used generic imports form > resource_management (from resource_management import *) > Ideally, for easier code tracking and performance, these import should be > more specific, such as: > from resource_management.libraries.script.script import Script > from resource_management.core.resources.system import Directory > This patch cleans up import * from resource_management for Hive service > scripts in common-services > > > Diffs > - > > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_common.py > e06b13f > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/params.py > 906d521 > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py > 7c09171 > > ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/package/scripts/status_params.py > bbae4a3 > > Diff: https://reviews.apache.org/r/47560/diff/ > > > Testing > --- > > Pass mvn clean test -DskipSurefireTests > Kerberos client installation > > Hadoop QA added a comment - 7 minutes ago > -1 overall. Here are the results of testing the latest attachment > http://issues.apache.org/jira/secure/attachment/12803967/AMBARI-16645.patch > against trunk revision . > +1 @author. The patch does not contain any @author tags. > -1 tests included. The patch doesn't appear to include any new or modified > tests. > Please justify why no new tests are needed for this patch. > Also please list what manual steps were performed to verify this patch. > +1 javac. The applied patch does not increase the total number of javac > compiler warnings. > +1 release audit. The applied patch does not increase the total number of > release audit warnings. > +1 core tests. The patch passed unit tests in ambari-server. > Test results: > https://builds.apache.org/job/Ambari-trunk-test-patch/6885//testReport/ > Console output: > https://builds.apache.org/job/Ambari-trunk-test-patch/6885//console > This message is automatically generated. > > > Thanks, > > Juanjo Marron > >
Re: Review Request 47477: clean up import * for HDFS service scripts in common-services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47477/#review133827 --- Ship it! Ship It! - Matt On May 18, 2016, 9:49 a.m., Juanjo Marron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47477/ > --- > > (Updated May 18, 2016, 9:49 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Matt. > > > Bugs: AMBARI-16362 > https://issues.apache.org/jira/browse/AMBARI-16362 > > > Repository: ambari > > > Description > --- > > Python code at at common-services level used generic imports form > resource_management (from resource_management import *) > Ideally, for easier code tracking and performance, these import should be > more specific, such as: > from resource_management.libraries.script.script import Script > from resource_management.core.resources.system import Directory > This patch cleans up import * from resource_management for HDFS service > scripts in common-services > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py > 79c32e4 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py > 1264284 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py > d9b76b8 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_datanode.py > a22fb99 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_snamenode.py > 500ed15 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py > 9c5a124 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py > b37f413 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_windows.py > da00985 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/service_check.py > 737ae04 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/snamenode.py > 30eee07 > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py > ceee738 > > Diff: https://reviews.apache.org/r/47477/diff/ > > > Testing > --- > > Pass mvn clean test -DskipSurefireTests > HDFS fresh installation and service check > Hadoop QA: > -1 overall. Here are the results of testing the latest attachment > http://issues.apache.org/jira/secure/attachment/12803107/AMBARI-16362.patch > against trunk revision . > +1 @author. The patch does not contain any @author tags. > -1 tests included. The patch doesn't appear to include any new or modified > tests. > Please justify why no new tests are needed for this patch. > Also please list what manual steps were performed to verify this patch. > +1 javac. The applied patch does not increase the total number of javac > compiler warnings. > +1 release audit. The applied patch does not increase the total number of > release audit warnings. > +1 core tests. The patch passed unit tests in ambari-server. > Test results: > https://builds.apache.org/job/Ambari-trunk-test-patch/6870//testReport/ > Console output: > https://builds.apache.org/job/Ambari-trunk-test-patch/6870//console > This message is automatically generated. > > > Thanks, > > Juanjo Marron > >
Review Request 47564: Activate HAWQSTANDBY fails on retry
HAWQ Check Case 5: Non HDFS-HA, YARN Resource Management YARN_HA, Not Kerberized ... 2016-05-18 16:27:14,713 - Executing HAWQ Check ... ok test_run_hawq_check_case6 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 6: Non HDFS-HA, YARN Resource Management YARN_HA, Kerberized ... 2016-05-18 16:27:14,719 - Executing HAWQ Check ... ok test_run_hawq_check_case7 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 7: HDFS-HA, Standalone Resource Management, Not Kerberized ... 2016-05-18 16:27:14,725 - Executing HAWQ Check ... ok test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, Kerberized ... 2016-05-18 16:27:14,732 - Executing HAWQ Check ... ok test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not Kerberized ... 2016-05-18 16:27:14,738 - Executing HAWQ Check ... ok test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-05-18 16:27:14,745 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to false or is not available in hawq-env.xml ok test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster) Run Stop HAWQ Cluster Immediate Mode ... ok test_stop_default (test_hawqmaster.TestHawqMaster) Run Stop HAWQMASTER ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_activate_hawq_standby (test_hawqstandby.TestHawqStandby) Test Activate HAWQ Standby Command ... 2016-05-18 16:27:14,783 - Activating HAWQ standby... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok test_getServiceConfigurationRecommendations (test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok -- Ran 52 tests in 0.199s OK Thanks, Matt
Re: Review Request 47570: Recommend value for hawq_rm_memory_limit_perseg for HAWQ
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47570/#review133849 --- Ship it! Ship It! - Matt On May 18, 2016, 5:19 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47570/ > --- > > (Updated May 18, 2016, 5:19 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16745 > https://issues.apache.org/jira/browse/AMBARI-16745 > > > Repository: ambari > > > Description > --- > > Recommend value for hawq_rm_memory_limit_perseg for HAWQ. Below is the > recommendation basis. > vm.overcommit_memory recommedation existing > 1) If RAM >=32 then 2 else 1 > > For recommendation % for memory based on the size of cluster > RAM = RAM / 2 if vm.overcommit == 2 else 1 > 1) If HAWQ Segment Hosts has RAM <= 64 GB, allocate 75% > 2) If HAWQ Segment Hosts has 64 < RAM <= 512, allocate 85% > 3) If HAWQ Segment Hosts has 512 < RAM, allocate 95% > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py > 276dd3a > ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py > 50f3a1f > > Diff: https://reviews.apache.org/r/47570/diff/ > > > Testing > --- > > yes. > > > test_getServiceConfigurationRecommendations > (test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok > > > -- > Ran 51 tests in 0.317s > > OK > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 47573: Next button is disabled on assign master page in Add Hawq Standby wizard
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47573/#review133864 --- ambari-web/app/views/main/admin/highAvailability/hawq/addStandby/wizard_view.js (line 64) <https://reviews.apache.org/r/47573/#comment198511> Is this something new that applies only to trunk and branch-2.4? - Matt On May 18, 2016, 6:29 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47573/ > --- > > (Updated May 18, 2016, 6:29 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16754 > https://issues.apache.org/jira/browse/AMBARI-16754 > > > Repository: ambari > > > Description > --- > > Next button is disabled on assign master page in Add Hawq Standby wizard. > Host maintenance state is not propageted from the wizard view to the > underlying controllers. > > > Diffs > - > > > ambari-web/app/views/main/admin/highAvailability/hawq/addStandby/wizard_view.js > def93f3 > > Diff: https://reviews.apache.org/r/47573/diff/ > > > Testing > --- > > yes. > > > 27821 tests complete (31 seconds) > 154 tests pending > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 01:32 min > [INFO] Finished at: 2016-05-18T18:26:48-07:00 > [INFO] Final Memory: 14M/339M > > > Thanks, > > bhuvnesh chaudhary > >
Review Request 47580: Add pxf-hbase.jar to HBASE_CLASSPATH through service_advisor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47580/ --- Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav Jain. Bugs: AMBARI-16752 https://issues.apache.org/jira/browse/AMBARI-16752 Repository: ambari Description --- In service install wizard, pxf-hbase.jar should be added to the HBASE_CLASSPATH (recommendation and validation) Diffs - ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py e2510eb ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py PRE-CREATION Diff: https://reviews.apache.org/r/47580/diff/ Testing --- Manually Tested. Unit tests added. Hadoop QA: ``` +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12804840/AMBARI-16752-trunk-orig.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in ambari-server. Test results: https://builds.apache.org/job/Ambari-trunk-test-patch/6889//testReport/ Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/6889//console This message is automatically generated. ``` Thanks, Matt
Re: Review Request 47564: Activate HAWQSTANDBY fails on retry
> On May 18, 2016, 6:32 p.m., Lav Jain wrote: > > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py, line 197 > > <https://reviews.apache.org/r/47564/diff/1/?file=1387503#file1387503line197> > > > > You may want to take this opportunity to refactor the test code. > > > > ExecuteScript can be a separate function with command passed in as > > parameter. > > > > Similarily, common.stop_component can be refactored into a separate > > function with both test_stop_default and test_activate_hawq_standby calling > > it. > > > > If you want to go one step further, create a base class of > > TestHawqCommon(RMFTestCase) that defines the corresponding routines in > > common.py with TestHawqStandby(TestHawqCommon). > > > > assertResourceCalled can be defined using functools.partial I did think about refactoring while working on this JIRA. And I thought should do the refactoring separately as part of refactoring HAWQ's python unit tests. I did not want to overdo with the refactoring at this point. I've added the JIRA: https://issues.apache.org/jira/browse/AMBARI-16758 - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47564/#review133857 --- On May 18, 2016, 4:28 p.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47564/ > --- > > (Updated May 18, 2016, 4:28 p.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav > Jain. > > > Bugs: AMBARI-16750 > https://issues.apache.org/jira/browse/AMBARI-16750 > > > Repository: ambari > > > Description > --- > > If wizard failed on the Activate Standby step, retrying the step would cause > a failure, because the master hosts have been switched already in the > configuration. > Resolution: Before retrying the command, rewrite the configuration with > current topology. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/hawqstandby.py > 45dcff0 > ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py e247d28 > > Diff: https://reviews.apache.org/r/47564/diff/ > > > Testing > --- > > Manually Tested. > > Added unit test. > > test_hawq_master_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_segment_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_hawq_standby_critical > (test_alert_component_status.TestAlertComponentStatus) ... ok > test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_missing_configs (test_alert_component_status.TestAlertComponentStatus) > ... ok > test_exception_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No > handlers could be found for logger "ambari_alerts" > ok > test_missing_configs > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_slave_file > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_successful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_empty_db_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_unsuccessful_registration_status_plural > (test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok > test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... > ok > test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok > test_
Re: Review Request 47573: Next button is disabled on assign master page in Add Hawq Standby wizard
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47573/#review133890 --- Ship it! Ship It! - Matt On May 18, 2016, 6:29 p.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47573/ > --- > > (Updated May 18, 2016, 6:29 p.m.) > > > Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr > Diachenko. > > > Bugs: AMBARI-16754 > https://issues.apache.org/jira/browse/AMBARI-16754 > > > Repository: ambari > > > Description > --- > > Next button is disabled on assign master page in Add Hawq Standby wizard. > Host maintenance state is not propageted from the wizard view to the > underlying controllers. > > > Diffs > - > > > ambari-web/app/views/main/admin/highAvailability/hawq/addStandby/wizard_view.js > def93f3 > > Diff: https://reviews.apache.org/r/47573/diff/ > > > Testing > --- > > yes. > > > 27821 tests complete (31 seconds) > 154 tests pending > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 01:32 min > [INFO] Finished at: 2016-05-18T18:26:48-07:00 > [INFO] Final Memory: 14M/339M > > > Thanks, > > bhuvnesh chaudhary > >
Re: Review Request 47580: Add pxf-hbase.jar to HBASE_CLASSPATH through service_advisor
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
test_execute (test_alerts_api_status.TestAlertsApiStatus) ... No handlers could
be found for logger "ambari_alerts"
ok
test_get_pxf_protocol_version (test_alerts_api_status.TestAlertsApiStatus) ...
ok
test_configure_default (test_pxf.TestPxf) ... ok
test_install_default (test_pxf.TestPxf) ... ok
test_start_default (test_pxf.TestPxf) ... ok
test_status_default (test_pxf.TestPxf) ... ok
test_stop_default (test_pxf.TestPxf) ... ok
test_getConfigurationsValidationItems
(test_service_advisor.TestPXF300ServiceAdvisor) ... ok
test_getServiceConfigurationRecommendations
(test_service_advisor.TestPXF300ServiceAdvisor) ... ok
--
Ran 9 tests in 0.033s
OK
SiteName: hbase-env, method: validatePXFHBaseEnvConfigurations
Site properties: {'content': '# Some hbase-env content text'}
Recommendations: {'content': '# Some hbase-env content text'}
```
Thanks,
Matt
Re: Review Request 47580: Add pxf-hbase.jar to HBASE_CLASSPATH through service_advisor
was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service PXF was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
ServiceAdvisor implementation for service HAWQ was loaded
test_execute (test_alerts_api_status.TestAlertsApiStatus) ... No handlers could
be found for logger "ambari_alerts"
ok
test_get_pxf_protocol_version (test_alerts_api_status.TestAlertsApiStatus) ...
ok
test_configure_default (test_pxf.TestPxf) ... ok
test_install_default (test_pxf.TestPxf) ... ok
test_start_default (test_pxf.TestPxf) ... ok
test_status_default (test_pxf.TestPxf) ... ok
test_stop_default (test_pxf.TestPxf) ... ok
test_getConfigurationsValidationItems
(test_service_advisor.TestPXF300ServiceAdvisor) ... ok
test_getServiceConfigurationRecommendations
(test_service_advisor.TestPXF300ServiceAdvisor) ... ok
--
Ran 9 tests in 0.033s
OK
SiteName: hbase-env, method: validatePXFHBaseEnvConfigurations
Site properties: {'content': '# Some hbase-env content text'}
Recommendations: {'content': '# Some hbase-env content text'}
```
Thanks,
Matt
Re: Review Request 47580: Add pxf-hbase.jar to HBASE_CLASSPATH through service_advisor
> On May 19, 2016, 10:14 a.m., Lav Jain wrote: > > ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py, line > > 29 > > <https://reviews.apache.org/r/47580/diff/1/?file=1387758#file1387758line29> > > > > Not sure if loading of PXFServiceAdvisor is necessary. The test should > > call stackAdvisor directly and that should automatically load PXF service > > advisor for the corresponding routines. Unit test would call service_advisor, and stack_advisor loading will be deleted in the future. This would be fixed after https://issues.apache.org/jira/browse/AMBARI-16663 - Matt --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47580/#review133974 --- On May 19, 2016, 11:58 a.m., Matt wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47580/ > --- > > (Updated May 19, 2016, 11:58 a.m.) > > > Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav > Jain. > > > Bugs: AMBARI-16752 > https://issues.apache.org/jira/browse/AMBARI-16752 > > > Repository: ambari > > > Description > --- > > In service install wizard, pxf-hbase.jar should be added to the > HBASE_CLASSPATH (recommendation and validation) > > > Diffs > - > > > ambari-server/src/main/resources/common-services/PXF/3.0.0/service_advisor.py > e2510eb > ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py > PRE-CREATION > > Diff: https://reviews.apache.org/r/47580/diff/ > > > Testing > --- > > Manually Tested. > > Unit tests added. > > ``` > test_createComponentLayoutRecommendations_hawq_1_Host > (test_stack_advisor.TestHDP23StackAdvisor) ... ok > test_createComponentLayoutRecommendations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSTANDBY is recommended on a 3-node cluster ... ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT does not get recommended during Add Service Wizard, > when HAWQ has already been installed ... ok > test_createComponentLayoutRecommendations_hawqsegment_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Add Service Wizard, > when HAWQ is selected for installation ... ok > test_createComponentLayoutRecommendations_hawqsegment_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that HAWQSEGMENT gets recommended correctly during Cluster Install > Wizard, when HAWQ is selected for installation ... ok > test_createComponentLayoutRecommendations_no_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test no failures when there are no HAWQ components ... ok > test_createComponentLayoutRecommendations_pxf_add_service_wizard_already_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF does not get recommended during Add Service Wizard, when PXF > has already been installed ... ok > test_createComponentLayoutRecommendations_pxf_add_service_wizard_to_be_installed > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF gets recommended correctly during Add Service Wizard, when PXF > is selected for installation ... ok > test_createComponentLayoutRecommendations_pxf_cluster_install > (test_stack_advisor.TestHDP23StackAdvisor) > Test that PXF gets recommended correctly during Cluster Install Wizard, when > PXF is selected for installation ... ok > test_getComponentLayoutValidations_hawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test layout validations for HAWQ components on a 3-node cluster ... ok > test_getComponentLayoutValidations_hawqsegment_not_co_located_with_datanode > (test_stack_advisor.TestHDP23StackAdvisor) > Test validation warning for HAWQ segment not colocated with DATANODE ... ok > test_getComponentLayoutValidations_nohawq_3_Hosts > (test_stack_advisor.TestHDP23StackAdvisor) > Test no failures when there are no HAWQ components on a 3-node cluster ... ok > test_getComponentLayoutValidations_pxf_co_located_with_nn_and_dn > (test_stack_advisor.TestHDP23StackAdvisor) > Test NO warning is generated when PXF is co-located with NAMENODE and > DATANODE ... ok > test_getComponentLayoutValidations_pxf_not_co_located_with_dn > (test_stack_advisor.TestHDP23StackAdvisor) > Test war
Review Request 47616: Remove ams-ssl-client from HAWQ metainfo.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47616/ --- Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, and Lav Jain. Bugs: AMBARI-16783 https://issues.apache.org/jira/browse/AMBARI-16783 Repository: ambari Description --- Remove ams-ssl-client from HAWQ metainfo.xml Diffs - ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml 1ea2ed1 Diff: https://reviews.apache.org/r/47616/diff/ Testing --- Manually tested. Thanks, Matt
Re: Review Request 47619: clean up import * for MAHOUT service scripts in common-services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47619/#review134111 --- Ship it! Ship It! - Matt On May 19, 2016, 4:27 p.m., Juanjo Marron wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47619/ > --- > > (Updated May 19, 2016, 4:27 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jayush Luniya, and Matt. > > > Bugs: AMBARI-16671 > https://issues.apache.org/jira/browse/AMBARI-16671 > > > Repository: ambari > > > Description > --- > > Python code at at common-services level used generic imports form > resource_management (from resource_management import *) > Ideally, for easier code tracking and performance, these import should be > more specific, such as: > from resource_management.libraries.script.script import Script > from resource_management.core.resources.system import Directory > This patch cleans up import * from resource_management for Hive service > scripts in common-services > > > Diffs > - > > > ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout.py > f2c3c18 > > ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py > b6fc153 > > ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/service_check.py > 11a85e1 > > Diff: https://reviews.apache.org/r/47619/diff/ > > > Testing > --- > > Pass mvn clean test -DskipSurefireTests > MAHOUT fresh installation and service check > > Hadoop QA added a comment - 2 hours ago > -1 overall. Here are the results of testing the latest attachment > http://issues.apache.org/jira/secure/attachment/12804463/AMBARI-16671.patch > against trunk revision . > +1 @author. The patch does not contain any @author tags. > -1 tests included. The patch doesn't appear to include any new or modified > tests. > Please justify why no new tests are needed for this patch. > Also please list what manual steps were performed to verify this patch. > +1 javac. The applied patch does not increase the total number of javac > compiler warnings. > +1 release audit. The applied patch does not increase the total number of > release audit warnings. > +1 core tests. The patch passed unit tests in ambari-server. > Test results: > https://builds.apache.org/job/Ambari-trunk-test-patch/6906//testReport/ > Console output: > https://builds.apache.org/job/Ambari-trunk-test-patch/6906//console > > > Thanks, > > Juanjo Marron > >
Review Request 47642: Add HAWQ Standby wizard should recommend HAWQSTANDBY correctly
Management, Not Kerberized ... 2016-05-20 01:12:04,865 - Executing HAWQ Check ... ok test_run_hawq_check_case8 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 8: HDFS-HA, Standalone Resource Management, Kerberized ... 2016-05-20 01:12:04,872 - Executing HAWQ Check ... ok test_run_hawq_check_case9 (test_hawqmaster.TestHawqMaster) Running HAWQ Check Case 9: HDFS-HA, YARN Resource Management Non YARN_HA, Not Kerberized ... 2016-05-20 01:12:04,878 - Executing HAWQ Check ... ok test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-05-20 01:12:04,885 - Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to false or is not available in hawq-env.xml ok test_stop_cluster_immediate (test_hawqmaster.TestHawqMaster) Run Stop HAWQ Cluster Immediate Mode ... ok test_stop_default (test_hawqmaster.TestHawqMaster) Run Stop HAWQMASTER ... ok test_configure_default (test_hawqsegment.TestHawqSegment) ... ok test_install_default (test_hawqsegment.TestHawqSegment) ... ok test_start_default (test_hawqsegment.TestHawqSegment) ... ok test_stop_default (test_hawqsegment.TestHawqSegment) ... ok test_activate_hawq_standby (test_hawqstandby.TestHawqStandby) Test Activate HAWQ Standby Command ... 2016-05-20 01:12:04,930 - Activating HAWQ standby... ok test_configure_default (test_hawqstandby.TestHawqStandby) ... ok test_install_default (test_hawqstandby.TestHawqStandby) ... ok test_start_default (test_hawqstandby.TestHawqStandby) ... ok test_stop_default (test_hawqstandby.TestHawqStandby) ... ok test_getHostsForMasterComponent (test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok test_getServiceConfigurationRecommendations (test_service_advisor.TestHAWQ200ServiceAdvisor) ... ok test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok -- Ran 53 tests in 0.240s OK Thanks, Matt
Re: Review Request 47657: Hawq password to use !h flag to ensure that its only hidden and no special processing is done for it.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47657/#review134186 --- Ship it! Ship It! - Matt On May 20, 2016, 11:59 a.m., bhuvnesh chaudhary wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47657/ > --- > > (Updated May 20, 2016, 11:59 a.m.) > > > Review request for Ambari, Alexander Denissov, Lav Jain, and Matt. > > > Bugs: AMBARI-16800 > https://issues.apache.org/jira/browse/AMBARI-16800 > > > Repository: ambari > > > Description > --- > > Hawq password to use !h flag to ensure that its only hidden and no special > processing is done for it. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/master_helper.py > b52b477 > > Diff: https://reviews.apache.org/r/47657/diff/ > > > Testing > --- > > yes. > > > Thanks, > > bhuvnesh chaudhary > >
