[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[lisakowen]

Github user lisakowen closed the pull request at:

https://github.com/apache/incubator-hawq-docs/pull/97


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[dyozie]

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105317729
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -368,6 +348,49 @@ All of the listed steps are mandatory. This ensures 
that HAWQ service remains fu
 4. Restart HAWQ service to propagate the configuration change to all 
Ambari agents.
 
 This will synchronize the password on the host machines with the 
password that you specified in Ambari.
+
+## Updating HAWQ Configuration Without 
Cluster Restart
+There may be circumstances, such as during dynamic cluster expansion, when 
you must update HAWQ configuration parameters but cannot tolerate cluster 
downtime in your Ambari-managed cluster. In these situations, perform the 
following workaround to update and reload HAWQ configuration changes without a 
complete cluster restart.
+
+**Note**: Use this procedure for setting or updating only those HAWQ 
server configuration parameters with a set classification of reload.
+
+1.  Use the Ambari console UI to set and save the desired HAWQ 
configuration parameter(s) via the **HAWQ** service **Settings**, **Configs 
>Settings**, and/or **Configs >Advanced** tabs. Make sure to note the 
configuration parameter names and new values. You may need to hover the mouse 
over the configuration field to identify the HAWQ parameter name.
+
+**Note**: After updating the configuration parameters, do *not* 
restart the HAWQ service. 
+
+1.  Use HAWQ command line utilities to update the same HAWQ configuration 
parameters:
+1. Log in to the HAWQ master host as a HAWQ administrator and source 
greenplum_path.sh:
+
+``` shell
+$ ssh gpadmin@
+gpadmin@master$ source /usr/local/hawq/greenplum_path.sh
+```
+
+2. Identify the current value of the HAWQ configuration parameter:
+
+``` shell
+gpadmin@master$ hawq config -s 
+```
+
+3. Set the configuration parameter to the new value:
+
+``` shell
+gpadmin@master$ hawq config -c  -v 
+```
+
+Perform Steps 2 and 3 for each configuration parameter you set or 
updated via Ambari.
+
+3. Reload the HAWQ configuration; this operation does not restart the 
cluster:
+
+``` shell
+gpadmin@master$ hawq stop cluster --reload
+```
+
+4. Verify that each parameter was updated:
+
+```shell
+gpadmin@master$ hawq config -s 
+```
--- End diff --

I guess I was misunderstanding something from the earlier workaround:  
"When the HAWQ service is ready to be restarted via Ambari, Ambari will refresh 
the new configurations."  I guess I'm not entirely sure what that means either. 
 So when exactly is the change "complete"?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[lisakowen]

Github user lisakowen commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105307485
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -206,44 +208,22 @@ First ensure that the new node(s) has been configured 
per the instructions found
 
 23.  (Optional.) If you enabled temporary password-based authentication 
while preparing/configuring your HAWQ host systems, turn off password-based 
authentication as described in [Apache HAWQ System 
Requirements](../requirements/system-requirements.html#topic_pwdleh).
 
- Manually Updating the HAWQ 
Configuration
-If you need to expand your HAWQ cluster without restarting the HAWQ 
service, follow these steps to manually apply the new HAWQ configuration. (Use 
these steps *instead* of following Step 7 in the above procedure.):
+ Cluster Expansion Workaround 
+If you need to expand your HAWQ cluster without restarting the HAWQ 
service, follow this procedure to manually apply the HAWQ configuration changes 
required for this operation. (Perform these steps *instead of* following Step 
9b in the above procedure.)
 
-1.  Update your configuration to use the new 
`default_hash_table_bucket_number` value that you calculated:
-  1. SSH into the HAWQ master host as the `gpadmin` user:
-```shell
-$ ssh gpadmin@
-```
-   2. Source the `greenplum_path.sh` file to update the shell environment:
-```shell
-$ source /usr/local/hawq/greenplum_path.sh
-```
-   3. Verify the current value of `default_hash_table_bucket_number`:
-```shell
-$ hawq config -s default_hash_table_bucket_number
-```
-   4. Update `default_hash_table_bucket_number` to the new value that you 
calculated:
-```shell
-$ hawq config -c default_hash_table_bucket_number -v 
-```
-   5. Reload the configuration without restarting the cluster:
-```shell
-$ hawq stop cluster -u
-```
-   6. Verify that the `default_hash_table_bucket_number` value was updated:
-```shell
-$ hawq config -s default_hash_table_bucket_number
-```
-2.  Edit the `/usr/local/hawq/etc/slaves` file and add the new HAWQ 
hostname(s) to the end of the file. Separate multiple hosts with new lines. For 
example, after adding host4 and host5 to a cluster already contains hosts 1-3, 
the updated file contents would be:
+1.  Perform the procedure identified in [Updating HAWQ Configuration 
Without Cluster Restart](#manual-cfg-norestart) to update your configuration to 
use the new `default_hash_table_bucket_number` value that you calculated and 
set via the Ambari UI. You may skip Step 1 of this procedure.
--- End diff --

yes, repeating the config steps in the workaround is much clearer for the 
user.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[lisakowen]

Github user lisakowen commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105305989
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -368,6 +348,49 @@ All of the listed steps are mandatory. This ensures 
that HAWQ service remains fu
 4. Restart HAWQ service to propagate the configuration change to all 
Ambari agents.
 
 This will synchronize the password on the host machines with the 
password that you specified in Ambari.
+
+## Updating HAWQ Configuration Without 
Cluster Restart
+There may be circumstances, such as during dynamic cluster expansion, when 
you must update HAWQ configuration parameters but cannot tolerate cluster 
downtime in your Ambari-managed cluster. In these situations, perform the 
following workaround to update and reload HAWQ configuration changes without a 
complete cluster restart.
+
+**Note**: Use this procedure for setting or updating only those HAWQ 
server configuration parameters with a set classification of reload.
+
+1.  Use the Ambari console UI to set and save the desired HAWQ 
configuration parameter(s) via the **HAWQ** service **Settings**, **Configs 
>Settings**, and/or **Configs >Advanced** tabs. Make sure to note the 
configuration parameter names and new values. You may need to hover the mouse 
over the configuration field to identify the HAWQ parameter name.
+
+**Note**: After updating the configuration parameters, do *not* 
restart the HAWQ service. 
+
+1.  Use HAWQ command line utilities to update the same HAWQ configuration 
parameters:
+1. Log in to the HAWQ master host as a HAWQ administrator and source 
greenplum_path.sh:
+
+``` shell
+$ ssh gpadmin@
+gpadmin@master$ source /usr/local/hawq/greenplum_path.sh
+```
+
+2. Identify the current value of the HAWQ configuration parameter:
+
+``` shell
+gpadmin@master$ hawq config -s 
+```
+
+3. Set the configuration parameter to the new value:
+
+``` shell
+gpadmin@master$ hawq config -c  -v 
+```
+
+Perform Steps 2 and 3 for each configuration parameter you set or 
updated via Ambari.
+
+3. Reload the HAWQ configuration; this operation does not restart the 
cluster:
+
+``` shell
+gpadmin@master$ hawq stop cluster --reload
+```
+
+4. Verify that each parameter was updated:
+
+```shell
+gpadmin@master$ hawq config -s 
+```
--- End diff --

can you clarify what you are looking for here, @dyozie?  the config was 
changed first in ambari (local only) and then via the CLI (changed in running 
cluster).  when ambari is restarted, it should effectively then know about the 
change that was made via the CLI.  i don't think it would be new information, 
would it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[dyozie]

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105213393
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -206,44 +208,22 @@ First ensure that the new node(s) has been configured 
per the instructions found
 
 23.  (Optional.) If you enabled temporary password-based authentication 
while preparing/configuring your HAWQ host systems, turn off password-based 
authentication as described in [Apache HAWQ System 
Requirements](../requirements/system-requirements.html#topic_pwdleh).
 
- Manually Updating the HAWQ 
Configuration
-If you need to expand your HAWQ cluster without restarting the HAWQ 
service, follow these steps to manually apply the new HAWQ configuration. (Use 
these steps *instead* of following Step 7 in the above procedure.):
+ Cluster Expansion Workaround 
+If you need to expand your HAWQ cluster without restarting the HAWQ 
service, follow this procedure to manually apply the HAWQ configuration changes 
required for this operation. (Perform these steps *instead of* following Step 
9b in the above procedure.)
 
-1.  Update your configuration to use the new 
`default_hash_table_bucket_number` value that you calculated:
-  1. SSH into the HAWQ master host as the `gpadmin` user:
-```shell
-$ ssh gpadmin@
-```
-   2. Source the `greenplum_path.sh` file to update the shell environment:
-```shell
-$ source /usr/local/hawq/greenplum_path.sh
-```
-   3. Verify the current value of `default_hash_table_bucket_number`:
-```shell
-$ hawq config -s default_hash_table_bucket_number
-```
-   4. Update `default_hash_table_bucket_number` to the new value that you 
calculated:
-```shell
-$ hawq config -c default_hash_table_bucket_number -v 
-```
-   5. Reload the configuration without restarting the cluster:
-```shell
-$ hawq stop cluster -u
-```
-   6. Verify that the `default_hash_table_bucket_number` value was updated:
-```shell
-$ hawq config -s default_hash_table_bucket_number
-```
-2.  Edit the `/usr/local/hawq/etc/slaves` file and add the new HAWQ 
hostname(s) to the end of the file. Separate multiple hosts with new lines. For 
example, after adding host4 and host5 to a cluster already contains hosts 1-3, 
the updated file contents would be:
+1.  Perform the procedure identified in [Updating HAWQ Configuration 
Without Cluster Restart](#manual-cfg-norestart) to update your configuration to 
use the new `default_hash_table_bucket_number` value that you calculated and 
set via the Ambari UI. You may skip Step 1 of this procedure.
+
+2.  Edit the `/usr/local/hawq/etc/slaves` file and add the new HAWQ 
hostname(s) to the end of the file. Identify each host on a separate line. For 
example, after adding host4 and host5 to a cluster already containing hosts 
1-3, the `slaves` file would appear similar to the following:
 
  ```
  host1
  host2
  host3
  host4
  host5
- ```
-3.  Continue with Step 8 in the previous procedure, [Expanding the HAWQ 
Cluster](#amb-expand).  When the HAWQ service is ready to be restarted via 
Ambari, Ambari will refresh the new configurations.
+ ```
+ 
+3.  Continue with Step 10 in the previous procedure, [Expanding the HAWQ 
Cluster](#amb-expand).  When the HAWQ service is ready to be restarted via 
Ambari, Ambari will refresh the new configurations.
--- End diff --

Let's insert and use an anchor to get them right back to step 10 here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[dyozie]

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105214050
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -206,44 +208,22 @@ First ensure that the new node(s) has been configured 
per the instructions found
 
 23.  (Optional.) If you enabled temporary password-based authentication 
while preparing/configuring your HAWQ host systems, turn off password-based 
authentication as described in [Apache HAWQ System 
Requirements](../requirements/system-requirements.html#topic_pwdleh).
 
- Manually Updating the HAWQ 
Configuration
-If you need to expand your HAWQ cluster without restarting the HAWQ 
service, follow these steps to manually apply the new HAWQ configuration. (Use 
these steps *instead* of following Step 7 in the above procedure.):
+ Cluster Expansion Workaround 
--- End diff --

Maybe rename this to "Workaround for Dynamic Cluster Expansion"?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[dyozie]

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105208124
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -182,20 +182,22 @@ First ensure that the new node(s) has been configured 
per the instructions found
4. Click **Confirm Add** to acknowledge the component to add. Click 
**OK** when the task completes.
5. In the Components summary, select **Add > PXF**.
6. Click **Confirm Add** to acknowledge the component to add. Click 
**OK** when the task completes.
-17. (Optional) If you are using hash tables, adjust the **Default buckets 
for Hash Distributed tables** setting (`default_hash_table_bucket_number`) on 
the HAWQ service's **Configs > Settings** tab. Update this property's value by 
multiplying the new number of nodes in the cluster by the appropriate number 
indicated below.
-
-|Number of Nodes After Expansion|Suggested 
default\_hash\_table\_bucket\_number value|
-|---|--|
-|<= 85|6 \* \#nodes|
-|\> 85 and <= 102|5 \* \#nodes|
-|\> 102 and <= 128|4 \* \#nodes|
-|\> 128 and <= 170|3 \* \#nodes|
-|\> 170 and <= 256|2 \* \#nodes|
-|\> 256 and <= 512|1 \* \#nodes|
-|\> 512|512|
-18.  Ambari requires the HAWQ service to be restarted in order to apply 
the configuration changes. If you need to apply the configuration *without* 
restarting HAWQ (for dynamic cluster expansion), then you can use the HAWQ CLI 
commands described in [Manually Updating the HAWQ 
Configuration](#manual-config-steps) *instead* of following this step.
-Stop and then start the HAWQ service to apply your 
configuration changes via Ambari. Select **Service Actions > Stop**, followed 
by **Service Actions > Start** to ensure that the HAWQ Master starts before the 
newly-added segment. During the HAWQ startup, Ambari exchanges ssh keys for the 
`gpadmin` user, and applies the new configuration.
->**Note:** Do not use the **Restart All** service action to complete 
this step.
+17. (Optional) If you are using hash tables, adjust the **Default buckets 
for Hash Distributed tables** setting (`default_hash_table_bucket_number`).
+
+1. Open the HAWQ service's **Configs > Settings** tab. Update the 
`default_hash_table_bucket_number` property's value by multiplying the new 
number of nodes in the cluster by the appropriate number indicated below.
+
+|Number of Nodes After Expansion|Suggested 
default\_hash\_table\_bucket\_number value|
+|---|--|
+|<= 85|6 \* \#nodes|
+|\> 85 and <= 102|5 \* \#nodes|
+|\> 102 and <= 128|4 \* \#nodes|
+|\> 128 and <= 170|3 \* \#nodes|
+|\> 170 and <= 256|2 \* \#nodes|
+|\> 256 and <= 512|1 \* \#nodes|
+|\> 512|512|
+18.  Ambari requires that the HAWQ service be restarted in order to 
apply the configuration changes. If you need to apply the configuration 
*without* restarting HAWQ (for dynamic cluster expansion), then use the 
procedure described in [Cluster Expansion Workaround](#manual-config-steps) 
*instead* of following this step.
--- End diff --

Reword this to "Ambari requires you to restart the HAWQ service in order to 
apply the configuration changes."


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[dyozie]

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105239316
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -368,6 +348,49 @@ All of the listed steps are mandatory. This ensures 
that HAWQ service remains fu
 4. Restart HAWQ service to propagate the configuration change to all 
Ambari agents.
 
 This will synchronize the password on the host machines with the 
password that you specified in Ambari.
+
+## Updating HAWQ Configuration Without 
Cluster Restart
+There may be circumstances, such as during dynamic cluster expansion, when 
you must update HAWQ configuration parameters but cannot tolerate cluster 
downtime in your Ambari-managed cluster. In these situations, perform the 
following workaround to update and reload HAWQ configuration changes without a 
complete cluster restart.
+
+**Note**: Use this procedure for setting or updating only those HAWQ 
server configuration parameters with a set classification of reload.
+
+1.  Use the Ambari console UI to set and save the desired HAWQ 
configuration parameter(s) via the **HAWQ** service **Settings**, **Configs 
>Settings**, and/or **Configs >Advanced** tabs. Make sure to note the 
configuration parameter names and new values. You may need to hover the mouse 
over the configuration field to identify the HAWQ parameter name.
+
+**Note**: After updating the configuration parameters, do *not* 
restart the HAWQ service. 
+
+1.  Use HAWQ command line utilities to update the same HAWQ configuration 
parameters:
+1. Log in to the HAWQ master host as a HAWQ administrator and source 
greenplum_path.sh:
+
+``` shell
+$ ssh gpadmin@
+gpadmin@master$ source /usr/local/hawq/greenplum_path.sh
+```
+
+2. Identify the current value of the HAWQ configuration parameter:
+
+``` shell
+gpadmin@master$ hawq config -s 
+```
+
+3. Set the configuration parameter to the new value:
+
+``` shell
+gpadmin@master$ hawq config -c  -v 
+```
+
+Perform Steps 2 and 3 for each configuration parameter you set or 
updated via Ambari.
+
+3. Reload the HAWQ configuration; this operation does not restart the 
cluster:
+
+``` shell
+gpadmin@master$ hawq stop cluster --reload
+```
+
+4. Verify that each parameter was updated:
+
+```shell
+gpadmin@master$ hawq config -s 
+```
--- End diff --

Seems like this section also needs a closing sentence about when the 
changes refresh in Ambari?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[dyozie]

Github user dyozie commented on a diff in the pull request:

https://github.com/apache/incubator-hawq-docs/pull/97#discussion_r105217462
  
--- Diff: markdown/admin/ambari-admin.html.md.erb ---
@@ -368,6 +348,49 @@ All of the listed steps are mandatory. This ensures 
that HAWQ service remains fu
 4. Restart HAWQ service to propagate the configuration change to all 
Ambari agents.
 
 This will synchronize the password on the host machines with the 
password that you specified in Ambari.
+
+## Updating HAWQ Configuration Without 
Cluster Restart
+There may be circumstances, such as during dynamic cluster expansion, when 
you must update HAWQ configuration parameters but cannot tolerate cluster 
downtime in your Ambari-managed cluster. In these situations, perform the 
following workaround to update and reload HAWQ configuration changes without a 
complete cluster restart.
+
+**Note**: Use this procedure for setting or updating only those HAWQ 
server configuration parameters with a set classification of reload.
--- End diff --

Need to style `reload` or at least quote it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-hawq-docs pull request #97: HAWQ-1372 - doc config change without ...

[lisakowen]

GitHub user lisakowen opened a pull request:

https://github.com/apache/incubator-hawq-docs/pull/97

HAWQ-1372 - doc config change without cluster restart for ambari-managed 
clusters

cluster restart for config change in an ambari-managed deployment may not 
be tolerated in some cases.  document the procedure to update hawq config and 
reload for ambari-managed clusters.

also including a small rework to master mirroring standby resynchronization 
section with this PR.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lisakowen/incubator-hawq-docs 
feature/HAWQ-1372-ambari-no-restart

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq-docs/pull/97.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #97


commit 1ac4e5bae744e6771f66ec0157914cf7fd593485
Author: Lisa Owen 
Date:   2017-03-02T22:12:52Z

HAWQ-1372 - ambari cfg change w/o cluster restart

commit 9610a267406f9c06722aeced484152e6ce76a877
Author: Lisa Owen 
Date:   2017-03-02T22:51:02Z

rework master mirror standby resync section




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---