HAWQ-1307 Configuring HAWQ Best Practices page (closes #93)
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/7a7a87a6 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/7a7a87a6 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/7a7a87a6 Branch: refs/heads/release/2.1.0.0-incubating Commit: 7a7a87a6584588a36955361bab6432f9901af762 Parents: 10938ae Author: Jane Beckman <[email protected]> Authored: Thu Feb 9 08:53:27 2017 -0800 Committer: David Yozie <[email protected]> Committed: Thu Feb 9 08:53:27 2017 -0800 ---------------------------------------------------------------------- .../source/subnavs/apache-hawq-nav-210.erb | 1 + .../HAWQBestPracticesOverview.html.md.erb | 3 ++ .../config_hawq_bestpractices.html.md.erb | 32 +++++++++++++++++ .../operating_hawq_bestpractices.html.md.erb | 36 +++++++++----------- 4 files changed, 52 insertions(+), 20 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb ---------------------------------------------------------------------- diff --git a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb index bede889..3316cba 100644 --- a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb +++ b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb @@ -396,6 +396,7 @@ </li> <li class="has_submenu"><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/HAWQBestPracticesOverview.html">Best Practices</a> <ul> + <li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/config_hawq_bestpractices.html">Configuring HAWQ</a></li> <li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/operating_hawq_bestpractices.html">Operating HAWQ</a></li> <li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/secure_bestpractices.html">Securing HAWQ</a></li> <li><a href="/docs/userguide/2.1.0.0-incubating/bestpractices/managing_resources_bestpractices.html">Managing Resources</a></li> http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb index 5fa7769..ddca235 100644 --- a/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb +++ b/markdown/bestpractices/HAWQBestPracticesOverview.html.md.erb @@ -23,7 +23,10 @@ under the License. This chapter provides best practices on using the components and features that are part of a HAWQ system. +- **[Best Practices for Configuring HAWQ Parameters](../bestpractices/config_hawq_bestpractices.html)** + This topic provides best practices for configuring the parameters in `hawq-site.xml`. + - **[Best Practices for Operating HAWQ](../bestpractices/operating_hawq_bestpractices.html)** This topic provides best practices for operating HAWQ, including recommendations for stopping, starting and monitoring HAWQ. http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/config_hawq_bestpractices.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/bestpractices/config_hawq_bestpractices.html.md.erb b/markdown/bestpractices/config_hawq_bestpractices.html.md.erb new file mode 100644 index 0000000..e4b27b0 --- /dev/null +++ b/markdown/bestpractices/config_hawq_bestpractices.html.md.erb @@ -0,0 +1,32 @@ +--- +title: Best Practices for Configuring HAWQ Parameters +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + + +The configuration parameters for HAWQ are maintained in the `$GPHOME/etc/hawq-site.xml` file. This file resides on all HAWQ instances and can be modified either via Ambari or by using the HAWQ command line interface. Use a consistent strategy (Ambari or command line interface) to maintain the contents of `hawq-site.xml`. + +For Ambari-managed clusters, always use Ambari for configuring `hawq-site.xml` parameters; restarting or reconfiguring HAWQ via Ambari will overwrite any configuration changes to `hawq-site.xml` made outside the Ambari interface. + +If you manage your cluster using command line tools instead of Ambari, use a consistent `hawq-site.xml` file to configure your entire cluster. + +**Note:** Do not configure HAWQ parameters through the `postgresql.conf` file. Use only `hawq-site.xml` to configure your HAWQ cluster. Any parameters defined in `hawq-site.xml` will overwrite parameters in `postgresql.conf`. + http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/7a7a87a6/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb ---------------------------------------------------------------------- diff --git a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb index b19ba25..61687df 100644 --- a/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb +++ b/markdown/bestpractices/operating_hawq_bestpractices.html.md.erb @@ -23,21 +23,14 @@ under the License. This topic provides best practices for operating HAWQ, including recommendations for stopping, starting and monitoring HAWQ. -## <a id="best_practice_config"></a>Best Practices for Configuring HAWQ Parameters +## <a id="task_qgk_bz3_1v"></a>Best Practices Using the Command Line to Start/Stop HAWQ Cluster Members -The HAWQ configuration guc/parameters are located in `$GPHOME/etc/hawq-site.xml`. This configuration file resides on all HAWQ instances and can be modified either by the Ambari interface or the command line. +The following best practices are recommended when using `hawq start` and `hawq stop` to manage your HAWQ cluster. -If you install and manage HAWQ using Ambari, use the Ambari interface for all configuration changes. Do not use command line utilities such as `hawq config` to set or change HAWQ configuration properties for Ambari-managed clusters. Configuration changes to `hawq-site.xml` made outside the Ambari interface will be overwritten when you restart or reconfigure HAWQ using Ambari. - -If you manage your cluster using command line tools instead of Ambari, use a consistent `hawq-site.xml` file to configure your entire cluster. - -**Note:** While `postgresql.conf` still exists in HAWQ, any parameters defined in `hawq-site.xml` will overwrite configurations in `postgresql.conf`. For this reason, we recommend that you only use `hawq-site.xml` to configure your HAWQ cluster. For Ambari clusters, always use Ambari for configuring `hawq-site.xml` parameters. - -## <a id="task_qgk_bz3_1v"></a>Best Practices to Start/Stop HAWQ Cluster Members - -For best results in using `hawq start` and `hawq stop` to manage your HAWQ system, the following best practices are recommended. - -- Issue the `CHECKPOINT` command to update and flush all data files to disk and update the log file before stopping the cluster. A checkpoint ensures that, in the event of a crash, files can be restored from the checkpoint snapshot. +- Always use HAWQ management commands to start and stop HAWQ, instead of Postgres equivalents. +- Use `hawq start cluster` or `hawq restart cluster` to start the entire cluster, rather than starting the master and individual segments. +- If you do decide to use `hawq start standby|master|segment` to start nodes individually, always start the standby before the active master. Otherwise, the standby can become unsynchronized with the active master. +- When stopping a cluster, issue the `CHECKPOINT` command to update and flush all data files to disk and update the log file before stopping the cluster. A checkpoint ensures that, in the event of a crash, files can be restored from the checkpoint snapshot. - Stop the entire HAWQ system by stopping the cluster on the master host: ```shell $ hawq stop cluster @@ -70,8 +63,6 @@ For best results in using `hawq start` and `hawq stop` to manage your HAWQ syste $ hawq stop allsegments ``` -- Typically you should always use `hawq start cluster` or `hawq restart cluster` to start the cluster. If you do end up using `hawq start standby|master|segment` to start nodes individually, make sure you always start the standby before the active master. Otherwise, the standby can become unsynchronized with the active master. - ## <a id="id_trr_m1j_1v"></a>Guidelines for Cluster Expansion This topic provides some guidelines around expanding your HAWQ cluster. @@ -84,7 +75,12 @@ There are several recommendations to keep in mind when modifying the size of you - Note that for hash distributed tables, expanding the cluster will not immediately improve performance since hash distributed tables use a fixed number of virtual segments. In order to obtain better performance with hash distributed tables, you must redistribute the table to the updated cluster by either the [ALTER TABLE](../reference/sql/ALTER-TABLE.html) or [CREATE TABLE AS](../reference/sql/CREATE-TABLE-AS.html#topic1) command. - If you are using hash tables, consider updating the `default_hash_table_bucket_number` server configuration parameter to a larger value after expanding the cluster but before redistributing the hash tables. -## <a id="id_o5n_p1j_1v"></a>Database State Monitoring Activities +## <a id="best_pract_monitoring"></a>Best Practices for Monitoring a HAWQ System +The sections below provide recommendations for common monitoring tasks. + +**Note:** If your HAWQ cluster is managed through the Ambari interface, use Ambari to perform the suggested monitoring tasks. + +### <a id="id_o5n_p1j_1v"></a>Database State Monitoring Activities <a id="id_o5n_p1j_1v__d112e31"></a> @@ -120,7 +116,7 @@ WHERE status <> 'u';</code></pre></td> </table> -## <a id="id_d3w_p1j_1v"></a>Hardware and Operating System Monitoring +### <a id="id_d3w_p1j_1v"></a>Hardware and Operating System Monitoring <a id="id_d3w_p1j_1v__d112e111"></a> @@ -195,7 +191,7 @@ If transfer rates are lower than expected, consult with your data architect rega </table> -## <a id="id_khd_q1j_1v"></a>Data Maintenance +### <a id="id_khd_q1j_1v"></a>Data Maintenance <a id="id_khd_q1j_1v__d112e279"></a> @@ -224,7 +220,7 @@ If transfer rates are lower than expected, consult with your data architect rega </table> -## <a id="id_lx4_q1j_1v"></a>Database Maintenance +### <a id="id_lx4_q1j_1v"></a>Database Maintenance <a id="id_lx4_q1j_1v__d112e343"></a> @@ -277,7 +273,7 @@ If transfer rates are lower than expected, consult with your data architect rega </table> -## <a id="id_blv_q1j_1v"></a>Patching and Upgrading +### <a id="id_blv_q1j_1v"></a>Patching and Upgrading <a id="id_blv_q1j_1v__d112e472"></a>
