IMPALA-3401 [DOCS] Part2 Removing 'Cloudera Manager' from upstream docs. Used DITA attribute 'audience=hidden' when needed to preserve information for a future clean-up step. When it made sense, 'Cloudera Manager' was removed and sections were rewritten. This clean-up task does not completely remove 'Cloudera Manager' from the upstream docs source XML, but it is now hidden in these files so it won't show up in the rendered version. A subsequent cleanup project will remove references in the XML source.
Change-Id: I76c9b53f587bc85c5c21e195f0a771183d4ef3a0 Reviewed-on: http://gerrit.cloudera.org:8080/6064 Reviewed-by: John Russell <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/685cdc55 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/685cdc55 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/685cdc55 Branch: refs/heads/master Commit: 685cdc55ee7d4b0b8ae64331c666bca591b8438e Parents: ae79288 Author: Laurel Hale <[email protected]> Authored: Fri Feb 17 14:21:25 2017 -0800 Committer: Impala Public Jenkins <[email protected]> Committed: Tue Mar 7 21:14:19 2017 +0000 ---------------------------------------------------------------------- docs/topics/impala_admission.xml | 127 +++++++++++++++---------- docs/topics/impala_config_performance.xml | 29 ++++-- docs/topics/impala_noncm_installation.xml | 10 ++ docs/topics/impala_prereqs.xml | 14 ++- 4 files changed, 119 insertions(+), 61 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/685cdc55/docs/topics/impala_admission.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_admission.xml b/docs/topics/impala_admission.xml index cceae93..4f5388b 100644 --- a/docs/topics/impala_admission.xml +++ b/docs/topics/impala_admission.xml @@ -187,8 +187,7 @@ under the License. <conbody> <p> - The admission control feature is similar in some ways to the Cloudera Manager - static partitioning feature, as well as the YARN resource management framework. These features + The admission control feature is similar in some ways to the YARN resource management framework. These features can be used separately or together. This section describes some similarities and differences, to help you decide which combination of resource management features to use for Impala. </p> @@ -214,8 +213,14 @@ under the License. The Impala admission control feature uses the same configuration mechanism as the YARN resource manager to map users to pools and authenticate them. </p> + + <p> + Although the Impala admission control feature uses a <codeph>fair-scheduler.xml</codeph> configuration file + behind the scenes, this file does not depend on which scheduler is used for YARN. You still use this file + even when YARN is using the capacity scheduler. + </p> - <p rev="DOCS-648"> + <p rev="DOCS-648" audience="hidden"> Although the Impala admission control feature uses a <codeph>fair-scheduler.xml</codeph> configuration file behind the scenes, this file does not depend on which scheduler is used for YARN. You still use this file, and Cloudera Manager can generate it for you, even when YARN is using the capacity scheduler. @@ -393,6 +398,13 @@ under the License. <p> The configuration options for admission control range from the simple (a single resource pool with a single set of options) to the complex (multiple resource pools with different options, each pool handling queries + for a different set of users and groups). + </p> + + + <p audience="hidden"> + The configuration options for admission control range from the simple (a single resource pool with a single + set of options) to the complex (multiple resource pools with different options, each pool handling queries for a different set of users and groups). <ph rev="upstream">Cloudera</ph> recommends configuring the settings through the Cloudera Manager user interface. <!-- @@ -403,7 +415,7 @@ under the License. <!-- To do: reconcile the similar notes in impala_admission.xml and admin_impala_admission_control.xml and make into a conref in both places. --> - <note type="important"> + <note type="important" audience="hidden"> Although the following options are still present in the Cloudera Manager interface under the <uicontrol>Admission Control</uicontrol> configuration settings dialog, <ph rev="upstream">Cloudera</ph> recommends you not use them in <keyword keyref="impala25_full"/> and higher. @@ -613,9 +625,9 @@ under the License. <conbody> <p> - If you do not use Cloudera Manager, you use a combination of startup options for the Impala daemon, and - optionally editing or manually constructing the configuration files - <filepath>fair-scheduler.xml</filepath> and <filepath>llama-site.xml</filepath>. + To configure admission control, use a combination of startup options for the Impala daemon and edit + or create the configuration files <filepath>fair-scheduler.xml</filepath> and + <filepath>llama-site.xml</filepath>. </p> <p> @@ -674,45 +686,41 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph <concept id="admission_examples"> <!-- Pruning the CM examples and screenshots because in Impala 2.5 the defaults match up much better with our recommendations. --> - <title>Examples of Admission Control Configurations</title> + <title>Example of Admission Control Configuration</title> <conbody> + + <p> Here are sample <filepath>fair-scheduler.xml</filepath> and + <filepath>llama-site.xml</filepath> files that define resource pools + <codeph>root.default</codeph>, <codeph>root.development</codeph>, and + <codeph>root.production</codeph>. These sample files are stripped down: in a real + deployment they might contain other settings for use with various aspects of the YARN + component. The settings shown here are the significant ones for the Impala admission + control feature. </p> + + <p> + <b>fair-scheduler.xml:</b> + </p> + + <p> + Although Impala does not use the <codeph>vcores</codeph> value, you must still specify it to satisfy + YARN requirements for the file contents. + </p> + + <p> + Each <codeph><aclSubmitApps></codeph> tag (other than the one for <codeph>root</codeph>) contains + a comma-separated list of users, then a space, then a comma-separated list of groups; these are the + users and groups allowed to submit Impala statements to the corresponding resource pool. + </p> + + <p> + If you leave the <codeph><aclSubmitApps></codeph> element empty for a pool, nobody can submit + directly to that pool; child pools can specify their own <codeph><aclSubmitApps></codeph> values + to authorize users and groups to submit to those pools. + </p> + + <codeblock><![CDATA[<allocations> - <section id="section_etq_qgb_rq"> - - <title>Example Admission Control Configurations Using Configuration Files</title> - - <p> - For clusters not managed by Cloudera Manager, here are sample <filepath>fair-scheduler.xml</filepath> - and <filepath>llama-site.xml</filepath> files that define resource pools <codeph>root.default</codeph>, - <codeph>root.development</codeph>, and <codeph>root.production</codeph>. - These sample files are stripped down: in a real deployment they - might contain other settings for use with various aspects of the YARN component. The - settings shown here are the significant ones for the Impala admission control feature. - </p> - - <p> - <b>fair-scheduler.xml:</b> - </p> - - <p> - Although Impala does not use the <codeph>vcores</codeph> value, you must still specify it to satisfy - YARN requirements for the file contents. - </p> - - <p> - Each <codeph><aclSubmitApps></codeph> tag (other than the one for <codeph>root</codeph>) contains - a comma-separated list of users, then a space, then a comma-separated list of groups; these are the - users and groups allowed to submit Impala statements to the corresponding resource pool. - </p> - - <p> - If you leave the <codeph><aclSubmitApps></codeph> element empty for a pool, nobody can submit - directly to that pool; child pools can specify their own <codeph><aclSubmitApps></codeph> values - to authorize users and groups to submit to those pools. - </p> - -<codeblock><![CDATA[<allocations> <queue name="root"> <aclSubmitApps> </aclSubmitApps> <queue name="default"> @@ -735,12 +743,12 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph </allocations> ]]> </codeblock> - - <p> - <b>llama-site.xml:</b> - </p> - -<codeblock rev="2.5.0 IMPALA-2538"><![CDATA[ + + <p> + <b>llama-site.xml:</b> + </p> + + <codeblock rev="2.5.0 IMPALA-2538"><![CDATA[ <?xml version="1.0" encoding="UTF-8"?> <configuration> <property> @@ -813,10 +821,24 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph </configuration> ]]> </codeblock> + + + + + + <section id="section_fqn_qgb_rq" audience="hidden"> + + <title>Example Admission Control Configurations Using Cloudera Manager</title> + + <p> + For full instructions about configuring dynamic resource pools through Cloudera Manager, see + <xref audience="integrated" href="cm_mc_resource_pools.xml#xd_583c10bfdbd326ba--43d5fd93-1410993f8c2--7ff2"/><xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_resource_pools.html"; scope="external" format="html"/>. + </p> + </section> + </conbody> </concept> - </concept> <!-- End Config --> @@ -843,7 +865,7 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph usage for the query, so you can fine-tune the configuration for the memory limits of the resource pools. </p> - <p> + <p audience="hidden"> Where practical, use Cloudera Manager to configure the admission control parameters. The Cloudera Manager GUI is much simpler than editing the configuration files directly. </p> @@ -921,4 +943,5 @@ drop table huge_table; </conbody> </concept> </concept> -<!-- Admission control --> +</concept> + http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/685cdc55/docs/topics/impala_config_performance.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_config_performance.xml b/docs/topics/impala_config_performance.xml index d14407e..ce9718b 100644 --- a/docs/topics/impala_config_performance.xml +++ b/docs/topics/impala_config_performance.xml @@ -34,12 +34,18 @@ under the License. <p id="p_24"> This section describes the mandatory and recommended configuration settings for Impala. If Impala is + installed using cluster management software, some of these configurations might be completed automatically; you must still + configure short-circuit reads manually. If you want to customize your environment, consider making the changes described in this topic. + </p> + + <p id="p_twenty-four" audience="hidden"> + This section describes the mandatory and recommended configuration settings for Impala. If Impala is installed using Cloudera Manager, some of these configurations are completed automatically; you must still configure short-circuit reads manually. If you installed Impala without Cloudera Manager, or if you want to customize your environment, consider making the changes described in this topic. </p> - <p> + <p audience="hidden"> <!-- Could conref this paragraph from ciiu_install.xml. --> In some cases, depending on the level of Impala, CDH, and Cloudera Manager, you might need to add particular component configuration details in one of the free-form fields on the Impala configuration pages within @@ -48,16 +54,21 @@ under the License. <ul> <li> - You must enable short-circuit reads, whether or not Impala was installed through Cloudera Manager. This - setting goes in the Impala configuration settings, not the Hadoop-wide settings. + You must enable short-circuit reads, whether or not Impala was installed with cluster + management software. This setting goes in the Impala configuration settings, not the Hadoop-wide settings. </li> - + <li> + You must enable block location tracking, and you can optionally enable native checksumming for optimal performance. + </li> + + + <li audience="hidden"> If you installed Impala in an environment that is not managed by Cloudera Manager, you must enable block location tracking, and you can optionally enable native checksumming for optimal performance. </li> - <li> + <li audience="hidden"> If you deployed Impala using Cloudera Manager see <xref href="impala_perf_testing.xml#performance_testing"/> to confirm proper configuration. </li> @@ -76,9 +87,13 @@ under the License. client. <codeph>libhadoop.so</codeph> is not available if you have installed from a tarball. You must install from an <codeph>.rpm</codeph>, <codeph>.deb</codeph>, or parcel to use - short-circuit local reads. <note> If you use Cloudera Manager, you can + short-circuit local reads. + + <note audience="hidden"> + If you use Cloudera Manager, you can enable short-circuit reads through a checkbox in the user interface - and that setting takes effect for Impala as well. </note> + and that setting takes effect for Impala as well. + </note> </p> <p> <b>To configure DataNodes for short-circuit reads:</b> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/685cdc55/docs/topics/impala_noncm_installation.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_noncm_installation.xml b/docs/topics/impala_noncm_installation.xml index ca2ba06..b00cbfb 100644 --- a/docs/topics/impala_noncm_installation.xml +++ b/docs/topics/impala_noncm_installation.xml @@ -171,8 +171,18 @@ $ sudo apt-get install impala-catalog # Service start/stop script <li> Complete any required or recommended configuration, as described in <xref href="impala_config_performance.xml#config_performance"/>. Some of these configuration changes are + mandatory. + </li> + + <li audience="hidden"> + Complete any required or recommended configuration, as described in + <xref href="impala_config_performance.xml#config_performance"/>. Some of these configuration changes are mandatory. (They are applied automatically when you install using Cloudera Manager.) </li> + + + + </ol> <p> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/685cdc55/docs/topics/impala_prereqs.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_prereqs.xml b/docs/topics/impala_prereqs.xml index 47e54fa..2ba5e13 100644 --- a/docs/topics/impala_prereqs.xml +++ b/docs/topics/impala_prereqs.xml @@ -52,7 +52,7 @@ under the License. <p outputclass="toc inpage"/> </conbody> - <concept id="product_compatibility_matrix"> + <concept id="product_compatibility_matrix" audience="hidden"> <title>Product Compatibility Matrix</title> @@ -130,13 +130,23 @@ under the License. the metastore database. For the process of installing and configuring the metastore, see <xref href="impala_install.xml#install"/>. </p> - <p> + <p audience="hidden"> Always configure a <b>Hive metastore service</b> rather than connecting directly to the metastore database. The Hive metastore service is required to interoperate between possibly different levels of metastore APIs used by CDH and Impala, and avoids known issues with connecting directly to the metastore database. The Hive metastore service is set up for you by default if you install through Cloudera Manager 4.5 or higher. </p> + + <p> + Always configure a <b>Hive metastore service</b> rather than connecting directly to the metastore + database. The Hive metastore service is required to interoperate between different levels of + metastore APIs if this is necessary for your environment, and using it avoids known issues with + connecting directly to the metastore database. + </p> + + + <p> A summary of the metastore installation process is as follows: </p>
