IMPALA-3401 [DOCS] Part 4 of "Cloudera Manager" removal. Most of these fixes involved hiding the paragraphs with the DITA attribute 'audience="hidden"' and then inserting a paragraph suitable for upstream documentation. This hides the mention of Cloudera Manager in the rendered documentation. In a subsequent cleanup project, the "Cloudera Manager" mentions will be removed from the XML.
Change-Id: I3c3c2177e0b9c4c81f1541820013c66a59c0c7b1 Reviewed-on: http://gerrit.cloudera.org:8080/6069 Reviewed-by: John Russell <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/64c7f5b8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/64c7f5b8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/64c7f5b8 Branch: refs/heads/master Commit: 64c7f5b8b55720f355dfbd548f5dd32bf5352da6 Parents: fdcea43 Author: Laurel Hale <[email protected]> Authored: Fri Feb 17 20:15:46 2017 -0800 Committer: Impala Public Jenkins <[email protected]> Committed: Tue Mar 7 23:16:40 2017 +0000 ---------------------------------------------------------------------- docs/shared/impala_common.xml | 31 ++++++++++++++++++------------ docs/topics/impala_perf_resources.xml | 9 ++++++++- docs/topics/impala_perf_skew.xml | 10 +++++++++- docs/topics/impala_perf_testing.xml | 7 +++++++ docs/topics/impala_scalability.xml | 10 +++++++++- docs/topics/impala_txtfile.xml | 24 +++++++++++++++++++++++ 6 files changed, 76 insertions(+), 15 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/64c7f5b8/docs/shared/impala_common.xml ---------------------------------------------------------------------- diff --git a/docs/shared/impala_common.xml b/docs/shared/impala_common.xml index 61b3365..17c0026 100644 --- a/docs/shared/impala_common.xml +++ b/docs/shared/impala_common.xml @@ -574,6 +574,14 @@ under the License. in the <cmdname>impalad</cmdname> and <cmdname>catalogd</cmdname> configuration settings. </p> </li> + <li audience="hidden"> + <p> + For clusters managed by Cloudera Manager, select the + <uicontrol>Use HDFS Rules to Map Kerberos Principals to Short Names</uicontrol> + checkbox to enable the service-wide <codeph>load_auth_to_local_rules</codeph> configuration setting. + Then restart the Impala service. + </p> + </li> </ul> </p> @@ -1455,15 +1463,14 @@ explain select s from yy2 where year in (select year from yy where year between operation finishes. (Multiple concurrent queries can perform operations that use the <q>spill to disk</q> technique, without any name conflicts for these temporary files.) You can specify a different location by starting the <cmdname>impalad</cmdname> daemon with the - <codeph>--scratch_dirs="<varname>path_to_directory</varname>"</codeph> configuration option or the - equivalent configuration option in the Cloudera Manager user interface. You can specify a single directory, - or a comma-separated list of directories. The scratch directories must be on the local filesystem, not in - HDFS. You might specify different directory paths for different hosts, depending on the capacity and speed - of the available storage devices. In CDH 5.5 / Impala 2.3 or higher, Impala successfully starts (with a warning - written to the log) if it cannot create or read and write files in one of the scratch directories. - If there is less than 1 GB free on the filesystem where that directory resides, Impala still runs, but writes a - warning message to its log. If Impala encounters an error reading or writing files in a scratch directory during - a query, Impala logs the error and the query fails. + <codeph>--scratch_dirs="<varname>path_to_directory</varname>"</codeph> configuration option. + You can specify a single directory, or a comma-separated list of directories. The scratch directories must + be on the local filesystem, not in HDFS. You might specify different directory paths for different hosts, + depending on the capacity and speed of the available storage devices. In CDH 5.5 / Impala 2.3 or higher, + Impala successfully starts (with a warning written to the log) if it cannot create or read and write files + in one of the scratch directories. If there is less than 1 GB free on the filesystem where that directory resides, + Impala still runs, but writes a warning message to its log. If Impala encounters an error reading or writing + files in a scratch directory during a query, Impala logs the error and the query fails. </p> <p id="order_by_view_restriction"> @@ -3114,7 +3121,7 @@ select * from header_line limit 10; Other security settings may prevent Impala from writing core dumps even when this option is enabled. </p> </li> - <li rev="CDH-34070" audience="Cloudera"> + <li rev="CDH-34070" audience="hidden"> <p> On systems managed by Cloudera Manager, the default location for core dumps is on a temporary filesystem, which can lead to out-of-space issues if the core dumps are large, frequent, or @@ -3177,7 +3184,7 @@ sudo pip-python install ssl</codeblock> <codeph>SYNC_DDL</codeph> query option. </note> - <p rev="1.2" id="cm48_upgrade" audience="Cloudera"><!-- conref'ed in 2 places under 'incompatible changes' --> + <p rev="1.2" id="cm48_upgrade" audience="hidden"><!-- conref'ed in 2 places under 'incompatible changes' --> In a Cloudera Manager environment, the catalog service is not recognized or managed by Cloudera Manager versions prior to 4.8. Cloudera Manager 4.8 and higher require the catalog service to be present for Impala. Therefore, if you upgrade to Cloudera Manager 4.8 or higher, you must also upgrade Impala to 1.2.1 @@ -3644,7 +3651,7 @@ sudo pip-python install ssl</codeblock> JAVA_TOOL_OPTIONS="-Xmx8g" </codeblock> </li> - <li audience="Cloudera"> + <li audience="hidden"> <p rev="OPSAPS-26483"> On systems managed by Cloudera Manager, include this value in the configuration field <uicontrol>Java Heap Size of Catalog Server in Bytes</uicontrol> (Cloudera Manager 5.7 and higher), or http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/64c7f5b8/docs/topics/impala_perf_resources.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_perf_resources.xml b/docs/topics/impala_perf_resources.xml index 3c1f8f3..907f36c 100644 --- a/docs/topics/impala_perf_resources.xml +++ b/docs/topics/impala_perf_resources.xml @@ -64,9 +64,16 @@ under the License. hold cached metadata. </p> </li> - + <li> <p> + For production deployments, implement resource isolation using your cluster management + tool. + </p> + </li> + + <li audience="hidden"> + <p> For production deployment, <ph rev="upstream">Cloudera</ph> recommends that you implement resource isolation using mechanisms such as cgroups, which you can configure using Cloudera Manager. For details, see the <xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_service_pools.html"; scope="external" format="html">Static http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/64c7f5b8/docs/topics/impala_perf_skew.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_perf_skew.xml b/docs/topics/impala_perf_skew.xml index cfcc84a..277ee9e 100644 --- a/docs/topics/impala_perf_skew.xml +++ b/docs/topics/impala_perf_skew.xml @@ -67,9 +67,17 @@ under the License. by a detailed performance analysis. </p> </li> - + <li> <p> + In the Impala debug web UI, click on the <uicontrol>Profile</uicontrol> link associated with the query after it is + complete. The executive summary information is displayed early in the profile output. + </p> + </li> + + + <li audience="hidden"> + <p> In the Cloudera Manager interface or the Impala debug web UI, click on the <uicontrol>Profile</uicontrol> link associated with the query after it is complete. The executive summary information is displayed early in the profile output. http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/64c7f5b8/docs/topics/impala_perf_testing.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_perf_testing.xml b/docs/topics/impala_perf_testing.xml index 27e2f31..858095f 100644 --- a/docs/topics/impala_perf_testing.xml +++ b/docs/topics/impala_perf_testing.xml @@ -39,6 +39,13 @@ under the License. <conbody> <p> + Test to ensure that Impala is configured for optimal performance. If you have installed Impala with cluster + management software, complete the processes described in this topic to help ensure a proper + configuration. These procedures can be used to verify that Impala is set up correctly. + </p> + + + <p audience="hidden"> Test to ensure that Impala is configured for optimal performance. If you have installed Impala without Cloudera Manager, complete the processes described in this topic to help ensure a proper configuration. Even if you installed Impala with Cloudera Manager, which automatically applies appropriate configurations, these http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/64c7f5b8/docs/topics/impala_scalability.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_scalability.xml b/docs/topics/impala_scalability.xml index 9bc8b7c..f3a7d4c 100644 --- a/docs/topics/impala_scalability.xml +++ b/docs/topics/impala_scalability.xml @@ -294,7 +294,7 @@ Memory Usage: Additional Notes </dlentry> </dl> - <p> + <p audience="hidden"> As of CDH 5.3, not all of these flags are present in the Cloudera Manager user interface. Some must be set using the <uicontrol>Advanced Configuration Snippet</uicontrol> fields for the statestore component. </p> @@ -795,7 +795,15 @@ While these authentication requests are being processed, any submitted Impala qu During this period, the KDC and DNS may be slow to respond to requests from components other than Impala, so other secure services might be affected temporarily. </p> + <p> + To reduce the frequency of the <codeph>kinit</codeph> renewal that initiates + a new set of authentication requests, increase the <codeph>kerberos_reinit_interval</codeph> + configuration setting for the <cmdname>impalad</cmdname> daemons. Currently, the default is 60 minutes. + Consider using a higher value such as 360 (6 hours). +</p> + +<p audience="hidden"> To reduce the frequency of the <codeph>kinit</codeph> renewal that initiates a new set of authentication requests, increase the <codeph>kerberos_reinit_interval</codeph> configuration setting for the <cmdname>impalad</cmdname> daemons. Currently, the default for a cluster not managed by http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/64c7f5b8/docs/topics/impala_txtfile.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_txtfile.xml b/docs/topics/impala_txtfile.xml index 4b29bf0..ea1ada8 100644 --- a/docs/topics/impala_txtfile.xml +++ b/docs/topics/impala_txtfile.xml @@ -475,6 +475,14 @@ INSERT INTO csv SELECT * FROM other_file_format_table;</codeblock> <p> Before using LZO-compressed tables in Impala, do the following one-time setup for each machine in the + cluster. Install the necessary packages using either the public repository, a private repository + you establish, or by using packages. You must do these steps manually, whether or not you + are using cluster management software. + </p> + + + <p audience="hidden"> + Before using LZO-compressed tables in Impala, do the following one-time setup for each machine in the cluster. Install the necessary packages using either the Cloudera public repository, a private repository you establish, or by using packages. You must do these steps manually, whether or not the cluster is managed by the Cloudera Manager product. @@ -484,7 +492,23 @@ INSERT INTO csv SELECT * FROM other_file_format_table;</codeblock> <li> <b>Prepare your systems to work with LZO by downloading and installing the appropriate libraries:</b> + <p audience="hidden"> + <b>On systems managed by Cloudera Manager using parcels:</b> + </p> + + <p audience="hidden"> + See the setup instructions for the LZO parcel in the Cloudera Manager documentation for + <xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_gpl_extras.html"; scope="external" format="html">Cloudera + Manager 5</xref>. + </p> + + <p> + <b>On systems using cluster management software or those not using cluster management + software:</b> + </p> + + <p audience="hidden"> <b>On systems managed by Cloudera Manager using packages, or not managed by Cloudera Manager:</b> </p>
