IMPALA-3401 [DOCS] Phase 3 of removing Cloudera Manager from upstream docs.
Hid instances of CM and rewrote for upstream docs when necessary. This still leaves occurences of CM in the XML, but not in the rendered documentation. A later project will remove all occurrences of CM from the XML. Change-Id: I4748300edc43b7071afc50e7cc7ddd64120c0d8d Reviewed-on: http://gerrit.cloudera.org:8080/6067 Reviewed-by: John Russell <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/fdcea430 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/fdcea430 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/fdcea430 Branch: refs/heads/master Commit: fdcea4304fbd06bdfef3ee5cc1d674a132a9160c Parents: 6f83711 Author: Laurel Hale <[email protected]> Authored: Fri Feb 17 16:54:22 2017 -0800 Committer: Impala Public Jenkins <[email protected]> Committed: Tue Mar 7 22:39:19 2017 +0000 ---------------------------------------------------------------------- docs/topics/impala_impala_shell.xml | 9 ++- docs/topics/impala_proxy.xml | 87 +++++++++++++++++++++++++ docs/topics/impala_resource_management.xml | 10 ++- docs/topics/impala_timeouts.xml | 12 +++- docs/topics/impala_udf.xml | 10 ++- 5 files changed, 124 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/fdcea430/docs/topics/impala_impala_shell.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_impala_shell.xml b/docs/topics/impala_impala_shell.xml index 53531a3..4780728 100644 --- a/docs/topics/impala_impala_shell.xml +++ b/docs/topics/impala_impala_shell.xml @@ -93,8 +93,15 @@ under the License. and edit previous commands. </li> </ul> - + <p> + + For information on installing the Impala shell, see <xref href="impala_install.xml#install"/>. + + </p> + + + <p audience="hidden"> For information on installing the Impala shell, see <xref href="impala_install.xml#install"/>. In Cloudera Manager 4.1 and higher, Cloudera Manager installs <cmdname>impala-shell</cmdname> automatically. You might install <cmdname>impala-shell</cmdname> manually on other systems not managed by Cloudera Manager, so that http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/fdcea430/docs/topics/impala_proxy.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_proxy.xml b/docs/topics/impala_proxy.xml index bb8d029..3c4aa10 100644 --- a/docs/topics/impala_proxy.xml +++ b/docs/topics/impala_proxy.xml @@ -260,7 +260,94 @@ under the License. running the <cmdname>impalad</cmdname> daemon. </li> + + <li rev="CDH-40363" audience="hidden"> + For a cluster managed by Cloudera Manager (5.4.2 or higher), fill in the Impala configuration setting + <uicontrol>Impala Daemons Load Balancer</uicontrol> with the appropriate host:port combination. + Then restart the Impala service. + For systems using a recent level of Cloudera Manager, this is all the configuration you need; you can skip the remaining steps in this procedure. + </li> + + <li> + Copy the keytab file from the proxy host to all other hosts in the cluster that run the + <cmdname>impalad</cmdname> daemon. (For optimal performance, <cmdname>impalad</cmdname> should be running + on all DataNodes in the cluster.) Put the keytab file in a secure location on each of these other hosts. + </li> + <li> + Add an entry <codeph>impala/<varname>actual_hostname</varname>@<varname>realm</varname></codeph> to the keytab on each + host running the <cmdname>impalad</cmdname> daemon. + </li> + + <li> + + For each impalad node, merge the existing keytab with the proxyâs keytab using + <cmdname>ktutil</cmdname>, producing a new keytab file. For example: + <codeblock>$ ktutil + ktutil: read_kt proxy.keytab + ktutil: read_kt impala.keytab + ktutil: write_kt proxy_impala.keytab + ktutil: quit</codeblock> + + </li> + + <li> + + To verify that the keytabs are merged, run the command: +<codeblock> +klist -k <varname>keytabfile</varname> +</codeblock> + which lists the credentials for both <codeph>principal</codeph> and <codeph>be_principal</codeph> on + all nodes. + </li> + + + <li> + + Make sure that the <codeph>impala</codeph> user has permission to read this merged keytab file. + + </li> + + <li> + Change the following configuration settings for each host in the cluster that participates + in the load balancing: + <ul> + <li> + In the <cmdname>impalad</cmdname> option definition, add: +<codeblock> +--principal=impala/<i>proxy_host@realm</i> + --be_principal=impala/<i>actual_host@realm</i> + --keytab_file=<i>path_to_merged_keytab</i> +</codeblock> + <note> + Every host has different <codeph>--be_principal</codeph> because the actual hostname + is different on each host. + + Specify the fully qualified domain name (FQDN) for the proxy host, not the IP + address. Use the exact FQDN as returned by a reverse DNS lookup for the associated + IP address. + + </note> + </li> + + <li> + Modify the startup options. See <xref href="impala_config_options.xml#config_options"/> for the procedure to modify the startup + options. + </li> + </ul> + </li> + + + <li> + + Restart Impala to make the changes take effect. Restart the <cmdname>impalad</cmdname> daemons on all + hosts in the cluster, as well as the <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname> + daemons. + + </li> + + + <li audience="hidden"> On systems not managed by Cloudera Manager, or systems using Cloudera Manager earlier than 5.4.2: <ol> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/fdcea430/docs/topics/impala_resource_management.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_resource_management.xml b/docs/topics/impala_resource_management.xml index 80dbbc4..ade204f 100644 --- a/docs/topics/impala_resource_management.xml +++ b/docs/topics/impala_resource_management.xml @@ -55,7 +55,15 @@ under the License. <conbody> - <ul> + <p> + Limits on memory usage are enforced by Impala's process memory limit (the <codeph>MEM_LIMIT</codeph> + query option setting). The admission control feature checks this setting to decide how many queries + can be safely run at the same time. Then the Impala daemon enforces the limit by activating the + spill-to-disk mechanism when necessary, or cancelling a query altogether if the limit is exceeded at runtime. + </p> + + + <ul audience="hidden"> <li> If Cloudera Manager Static Partitioning is used, it creates a cgroup in which Impala runs. This cgroup limits CPU, network, and IO according to the static partitioning policy. http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/fdcea430/docs/topics/impala_timeouts.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_timeouts.xml b/docs/topics/impala_timeouts.xml index df7bc1e..0857f11 100644 --- a/docs/topics/impala_timeouts.xml +++ b/docs/topics/impala_timeouts.xml @@ -169,12 +169,22 @@ Trying to re-register with state-store</codeblock> <title>Cancelling a Query</title> <conbody> - + <p> Sometimes, an Impala query might run for an unexpectedly long time, tying up resources in the cluster. You can cancel the query explicitly, independent of the timeout period, by going into the web UI for the <cmdname>impalad</cmdname> host (on port 25000 by default), and using the link on the <codeph>/queries</codeph> tab to cancel the running + query. For example, press <codeph>^C</codeph> in <cmdname>impala-shell</cmdname>. + </p> + + + + <p audience="hidden"> + Sometimes, an Impala query might run for an unexpectedly long time, tying up resources + in the cluster. You can cancel the query explicitly, independent of the timeout period, + by going into the web UI for the <cmdname>impalad</cmdname> host (on port 25000 by + default), and using the link on the <codeph>/queries</codeph> tab to cancel the running query. Various client applications let you interactively cancel queries submitted or monitored through those applications. For example, by pressing <codeph>^C</codeph> in <cmdname>impala-shell</cmdname>, clicking the <uicontrol>Cancel</uicontrol> button from http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/fdcea430/docs/topics/impala_udf.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_udf.xml b/docs/topics/impala_udf.xml index 1f63ec1..b55739c 100644 --- a/docs/topics/impala_udf.xml +++ b/docs/topics/impala_udf.xml @@ -437,8 +437,16 @@ and other examples demonstrating this technique in (RHEL-based distributions) or <codeph>impala-udf-dev</codeph> (Ubuntu and Debian). </li> </ol> - + + <note> + + The UDF development code does not rely on Impala being installed on the same machine. You can write and + compile UDFs on a minimal development system, then deploy them on a different one for use with Impala. + + </note> + + <note audience="hidden"> The UDF development code does not rely on Impala being installed on the same machine. You can write and compile UDFs on a minimal development system, then deploy them on a different one for use with Impala. If you develop UDFs on a server managed by Cloudera Manager through the parcel mechanism, you still install
