Repository: incubator-impala Updated Branches: refs/heads/master 685cdc55e -> ec469347f
IMPALA-3401 [DOCS] Phase 6 of "Cloudera Manager" removal Removed references to "Cloudera Manager" mostly by hiding the paragraphs that contained the references and rewriting them for the upstreams docs without referencing CM. This removes the references in the rendered docs, but does not remove them from the source XML. That part of the removal will be done in a subsequent work item. Change-Id: I3a5ae82ee5adfd1e2f250bc4dc26e45047dc434c Reviewed-on: http://gerrit.cloudera.org:8080/6083 Reviewed-by: John Russell <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/6f837118 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/6f837118 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/6f837118 Branch: refs/heads/master Commit: 6f837118b864f6a3d679f13c5b17fd43ff7a6872 Parents: 685cdc5 Author: Laurel Hale <[email protected]> Authored: Mon Feb 20 11:00:12 2017 -0800 Committer: Impala Public Jenkins <[email protected]> Committed: Tue Mar 7 22:25:25 2017 +0000 ---------------------------------------------------------------------- docs/topics/impala_breakpad.xml | 22 +++++++++++++- docs/topics/impala_faq.xml | 5 +++- docs/topics/impala_fixed_issues.xml | 25 ++++++++++++++-- docs/topics/impala_incompatible_changes.xml | 25 ++++++++++++++-- docs/topics/impala_new_features.xml | 38 +++++++++++++++++++++--- 5 files changed, 105 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/6f837118/docs/topics/impala_breakpad.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_breakpad.xml b/docs/topics/impala_breakpad.xml index 5d755fd..d171ef7 100644 --- a/docs/topics/impala_breakpad.xml +++ b/docs/topics/impala_breakpad.xml @@ -134,7 +134,15 @@ under the License. <concept id="breakpad_minidump_logging"> <title>Detecting Crash Events</title> <conbody> + <p> + You can see in the Impala log files when crash events occur that generate + minidump files. Because each restart begins a new log file, the <q>crashed</q> message + is always at or near the bottom of the log file. There might be another later message + if core dumps are also enabled. + </p> + + <p audience="hidden"> You can see in the Impala log files or in the Cloudera Manager charts for Impala when crash events occur that generate minidump files. Because each restart begins a new log file, the <q>crashed</q> message is always at or near the bottom of the @@ -143,7 +151,7 @@ under the License. </conbody> </concept> - <concept id="breakpad_support_process" rev="CDH-39818"> + <concept id="breakpad_support_process" rev="CDH-39818" audience="hidden"> <title>Using the Minidump Files for Problem Resolution</title> <conbody> <p> @@ -174,7 +182,13 @@ under the License. simulate a <codeph>SIGSEGV</codeph> crash for an <cmdname>impalad</cmdname> process on a single DataNode, then examines the relevant log files and minidump file. </p> + <p> + First, as root on a worker node, kill the <cmdname>impalad</cmdname> process with a + <codeph>SIGSEGV</codeph> error. The original process ID was 23114. + </p> + + <p audience="hidden"> First, as root on a worker node, we kill the <cmdname>impalad</cmdname> process with a <codeph>SIGSEGV</codeph> error. The original process ID was 23114. (Cloudera Manager restarts the process with a new pid, as shown by the second <cmdname>ps</cmdname> command.) @@ -248,8 +262,14 @@ E0623 14:03:43.911002 23114 logging.cc:118] stderr will be logged to this file. Wrote minidump to /var/log/impala-minidumps/impalad/0980da2d-a905-01e1-25ff883a-04ee027a.dmp ]]> </codeblock> + <p> The resulting minidump file is much smaller than the corresponding core file, + making it much easier to supply diagnostic information to <keyword keyref="support_org"/>. + </p> + + <p audience="hidden"> + The resulting minidump file is much smaller than the corresponding core file, making it much easier to supply diagnostic information to <keyword keyref="support_org"/>. The transmission process for the minidump files is automated through Cloudera Manager. </p> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/6f837118/docs/topics/impala_faq.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_faq.xml b/docs/topics/impala_faq.xml index bb7b098..4dd5d4f 100644 --- a/docs/topics/impala_faq.xml +++ b/docs/topics/impala_faq.xml @@ -39,11 +39,14 @@ under the License. This section lists frequently asked questions for Apache Impala (incubating), the interactive SQL engine for Hadoop. </p> - + <p> This section is under construction. </p> + + </conbody> </concept> + http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/6f837118/docs/topics/impala_fixed_issues.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_fixed_issues.xml b/docs/topics/impala_fixed_issues.xml index a063ff3..ef56d4c 100644 --- a/docs/topics/impala_fixed_issues.xml +++ b/docs/topics/impala_fixed_issues.xml @@ -4122,7 +4122,7 @@ IMPALA-1120: Fetch column statistics using Hive 0.13 bulk API this one fix th <b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1019"; scope="external" format="html">IMPALA-1019</xref> </p> - + <p> <b>Workaround:</b> On CDH 5.0.x, upgrade to CDH 5.0.4 with Impala 1.3.2, where this issue is fixed. In Impala 1.3.0 or 1.3.1 on CDH 5.0.x, do not use HDFS caching for Impala data files in Impala internal or @@ -4130,6 +4130,17 @@ IMPALA-1120: Fetch column statistics using Hive 0.13 bulk API this one fix th components that take advantage of HDFS caching), set the query option <codeph>DISABLE_CACHED_READS=true</codeph>. To set that option for all Impala queries across all sessions, start <cmdname>impalad</cmdname> with the <codeph>-default_query_options</codeph> option and + include this setting in the option argument. + </p> + + + <p audience="hidden"> + <b>Workaround:</b> On CDH 5.0.x, upgrade to CDH 5.0.4 with Impala 1.3.2, where this issue is fixed. In + Impala 1.3.0 or 1.3.1 on CDH 5.0.x, do not use HDFS caching for Impala data files in Impala internal or + external tables. If some of these data files are cached (for example because they are used by other + components that take advantage of HDFS caching), set the query option + <codeph>DISABLE_CACHED_READS=true</codeph>. To set that option for all Impala queries across all + sessions, start <cmdname>impalad</cmdname> with the <codeph>-default_query_options</codeph> option and include this setting in the option argument, or on a cluster managed by Cloudera Manager, fill in this option setting on the <uicontrol>Impala Daemon</uicontrol> options page. </p> @@ -4371,7 +4382,7 @@ IMPALA-1120: Fetch column statistics using Hive 0.13 bulk API this one fix th <b>Bug:</b> <xref href="https://issues.cloudera.org/browse/IMPALA-1019"; scope="external" format="html">IMPALA-1019</xref> </p> - + <p> <b>Workaround:</b> Upgrade to CDH 5.0.4 with Impala 1.3.2, where this issue is fixed. In Impala 1.3.0 or 1.3.1, do not use HDFS caching for Impala data files in Impala internal or @@ -4379,6 +4390,16 @@ IMPALA-1120: Fetch column statistics using Hive 0.13 bulk API this one fix th components that take advantage of HDFS caching), set the query option <codeph>DISABLE_CACHED_READS=true</codeph>. To set that option for all Impala queries across all sessions, start <cmdname>impalad</cmdname> with the <codeph>-default_query_options</codeph> option and + include this setting in the option argument. + </p> + + <p audience="hidden"> + <b>Workaround:</b> On CDH 5.0.x, upgrade to CDH 5.0.4 with Impala 1.3.2, where this issue is fixed. In + Impala 1.3.0 or 1.3.1 on CDH 5.0.x, do not use HDFS caching for Impala data files in Impala internal or + external tables. If some of these data files are cached (for example because they are used by other + components that take advantage of HDFS caching), set the query option + <codeph>DISABLE_CACHED_READS=true</codeph>. To set that option for all Impala queries across all + sessions, start <cmdname>impalad</cmdname> with the <codeph>-default_query_options</codeph> option and include this setting in the option argument, or on a cluster managed by Cloudera Manager, fill in this option setting on the <uicontrol>Impala Daemon</uicontrol> options page. </p> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/6f837118/docs/topics/impala_incompatible_changes.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_incompatible_changes.xml b/docs/topics/impala_incompatible_changes.xml index c10555c..4d790c2 100644 --- a/docs/topics/impala_incompatible_changes.xml +++ b/docs/topics/impala_incompatible_changes.xml @@ -1116,7 +1116,7 @@ under the License. The HDFS property <codeph>dfs.client.file-block-storage-locations.timeout</codeph> was renamed to <codeph>dfs.client.file-block-storage-locations.timeout.millis</codeph>, to emphasize that the unit of measure is milliseconds, not seconds. Impala requires a timeout of at least 10 seconds, making the - minimum value for this setting 10000. On systems not managed by Cloudera Manager, you might need to + minimum value for this setting 10000. If you are not using cluster management software, you might need to edit the <filepath>hdfs-site.xml</filepath> file in the Impala configuration directory for the new name and minimum value. </p> @@ -1249,6 +1249,12 @@ select * from `cross`;</codeblock> </li> </ul> + <p audience="hidden"> + Impala 1.2.1 goes along with CDH 4.5 and Cloudera Manager 4.8. If you used the beta version Impala 1.2.0 + that came with the beta of CDH 5, Impala 1.2.1 includes all the features of Impala 1.2.0 except for + resource management, which relies on the YARN framework from CDH 5. + </p> + <p> The new <cmdname>catalogd</cmdname> service might require changes to any user-written scripts that stop, start, or restart Impala services, install or upgrade Impala packages, or issue <codeph>REFRESH</codeph> or @@ -1286,8 +1292,14 @@ select * from `cross`;</codeblock> <ul conref="../shared/impala_common.xml#common/catalogd_xrefs"> <li/> </ul> - + <p> + The new resource management feature interacts with both YARN and Llama services. <ph audience="PDF">See + <xref href="impala_resource_management.xml#resource_management"/> for usage information for Impala resource + management.</ph> + </p> + + <p audience="hidden"> The new resource management feature interacts with both YARN and Llama services, which are available in CDH 5. These services are set up for you automatically in a Cloudera Manager (CM) environment. For information about setting up the YARN and Llama services, see the instructions for @@ -1393,8 +1405,17 @@ ALTER TABLE <varname>table_name</varname> SET FILEFORMAT <codeph>impala-lzo-cdh4</codeph> to the latest level. See <xref href="impala_txtfile.xml#lzo"/> for details. </li> + + <li audience="hidden"> + Cloudera Manager 4.5.2 and higher only supports Impala 1.0 and higher, and vice versa. If you upgrade to + Impala 1.0 or higher managed by Cloudera Manager, you must also upgrade Cloudera Manager to version 4.5.2 + or higher. If you upgrade from an earlier version of Cloudera Manager, and were using Impala, you must + also upgrade Impala to version 1.0 or higher. The beta versions of Impala are no longer supported as of + the release of Impala 1.0. + </li> </ul> </conbody> </concept> + </concept> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/6f837118/docs/topics/impala_new_features.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_new_features.xml b/docs/topics/impala_new_features.xml index ed908b6..a403c27 100644 --- a/docs/topics/impala_new_features.xml +++ b/docs/topics/impala_new_features.xml @@ -1056,6 +1056,7 @@ under the License. now use an optimized code path. </p> </li> + <li> <p rev="IMPALA-3044 IMPALA-2538 IMPALA-1168 CDH-33289 CDH-34603"> Improvements to the memory reservation mechanism for the Impala @@ -1064,6 +1065,17 @@ under the License. resource pool instead of globally for the Impala instance. The default limit for concurrent queries (the <uicontrol>max requests</uicontrol> setting) is now unlimited instead of 200. + </p> + </li> + + <li audience="hidden"> + <p rev="IMPALA-3044 IMPALA-2538 IMPALA-1168 CDH-33289 CDH-34603"> + Improvements to the memory reservation mechanism for the Impala + admission control feature. You can specify more settings, such + as the timeout period and maximum aggregate memory used, for each + resource pool instead of globally for the Impala instance. The + default limit for concurrent queries (the <uicontrol>max requests</uicontrol> + setting) is now unlimited instead of 200. The Cloudera Manager user interface for admission control has been reworked, with the settings available under the <uicontrol>Dynamic Resource Pools</uicontrol> window. @@ -3504,6 +3516,13 @@ under the License. <ul> <li> Additional security feature: auditing. New startup options for <cmdname>impalad</cmdname> let you capture + information about Impala queries that succeed or are blocked due to insufficient privileges. For details, + see <xref href="impala_security.xml#security"/>. + </li> + + + <li audience="hidden"> + Additional security feature: auditing. New startup options for <cmdname>impalad</cmdname> let you capture information about Impala queries that succeed or are blocked due to insufficient privileges. To take full advantage of this feature with Cloudera Manager, upgrade to Cloudera Manager 4.7 or higher. For details, see <xref href="impala_security.xml#security"/>. @@ -3550,8 +3569,9 @@ under the License. groups. By assigning privileges for views, you can control access to table data at the column level. For details, see <xref href="impala_security.xml#security"/>. </li> - - <li> + + + <li audience="hidden"> Impala 1.1 works with Cloudera Manager 4.6 or higher. To use Cloudera Manager to manage authorization for the Impala web UI (the web pages served from port 25000 by default), use Cloudera Manager 4.6.2 or higher. @@ -3674,7 +3694,7 @@ under the License. <conbody> - <p> + <p audience="hidden"> The primary enhancements in Impala 1.0.1 are internal, for compatibility with the new Cloudera Manager 4.6 release. Try out the new <uicontrol>Impala Query Monitoring</uicontrol> feature in Cloudera Manager 4.6, which requires Impala 1.0.1. @@ -3798,6 +3818,12 @@ under the License. </li> </ul> + <p audience="hidden"> + In this version, both CDH 4.1 and 4.2 are supported, but due to performance improvements added, we highly + recommend you use CDH 4.2 or higher to see the full benefit. If you are using Cloudera Manager, version 4.5 + is required. + </p> + </conbody> </concept> @@ -3830,6 +3856,10 @@ under the License. </ul> </li> + <li audience="hidden"> + Cloudera Manager 4.5 and CDH 4.2 support Impala 0.6. + </li> + <li> Support for the RCFile file format. For more information on file formats, see <xref href="impala_file_formats.xml#file_formats">Understanding File Formats</xref>. @@ -3878,7 +3908,7 @@ under the License. Added support for Impala on RHEL5.7/Centos5.7. Impala is now supported on RHEL5.7/6.2 and Centos5.7/6.2. </li> - <li> + <li audience="hidden"> Cloudera Manager 4.1.3 supports Impala 0.4. </li>
