[DOCS] Wide-ranging cleanup of CDH and Cloudera references Genericize 3-part version numbers in "known issues".
Genericize CDH version numbers in 'ports' topic. Genericize 'Cloudera' and hostnames in 'Tables' topic. Genericize the version numbers in 'added in' blurbs. Remove lots of CDH / Impala notices from release notes. Remove obsolete conref'able elements that weren't actually being called from anywhere, that contained CDH version number wording. Reword 'Cloudera recommends'. Remove more hidden or commented material with Cloudera-specific wording. Remove obsolete CDH references from 'incompatible changes'. Change 'cloudera' HDFS username for LOAD DATA examples. Remove material related to big lists of CDH fixed JIRAs. Genericize some CDH-related language. Change-Id: Iaa5db6c20f4d010972ade4945a3ea59b32ef95de Reviewed-on: http://gerrit.cloudera.org:8080/6267 Reviewed-by: Ambreen Kazi <[email protected]> Reviewed-by: John Russell <[email protected]> Tested-by: Impala Public Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/18927ac8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/18927ac8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/18927ac8 Branch: refs/heads/master Commit: 18927ac852303288d3ada15ffd3f6b87b43b389e Parents: 3b7cece Author: John Russell <[email protected]> Authored: Fri Mar 3 11:47:08 2017 -0800 Committer: Impala Public Jenkins <[email protected]> Committed: Wed Mar 8 22:40:19 2017 +0000 ---------------------------------------------------------------------- docs/impala_keydefs.ditamap | 32 +- docs/shared/impala_common.xml | 351 +++------------ docs/topics/impala.xml | 49 --- docs/topics/impala_compute_stats.xml | 4 +- docs/topics/impala_create_function.xml | 42 +- docs/topics/impala_decimal.xml | 4 - docs/topics/impala_describe.xml | 16 +- docs/topics/impala_drop_function.xml | 16 +- docs/topics/impala_fixed_issues.xml | 515 +++-------------------- docs/topics/impala_incompatible_changes.xml | 79 +--- docs/topics/impala_known_issues.xml | 121 ++---- docs/topics/impala_load_data.xml | 22 +- docs/topics/impala_misc_functions.xml | 2 +- docs/topics/impala_new_features.xml | 222 +++------- docs/topics/impala_perf_stats.xml | 1 - docs/topics/impala_ports.xml | 10 +- docs/topics/impala_prereqs.xml | 33 -- docs/topics/impala_revoke.xml | 4 +- docs/topics/impala_runtime_filtering.xml | 20 +- docs/topics/impala_s3.xml | 38 +- docs/topics/impala_show.xml | 20 +- docs/topics/impala_tables.xml | 8 +- docs/topics/impala_timestamp.xml | 4 +- docs/topics/impala_tutorial.xml | 32 +- docs/topics/impala_v_cpu_cores.xml | 2 +- docs/topics/impala_varchar.xml | 2 +- 26 files changed, 344 insertions(+), 1305 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/impala_keydefs.ditamap ---------------------------------------------------------------------- diff --git a/docs/impala_keydefs.ditamap b/docs/impala_keydefs.ditamap index 1d35640..424beb8 100644 --- a/docs/impala_keydefs.ditamap +++ b/docs/impala_keydefs.ditamap @@ -10291,20 +10291,50 @@ https://issues.cloudera.org/secure/IssueNavigator.jspa?reset=true&jqlQuery=p <!-- Short form of mapping from Impala release to vendor-specific releases, for use in headings. --> + <keydef keys="impala29"><topicmeta><keywords><keyword>Impala 2.9</keyword></keywords></topicmeta></keydef> <keydef keys="impala28"><topicmeta><keywords><keyword>Impala 2.8</keyword></keywords></topicmeta></keydef> <keydef keys="impala27"><topicmeta><keywords><keyword>Impala 2.7</keyword></keywords></topicmeta></keydef> <keydef keys="impala26"><topicmeta><keywords><keyword>Impala 2.6</keyword></keywords></topicmeta></keydef> <keydef keys="impala25"><topicmeta><keywords><keyword>Impala 2.5</keyword></keywords></topicmeta></keydef> <keydef keys="impala24"><topicmeta><keywords><keyword>Impala 2.4</keyword></keywords></topicmeta></keydef> <keydef keys="impala23"><topicmeta><keywords><keyword>Impala 2.3</keyword></keywords></topicmeta></keydef> - <keydef keys="impala223"><topicmeta><keywords><keyword>Impala 2.2.3</keyword></keywords></topicmeta></keydef> <keydef keys="impala22"><topicmeta><keywords><keyword>Impala 2.2</keyword></keywords></topicmeta></keydef> <keydef keys="impala21"><topicmeta><keywords><keyword>Impala 2.1</keyword></keywords></topicmeta></keydef> <keydef keys="impala20"><topicmeta><keywords><keyword>Impala 2.0</keyword></keywords></topicmeta></keydef> <keydef keys="impala14"><topicmeta><keywords><keyword>Impala 1.4</keyword></keywords></topicmeta></keydef> <keydef keys="impala13"><topicmeta><keywords><keyword>Impala 1.3</keyword></keywords></topicmeta></keydef> +<!-- 3-part forms of version numbers, for use in release notes. --> + <keydef keys="impala290"><topicmeta><keywords><keyword>Impala 2.9.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala280"><topicmeta><keywords><keyword>Impala 2.8.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala270"><topicmeta><keywords><keyword>Impala 2.7.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala263"><topicmeta><keywords><keyword>Impala 2.6.3</keyword></keywords></topicmeta></keydef> + <keydef keys="impala262"><topicmeta><keywords><keyword>Impala 2.6.2</keyword></keywords></topicmeta></keydef> + <keydef keys="impala260"><topicmeta><keywords><keyword>Impala 2.6.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala254"><topicmeta><keywords><keyword>Impala 2.5.4</keyword></keywords></topicmeta></keydef> + <keydef keys="impala253"><topicmeta><keywords><keyword>Impala 2.5.3</keyword></keywords></topicmeta></keydef> + <keydef keys="impala252"><topicmeta><keywords><keyword>Impala 2.5.2</keyword></keywords></topicmeta></keydef> + <keydef keys="impala251"><topicmeta><keywords><keyword>Impala 2.5.1</keyword></keywords></topicmeta></keydef> + <keydef keys="impala250"><topicmeta><keywords><keyword>Impala 2.5.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala241"><topicmeta><keywords><keyword>Impala 2.4.1</keyword></keywords></topicmeta></keydef> + <keydef keys="impala240"><topicmeta><keywords><keyword>Impala 2.4.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala234"><topicmeta><keywords><keyword>Impala 2.3.4</keyword></keywords></topicmeta></keydef> + <keydef keys="impala232"><topicmeta><keywords><keyword>Impala 2.3.2</keyword></keywords></topicmeta></keydef> + <keydef keys="impala231"><topicmeta><keywords><keyword>Impala 2.3.1</keyword></keywords></topicmeta></keydef> + <keydef keys="impala230"><topicmeta><keywords><keyword>Impala 2.3.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala223"><topicmeta><keywords><keyword>Impala 2.2.3</keyword></keywords></topicmeta></keydef> + <keydef keys="impala2210"><topicmeta><keywords><keyword>Impala 2.2.10</keyword></keywords></topicmeta></keydef> + <keydef keys="impala229"><topicmeta><keywords><keyword>Impala 2.2.9</keyword></keywords></topicmeta></keydef> + <keydef keys="impala225"><topicmeta><keywords><keyword>Impala 2.2.5</keyword></keywords></topicmeta></keydef> + <keydef keys="impala220"><topicmeta><keywords><keyword>Impala 2.2.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala210"><topicmeta><keywords><keyword>Impala 2.1.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala200"><topicmeta><keywords><keyword>Impala 2.0.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala140"><topicmeta><keywords><keyword>Impala 1.4.0</keyword></keywords></topicmeta></keydef> + <keydef keys="impala132"><topicmeta><keywords><keyword>Impala 1.3.2</keyword></keywords></topicmeta></keydef> + <keydef keys="impala130"><topicmeta><keywords><keyword>Impala 1.3.0</keyword></keywords></topicmeta></keydef> + <!-- Long form of mapping from Impala release to vendor-specific releases, for use in running text. --> + <keydef keys="impala29_full"><topicmeta><keywords><keyword>Impala 2.9</keyword></keywords></topicmeta></keydef> <keydef keys="impala28_full"><topicmeta><keywords><keyword>Impala 2.8</keyword></keywords></topicmeta></keydef> <keydef keys="impala27_full"><topicmeta><keywords><keyword>Impala 2.7</keyword></keywords></topicmeta></keydef> <keydef keys="impala26_full"><topicmeta><keywords><keyword>Impala 2.6</keyword></keywords></topicmeta></keydef> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/shared/impala_common.xml ---------------------------------------------------------------------- diff --git a/docs/shared/impala_common.xml b/docs/shared/impala_common.xml index 17c0026..af62603 100644 --- a/docs/shared/impala_common.xml +++ b/docs/shared/impala_common.xml @@ -561,7 +561,7 @@ under the License. </tgroup> </table> - <p rev="IMPALA-2660 CDH-40241" id="auth_to_local_instructions"> + <p rev="IMPALA-2660" id="auth_to_local_instructions"> In <keyword keyref="impala26_full"/> and higher, Impala recognizes the <codeph>auth_to_local</codeph> setting, specified through the HDFS configuration setting <codeph>hadoop.security.auth_to_local</codeph>. @@ -860,7 +860,7 @@ select * from t2; </codeblock> <note rev="IMPALA-3334" id="one_but_not_true"> - In CDH 5.7.0 / Impala 2.5.0, only the value 1 enables the option, and the value + In <keyword keyref="impala250"/>, only the value 1 enables the option, and the value <codeph>true</codeph> is not recognized. This limitation is tracked by the issue <xref href="https://issues.cloudera.org/browse/IMPALA-3334"; scope="external" format="html">IMPALA-3334</xref>, @@ -983,8 +983,8 @@ alter table partitioned_data set tblproperties ('numRows'='1030000', 'STATS_GENE </li> <li> <p> - Be careful when using <cmdname>impala-shell</cmdname> on a pre-CDH 5.5 - system to connect to Impala running on a CDH 5.5 or higher system. + Be careful when using <cmdname>impala-shell</cmdname> on a pre-<keyword keyref="impala23"/> + system to connect to a system running <keyword keyref="impala23"/> or higher. The older <cmdname>impala-shell</cmdname> does not recognize these query option names. Upgrade <cmdname>impala-shell</cmdname> on the systems where you intend to use these query options. @@ -1047,7 +1047,7 @@ show tables in temp; <ph rev="2.3.0">-- <keyword keyref="impala23_full"/> and higher:</ph> <ph rev="2.3.0">drop database temp cascade;</ph> --- CDH 5.4 and lower: +-- Earlier releases: drop table temp.t3; drop database temp; </codeblock> @@ -1063,7 +1063,7 @@ drop database temp; </p> <p rev="2.3.0" id="current_timezone_tip"> - To determine the time zone of the server you are connected to, in CDH 5.5 / Impala 2.3 and + To determine the time zone of the server you are connected to, in <keyword keyref="impala23_full"/> and higher you can call the <codeph>timeofday()</codeph> function, which includes the time zone specifier in its return value. Remember that with cloud computing, the server you interact with might be in a different time zone than you are, or different sessions might connect to @@ -1077,8 +1077,8 @@ drop database temp; how Impala handles time zone considerations for the <codeph>TIMESTAMP</codeph> data type. </p> - <p rev="2.6.0 CDH-39913 IMPALA-3558" id="s3_drop_table_purge"> - For best compatibility with the S3 write support in CDH 5.8 / Impala 2.6 + <p rev="2.6.0 IMPALA-3558" id="s3_drop_table_purge"> + For best compatibility with the S3 write support in <keyword keyref="impala26_full"/> and higher: <ul> <li>Use native Hadoop techniques to create data files in S3 for querying through Impala.</li> @@ -1095,7 +1095,7 @@ drop database temp; <codeph>hadoop fs -cp</codeph>, or <codeph>INSERT</codeph> in Impala or Hive. </p> - <p rev="2.6.0 CDH-39913 IMPALA-1878" id="s3_dml_performance"> + <p rev="2.6.0 IMPALA-1878" id="s3_dml_performance"> Because of differences between S3 and traditional filesystems, DML operations for S3 tables can take longer than for tables on HDFS. For example, both the <codeph>LOAD DATA</codeph> statement and the final stage of the <codeph>INSERT</codeph> @@ -1104,14 +1104,14 @@ drop database temp; the files are moved from a temporary staging directory to the final destination directory.) Because S3 does not support a <q>rename</q> operation for existing objects, in these cases Impala actually copies the data files from one location to another and then removes the original files. - In CDH 5.8 / Impala 2.6, the <codeph>S3_SKIP_INSERT_STAGING</codeph> query option provides a way + In <keyword keyref="impala26_full"/>, the <codeph>S3_SKIP_INSERT_STAGING</codeph> query option provides a way to speed up <codeph>INSERT</codeph> statements for S3 tables and partitions, with the tradeoff that a problem during statement execution could leave data in an inconsistent state. It does not apply to <codeph>INSERT OVERWRITE</codeph> or <codeph>LOAD DATA</codeph> statements. See <xref href="../topics/impala_s3_skip_insert_staging.xml#s3_skip_insert_staging"/> for details. </p> - <p rev="2.6.0 CDH-40329 IMPALA-3453" id="s3_block_splitting"> + <p rev="2.6.0 IMPALA-3453" id="s3_block_splitting"> In <keyword keyref="impala26_full"/> and higher, Impala queries are optimized for files stored in Amazon S3. For Impala tables that use the file formats Parquet, RCFile, SequenceFile, Avro, and uncompressed text, the setting <codeph>fs.s3a.block.size</codeph> @@ -1126,7 +1126,7 @@ drop database temp; to 268435456 (256 MB) to match the row group size produced by Impala. </p> - <note rev="2.6.0 CDH-39913 IMPALA-1878" id="s3_production" type="important"> + <note rev="2.6.0 IMPALA-1878" id="s3_production" type="important"> <p> In <keyword keyref="impala26_full"/> and higher, Impala supports both queries (<codeph>SELECT</codeph>) and DML (<codeph>INSERT</codeph>, <codeph>LOAD DATA</codeph>, <codeph>CREATE TABLE AS SELECT</codeph>) @@ -1137,23 +1137,22 @@ drop database temp; </note> <note rev="2.2.0" id="s3_caveat" type="important"> - <p> Impala query support for Amazon S3 is included in CDH 5.4.0, but is - not currently supported or recommended for production use. To try this - feature, use it in a test environment until Cloudera resolves - currently existing issues and limitations to make it ready for - production use. </p> + <p> + Impala query support for Amazon S3 is included in <keyword keyref="impala22_full"/>, but is + not supported or recommended for production use in this version. + </p> </note> - <p rev="2.6.0 CDH-39913 IMPALA-1878" id="s3_ddl"> + <p rev="2.6.0 IMPALA-1878" id="s3_ddl"> In <keyword keyref="impala26_full"/> and higher, Impala DDL statements such as <codeph>CREATE DATABASE</codeph>, <codeph>CREATE TABLE</codeph>, <codeph>DROP DATABASE CASCADE</codeph>, <codeph>DROP TABLE</codeph>, and <codeph>ALTER TABLE [ADD|DROP] PARTITION</codeph> can create or remove folders - as needed in the Amazon S3 system. Prior to CDH 5.8 / Impala 2.6, you had to create folders yourself and point + as needed in the Amazon S3 system. Prior to <keyword keyref="impala26_full"/>, you had to create folders yourself and point Impala database, tables, or partitions at them, and manually remove folders when no longer needed. See <xref href="../topics/impala_s3.xml#s3"/> for details about reading and writing S3 data with Impala. </p> - <p rev="2.6.0 CDH-39913 IMPALA-1878" id="s3_dml"> + <p rev="2.6.0 IMPALA-1878" id="s3_dml"> In <keyword keyref="impala26_full"/> and higher, the Impala DML statements (<codeph>INSERT</codeph>, <codeph>LOAD DATA</codeph>, and <codeph>CREATE TABLE AS SELECT</codeph>) can write data into a table or partition that resides in the Amazon Simple Storage Service (S3). @@ -1235,7 +1234,7 @@ drop database temp; Prior to <keyword keyref="impala25_full"/> the requirement to reload functions after a restart applied to both C++ and Java functions. </p> - <p id="current_user_caveat" rev="CDH-36552"> + <p id="current_user_caveat" rev=""> The Hive <codeph>current_user()</codeph> function cannot be called from a Java UDF through Impala. </p> @@ -1466,7 +1465,8 @@ explain select s from yy2 where year in (select year from yy where year between <codeph>--scratch_dirs="<varname>path_to_directory</varname>"</codeph> configuration option. You can specify a single directory, or a comma-separated list of directories. The scratch directories must be on the local filesystem, not in HDFS. You might specify different directory paths for different hosts, - depending on the capacity and speed of the available storage devices. In CDH 5.5 / Impala 2.3 or higher, + depending on the capacity and speed + of the available storage devices. In <keyword keyref="impala23_full"/> or higher, Impala successfully starts (with a warning Impala successfully starts (with a warning written to the log) if it cannot create or read and write files in one of the scratch directories. If there is less than 1 GB free on the filesystem where that directory resides, Impala still runs, but writes a warning message to its log. If Impala encounters an error reading or writing @@ -1641,14 +1641,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*'; or file formats. </p> - <p rev="1.4.0" id="decimal_no_stats"> - Currently, the <codeph>COMPUTE STATS</codeph> statement under CDH 4 does not store any statistics for - <codeph>DECIMAL</codeph> columns. When Impala runs under CDH 5, which has better support for - <codeph>DECIMAL</codeph> in the metastore database, <codeph>COMPUTE STATS</codeph> does collect statistics - for <codeph>DECIMAL</codeph> columns and Impala uses the statistics to optimize query performance. - </p> - - <p rev="CDH-35866" id="hive_column_stats_caveat"> + <p rev="" id="hive_column_stats_caveat"> If you run the Hive statement <codeph>ANALYZE TABLE COMPUTE STATISTICS FOR COLUMNS</codeph>, Impala can only use the resulting column statistics if the table is unpartitioned. Impala cannot use Hive-generated column statistics for a partitioned table. @@ -1765,7 +1758,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*'; <b>Sqoop considerations:</b> </p> - <p id="sqoop_timestamp_caveat" rev="IMPALA-2111 CDH-37399"> If you use Sqoop to + <p id="sqoop_timestamp_caveat" rev="IMPALA-2111"> If you use Sqoop to convert RDBMS data to Parquet, be careful with interpreting any resulting values from <codeph>DATE</codeph>, <codeph>DATETIME</codeph>, or <codeph>TIMESTAMP</codeph> columns. The underlying values are @@ -1809,7 +1802,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*'; The <codeph>COMPUTE STATS</codeph> statement does not produce any statistics for columns of this data type. </p> </li> - <li rev="CDH-35868"> + <li rev=""> <p id="complex_types_max_length"> The maximum length of the column definition for any complex type, including declarations for any nested types, is 4000 characters. @@ -1909,7 +1902,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*'; are available in <keyword keyref="impala23_full"/> and higher. To use these types with JDBC requires version 2.5.28 or higher of the Cloudera JDBC Connector for Impala. To use these types with ODBC requires version 2.5.30 or higher of the Cloudera ODBC Connector for Impala. - Consider upgrading all JDBC and ODBC drivers at the same time you upgrade from CDH 5.5 or higher. + Consider upgrading all JDBC and ODBC drivers at the same time you upgrade from <keyword keyref="impala23_full"/> or higher. </p> <p rev="2.3.0" id="jdbc_odbc_complex_types_views"> @@ -2015,11 +2008,11 @@ order by r_name; <b>Hive considerations:</b> </p> - <p rev="CDH-19187" id="permissions_blurb"> + <p rev="" id="permissions_blurb"> <b>HDFS permissions:</b> </p> - <p rev="CDH-19187" id="permissions_blurb_no"> + <p rev="" id="permissions_blurb_no"> <b>HDFS permissions:</b> This statement does not touch any HDFS files or directories, therefore no HDFS permissions are required. </p> @@ -2108,16 +2101,17 @@ order by r_name; <b>Parquet considerations:</b> </p> +<!-- Github project for parquet-tools: https://github.com/Parquet/parquet-mr/tree/master/parquet-tools --> <p id="parquet_tools_blurb"> To examine the internal structure and data of Parquet files, you can use the - <cmdname>parquet-tools</cmdname> command that comes with CDH. Make sure this + <cmdname>parquet-tools</cmdname> command. Make sure this command is in your <codeph>$PATH</codeph>. (Typically, it is symlinked from <filepath>/usr/bin</filepath>; sometimes, depending on your installation setup, you - might need to locate it under a CDH-specific <codeph>bin</codeph> directory.) + might need to locate it under an alternative <codeph>bin</codeph> directory.) The arguments to this command let you perform operations such as: <ul> <li> - <codeph>cat</codeph>: Print a file's contents to standard out. In <ph rev="upstream">CDH 5.5</ph> and higher, you can use + <codeph>cat</codeph>: Print a file's contents to standard out. In <keyword keyref="impala23_full"/> and higher, you can use the <codeph>-j</codeph> option to output JSON. </li> <li> @@ -2215,7 +2209,7 @@ message schema { <codeblock><![CDATA[ $ parquet-tools meta sample.parq -creator: impala version 2.2.0-cdh5.4.3 (build 517bb0f71cd604a00369254ac6d88394df83e0f6) +creator: impala version 2.2.0-... file schema: schema ------------------------------------------------------------------- @@ -2256,7 +2250,7 @@ flight_num: INT32 SNAPPY DO:83456393 FPO:83488603 SZ:10216514/11474301 This function cannot be used in an analytic context. That is, the <codeph>OVER()</codeph> clause is not allowed at all with this function. </p> - <p rev="CDH-40418" id="analytic_partition_pruning_caveat"> + <p rev="" id="analytic_partition_pruning_caveat"> In queries involving both analytic functions and partitioned tables, partition pruning only occurs for columns named in the <codeph>PARTITION BY</codeph> clause of the analytic function call. For example, if an analytic function query has a clause such as <codeph>WHERE year=2016</codeph>, the way to make the query prune all other <codeph>YEAR</codeph> partitions is to include <codeph>PARTITION BY year</codeph>in the analytic function call; @@ -2476,26 +2470,26 @@ flight_num: INT32 SNAPPY DO:83456393 FPO:83488603 SZ:10216514/11474301 </p> <p rev="2.7.0" id="added_in_270"> - <b>Added in:</b> CDH 5.9.0 (Impala 2.7.0) + <b>Added in:</b> <keyword keyref="impala270"/> </p> <p rev="2.6.0" id="added_in_260"> - <b>Added in:</b> CDH 5.8.0 (Impala 2.6.0) + <b>Added in:</b> <keyword keyref="impala260"/> </p> <p rev="2.5.0" id="added_in_250"> - <b>Added in:</b> CDH 5.7.0 (Impala 2.5.0) + <b>Added in:</b> <keyword keyref="impala250"/> </p> <p rev="2.3.0" id="added_in_230"> - <b>Added in:</b> CDH 5.5.0 (Impala 2.3.0) + <b>Added in:</b> <keyword keyref="impala230"/> </p> <p rev="2.0.0" id="added_in_20"> - <b>Added in:</b> CDH 5.2.0 (Impala 2.0.0) + <b>Added in:</b> <keyword keyref="impala200"/> </p> <p rev="2.0.0" id="enhanced_in_20"> <b>Added in:</b> Available in earlier Impala releases, but new capabilities were added in - CDH 5.2.0 / Impala 2.0.0 + <keyword keyref="impala200"/> </p> <p id="added_forever"> @@ -2519,11 +2513,11 @@ flight_num: INT32 SNAPPY DO:83456393 FPO:83488603 SZ:10216514/11474301 </p> <p id="added_in_210" rev="2.1.0"> - <b>Added in:</b> CDH 5.3.0 (Impala 2.1.0) + <b>Added in:</b> <keyword keyref="impala210"/> </p> <p id="added_in_220" rev="2.2.0"> - <b>Added in:</b> CDH 5.4.0 (Impala 2.2.0) + <b>Added in:</b> <keyword keyref="impala220"/> </p> <p id="syntax_blurb"> @@ -2799,7 +2793,7 @@ select max(height), avg(height) from census_data where age > 20; <!-- This same text is conref'ed in the #views and the #partition_pruning topics. --> - <p id="partitions_and_views" rev="CDH-36224"> + <p id="partitions_and_views" rev=""> If a view applies to a partitioned table, any partition pruning considers the clauses on both the original query and any additional <codeph>WHERE</codeph> predicates in the query that refers to the view. Prior to Impala 1.4, only the <codeph>WHERE</codeph> clauses on the original query from the @@ -2823,7 +2817,7 @@ Query finished, fetching results ... | | NULL | NULL | | # Detailed Table Information | NULL | NULL | | Database: | views | NULL | -| Owner: | cloudera | NULL | +| Owner: | doc_demo | NULL | | CreateTime: | Mon Jul 08 15:56:27 EDT 2013 | NULL | | LastAccessTime: | UNKNOWN | NULL | | Protect Mode: | None | NULL | @@ -3121,16 +3115,6 @@ select * from header_line limit 10; Other security settings may prevent Impala from writing core dumps even when this option is enabled. </p> </li> - <li rev="CDH-34070" audience="hidden"> - <p> - On systems managed by Cloudera Manager, the default location for core dumps is on a temporary - filesystem, which can lead to out-of-space issues if the core dumps are large, frequent, or - not removed promptly. To specify an alternative location for the core dumps, filter the - Impala configuration settings to find the <codeph>core_dump_dir</codeph> option, which is - available in Cloudera Manager 5.4.3 and higher. This option lets you specify a different directory - for core dumps for each of the Impala-related daemons. - </p> - </li> </ul> </note> <p id="cpu_prereq" rev="2.2.0"> @@ -3157,67 +3141,21 @@ sudo pip-python install ssl</codeblock> </p> <note type="warning" id="impala_kerberos_ssl_caveat"> - Prior to CDH 5.5.2 / Impala 2.3.2, you could enable Kerberos authentication between Impala internal components, + Prior to <keyword keyref="impala232"/>, you could enable Kerberos authentication between Impala internal components, or SSL encryption between Impala internal components, but not both at the same time. This restriction has now been lifted. - See <xref href="https://issues.cloudera.org/browse/IMPALA-2598"; scope="external" format="html">IMPALA-2598</xref> - to see the maintenance releases for different levels of CDH where the fix has been published. + See <xref keyref="IMPALA-2598">IMPALA-2598</xref> + to see the maintenance releases for different levels of Impala where the fix has been published. </note> <p id="hive_jdbc_ssl_kerberos_caveat"> - Prior to CDH 5.7 / Impala 2.5, the Hive JDBC driver did not support connections that use both Kerberos authentication + Prior to <keyword keyref="impala25_full"/>, the Hive JDBC driver did not support connections that use both Kerberos authentication and SSL encryption. If your cluster is running an older release that has this restriction, to use both of these security features with Impala through a JDBC application, use the <xref href="http://www.cloudera.com/content/www/en-us/downloads.html.html"; scope="external" format="html">Cloudera JDBC Connector</xref> as the JDBC driver. </p> - <note rev="1.2" id="cdh4_cdh5_upgrade"> - Because Impala 1.2.2 works with CDH 4, while the Impala that comes with the CDH 5 beta is version 1.2.0, - upgrading from CDH 4 to the CDH 5 beta actually reverts to an earlier Impala version. The beta release of - Impala that comes with the CDH 5 beta includes the resource management feature that relies on the CDH 5 - infrastructure, as well as the much-requested user-defined function feature and the catalog service. - However, it does not include new features in Impala 1.2.3 such as join order optimizations, <codeph>COMPUTE - STATS</codeph> statement, <codeph>CROSS JOIN</codeph> operator, <codeph>SHOW CREATE TABLE</codeph> - statement, <codeph>SHOW TABLE STATS</codeph> and <codeph>SHOW COLUMN STATS</codeph> statements, - <codeph>OFFSET</codeph> and <codeph>NULLS FIRST/LAST</codeph> clauses for queries, and the - <codeph>SYNC_DDL</codeph> query option. - </note> - - <p rev="1.2" id="cm48_upgrade" audience="hidden"><!-- conref'ed in 2 places under 'incompatible changes' --> - In a Cloudera Manager environment, the catalog service is not recognized or managed by Cloudera Manager - versions prior to 4.8. Cloudera Manager 4.8 and higher require the catalog service to be present for - Impala. Therefore, if you upgrade to Cloudera Manager 4.8 or higher, you must also upgrade Impala to 1.2.1 - or higher. Likewise, if you upgrade Impala to 1.2.1 or higher, you must also upgrade Cloudera Manager to - 4.8 or higher. - </p> - - <p id="impala_hdfs_group_obsolete" rev="1.2"> - For the resource management feature to work (in combination with CDH 5 and the YARN and Llama components), - the <codeph>impala</codeph> user must be a member of the <codeph>hdfs</codeph> group. This setup is - performed automatically during a new install, but not when upgrading from earlier Impala releases to Impala - 1.2. If you are upgrading a node that already had Impala 1.1 or 1.0 installed, manually add the - <codeph>impala</codeph> user to the <codeph>hdfs</codeph> group. - </p> - - <p id="impala_hdfs_group" rev="1.2"> - Prior to CDH 5.5 / Impala 2.3, - the <codeph>impala</codeph> user was required to be a member of the <codeph>hdfs</codeph> group - for the resource management feature to work (in combination with CDH 5 and the YARN and Llama components). - This requirement has been lifted in <keyword keyref="impala23_full"/> and higher. The <codeph>impala</codeph> - user remains in the <codeph>hdfs</codeph> group on upgraded systems if it was already there, but is - no longer put into that group during new installs. - </p> - - <note id="impala13_cdh"> - <ul> - <li> - The Impala 1.3.1 release is available for both CDH 4 and CDH 5. This is the first release in the 1.3.x - series for CDH 4. - </li> - </ul> - </note> - </section> <section id="performance_conrefs"> @@ -3261,7 +3199,7 @@ sudo pip-python install ssl</codeblock> <title>Administration</title> - <p id="statestored_catalogd_ha_blurb" rev="CDH-39624"> + <p id="statestored_catalogd_ha_blurb" rev=""> Most considerations for load balancing and high availability apply to the <cmdname>impalad</cmdname> daemon. The <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname> daemons do not have special requirements for high availability, because problems with those daemons do not result in data loss. @@ -3354,203 +3292,26 @@ sudo pip-python install ssl</codeblock> the connection has been closed. </note> - <p id="impala_mr"> - For a detailed information about configuring a cluster to share resources between Impala queries and MapReduce jobs, see - <xref audience="integrated" href="../topics/admin_howto_multitenancy.xml#howto_multitenancy"/><xref audience="standalone" href="https://www.cloudera.com/documentation/enterprise/latest/topics/admin_howto_multitenancy.html"; format="html" scope="external"/> - <!--and <xref href="../topics/impala_howto_rm.xml#howto_impala_rm"/>-->. - </p> - - <note id="llama_beta" type="warning"> - In CDH 5.0.0, the Llama component is in beta. It is intended for evaluation of resource management in test - environments, in combination with Impala and YARN. It is currently not recommended for production - deployment. - </note> - </section> - <section id="cdh5_conrefs"> + <section id="upstream_conrefs"> - <title>CDH5 Integration</title> + <title>Upstream Cleanup</title> <p> - Snippets related to CDH 5 integration, for example phrase tags that are conditionalized in or out of + Snippets related to upstream cleanup work, for example phrase tags that are conditionalized in or out of 'integrated' and 'standalone' conditions to provide extra context for links that don't work in certain PDF contexts. </p> - <p id="cdh551_noop"> - The version of Impala that is included with CDH 5.5.1 is identical to the Impala for CDH 5.5.0. + <p id="impala231_noop"> + The version of Impala that is included with <keyword keyref="impala231"/> is identical to <keyword keyref="impala230"/>. There are no new bug fixes, new features, or incompatible changes. </p> -<!-- This next one is not actually used. --> - <note id="only_cdh5_260"> - Impala 2.6.x is available as part of CDH 5.8.x. - </note> - - <note id="only_cdh5_250"> - Impala 2.5.x is available as part of CDH 5.7.x and is not available for CDH 4. - Cloudera does not intend to release future versions of Impala for CDH 4 outside patch and maintenance releases if required. - Given the end-of-maintenance status for CDH 4, <ph rev="upstream">Cloudera</ph> recommends all customers to migrate to a recent CDH 5 release. - </note> - -<!-- These next 2 for Impala 2.4 / CDH 5.6 are not actually used. Trying to move away from the repetitive "don't use CDH 4" notes. --> - - <note id="only_cdh5_24x"> - Impala 2.4.x is available as part of CDH 5.6.x and is not available for CDH 4. - Cloudera does not intend to release future versions of Impala for CDH 4 outside patch and maintenance releases if required. - Given the end-of-maintenance status for CDH 4, <ph rev="upstream">Cloudera</ph> recommends all customers to migrate to a recent CDH 5 release. - </note> - - <note id="only_cdh5_240"> - Impala 2.4.0 is available as part of CDH 5.6.0 and is not available for CDH 4. - Cloudera does not intend to release future versions of Impala for CDH 4 outside patch and maintenance releases if required. - Given the end-of-maintenance status for CDH 4, <ph rev="upstream">Cloudera</ph> recommends all customers to migrate to a recent CDH 5 release. - </note> - - <note id="only_cdh5_23x"> - Impala 2.3.x is available as part of CDH 5.5.x and is not available for CDH 4. - Cloudera does not intend to release future versions of Impala for CDH 4 outside patch and maintenance releases if required. - Given the end-of-maintenance status for CDH 4, <ph rev="upstream">Cloudera</ph> recommends all customers to migrate to a recent CDH 5 release. - </note> - -<!-- My original suggested wording. Using Justin E's suggestion instead, below. - <note id="only_cdh5_220"> - Impala 2.2.0 is available as part of CDH 5.4.0, not under CDH 4. - With the Impala 2.2.x series, Cloudera intends all future - Impala feature work to be released only with CDH 5, rather than - having an equivalent CDH 4 release. - </note> ---> - - <note id="only_cdh5_229"> - Impala 2.2.9 is available as part of CDH 5.4.9, not under CDH 4. - </note> - - <note id="only_cdh5_228"> - Impala 2.2.8 is available as part of CDH 5.4.8, not under CDH 4. - </note> - - <note id="only_cdh5_227"> - Impala 2.2.7 is available as part of CDH 5.4.7, not under CDH 4. - </note> - - <note id="only_cdh5_226"> - Impala 2.2.6 is available as part of CDH 5.4.6, not under CDH 4. - </note> - - <note id="only_cdh5_225"> - Impala 2.2.5 is available as part of CDH 5.4.5, not under CDH 4. - </note> - - <note id="only_cdh5_224"> - Impala 2.2.4 is available as part of CDH 5.4.4, not under CDH 4. - </note> - - <note id="only_cdh5_223"> - Impala 2.2.3 is available as part of CDH 5.4.3, not under CDH 4. - </note> - - <note id="only_cdh5_222"> - Impala 2.2.2 is available as part of CDH 5.4.2, not under CDH 4. - </note> - - <note id="only_cdh5_221"> - Impala 2.2.1 is available as part of CDH 5.4.1, not under CDH 4. - </note> - - <note id="only_cdh5_22x"> - The Impala 2.2.x maintenance releases now use the CDH 5.4.x numbering system - rather than increasing the Impala version numbers. - Impala 2.2 and higher are not available under CDH 4. - </note> - - <note id="only_cdh5_220"> - Impala 2.2.0 is available as part of CDH 5.4.0 and is not available for CDH 4. - Cloudera does not intend to release future versions of Impala for CDH 4 outside patch and maintenance releases if required. - Given the end-of-maintenance status for CDH 4, <ph rev="upstream">Cloudera</ph> recommends all customers to migrate to a recent CDH 5 release. - </note> - - <note id="only_cdh5_21x"> - This Impala maintenance release is only available as part of CDH 5, not under CDH 4. - </note> - - <note id="only_cdh5_213"> - Impala 2.1.3 is available as part of CDH 5.3.3, not under CDH 4. - </note> - - <note id="only_cdh5_212"> - Impala 2.1.2 is available as part of CDH 5.3.2, not under CDH 4. - </note> - - <note id="only_cdh5_205"> - Impala 2.0.5 is available as part of CDH 5.2.6, not under CDH 4. - </note> - - <note id="only_cdh5_204"> - Impala 2.0.4 is available as part of CDH 5.2.5, not under CDH 4. - </note> - - <note id="only_cdh5_203"> - Impala 2.0.3 is available as part of CDH 5.2.4, not under CDH 4. - </note> - - <note id="only_cdh5_202"> - Impala 2.0.2 is available as part of CDH 5.2.3, not under CDH 4. - </note> - - <note id="only_cdh5_144"> - Impala 1.4.4 is available as part of CDH 5.1.5, not under CDH 4. - </note> - - <note id="only_cdh5_143"> - Impala 1.4.3 is available as part of CDH 5.1.4, and under CDH 4. - </note> - - <note id="only_cdh5_142"> - Impala 1.4.2 is only available as part of CDH 5.1.3, not under CDH 4. - </note> - - <note id="only_cdh5_133"> - Impala 1.3.3 is only available as part of CDH 5.0.5, not under CDH 4. - </note> - - <note id="only_cdh5_132"> - Impala 1.3.2 is only available as part of CDH 5.0.4, not under CDH 4. - </note> - - <note id="only_cdh5_141"> -<!-- Impala 1.4.1 is only available as part of CDH 5.1.2; this bug-fix release is not available for CDH 4. --> - Impala 1.4.1 is only available as part of CDH 5.1.2, not under CDH 4. - </note> - - <note id="standalone_release_notes_blurb"> - Starting in April 2016, future release note updates are being consolidated - in a single location to avoid duplication of stale or incomplete information. - You can view online the Impala - <xref href="http://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_new_features.html"; scope="external" format="html">New Features</xref>, - <xref href="http://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_incompatible_changes.html"; scope="external" format="html">Incompatible Changes</xref>, - <xref href="http://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_known_issues.html"; scope="external" format="html">Known Issues</xref>, and - <xref href="http://www.cloudera.com/documentation/enterprise/release-notes/topics/impala_fixed_issues.html"; scope="external" format="html">Fixed Issues</xref>. - You can view or print all of these by downloading <xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/impala.html"; scope="external" format="html">the latest Impala PDF</xref>. - </note> - <!-- The only significant text in this paragraph is inside the <ph> tags. Those are conref'ed into sentences similar in form to the ones below. --> - <p> - <ph id="impala_214_redux">Because CDH 5.3.5 does not include any code changes for Impala, Impala 2.1.4 - is included with both CDH 5.3.4 and 5.3.5.</ph> - See such-and-such a topic <ph id="cdh5ig" audience="integrated">in the <cite>CDH 5 Installation - Guide</cite></ph>. See such-and-such a topic <ph id="cdh5sg" audience="integrated">in the <cite>CDH 5 - Security Guide</cite></ph>. See such-and-such a topic <ph id="cdh5rn" audience="integrated">in the - <cite>CDH 5 Release Notes</cite></ph>. See such-and-such a topic <ph id="ciiu" audience="integrated">in the - <cite>Impala User Guide</cite></ph>. See such-and-such a topic - <ph id="cirn" audience="integrated">in the <cite>Impala Release Notes</cite></ph>. See - such-and-such a topic <ph id="cifaq" audience="integrated">in the <cite>Impala Frequently Asked - Questions</cite></ph>. - </p> - <note id="admission_compute_stats"> Impala relies on the statistics produced by the <codeph>COMPUTE STATS</codeph> statement to estimate memory usage for each query. See <xref href="../topics/impala_compute_stats.xml#compute_stats"/> for guidelines @@ -3608,7 +3369,7 @@ sudo pip-python install ssl</codeblock> </p> <p id="IMPALA-3662" rev="IMPALA-3662"> - The initial release of CDH 5.7 / Impala 2.5 sometimes has a higher peak memory usage than in previous releases + The initial release of <keyword keyref="impala25_full"/> sometimes has a higher peak memory usage than in previous releases while reading Parquet files. The following query options might help to reduce memory consumption in the Parquet scanner: <ul> @@ -3626,7 +3387,7 @@ sudo pip-python install ssl</codeblock> <xref href="https://issues.cloudera.org/browse/IMPALA-3662"; scope="external" format="html">IMPALA-3662</xref>. </p> - <p id="increase_catalogd_heap_size" rev="CDH-40801 TSB-168"> + <p id="increase_catalogd_heap_size" rev=""> For schemas with large numbers of tables, partitions, and data files, the <cmdname>catalogd</cmdname> daemon might encounter an out-of-memory error. To increase the memory limit for the <cmdname>catalogd</cmdname> daemon: http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/topics/impala.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala.xml b/docs/topics/impala.xml index 922a830..ba6884d 100644 --- a/docs/topics/impala.xml +++ b/docs/topics/impala.xml @@ -42,54 +42,5 @@ under the License. <p outputclass="toc"/> - <p audience="integrated"> - <b>Related information throughout the CDH 5 library:</b> - </p> - - <p audience="integrated"> - In CDH 5, the Impala documentation for Release Notes, Installation, Upgrading, and Security has been - integrated alongside the corresponding information for other Hadoop components: - </p> - -<!-- Same list is in impala.xml and Impala FAQs. Conref in both places. --> - - <ul> - <li> - <xref href="impala_new_features.xml#new_features">New features</xref> - </li> - - <li> - <xref href="impala_known_issues.xml#known_issues">Known and fixed issues</xref> - </li> - - <li> - <xref href="impala_incompatible_changes.xml#incompatible_changes">Incompatible changes</xref> - </li> - - <li> - <xref href="impala_install.xml#install">Installing Impala</xref> - </li> - - <li> - <xref href="impala_upgrading.xml#upgrading">Upgrading Impala</xref> - </li> - - <li> - <xref href="impala_config.xml#config">Configuring Impala</xref> - </li> - - <li> - <xref href="impala_processes.xml#processes">Starting Impala</xref> - </li> - - <li> - <xref href="impala_security.xml#security">Security for Impala</xref> - </li> - - <li> - <xref href="http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH-Version-and-Packaging-Information/CDH-Version-and-Packaging-Information.html"; scope="external" format="html">CDH - Version and Packaging Information</xref> - </li> - </ul> </conbody> </concept> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/topics/impala_compute_stats.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_compute_stats.xml b/docs/topics/impala_compute_stats.xml index bd21dae..dc14f1e 100644 --- a/docs/topics/impala_compute_stats.xml +++ b/docs/topics/impala_compute_stats.xml @@ -487,8 +487,6 @@ show table stats item_partitioned; <p conref="../shared/impala_common.xml#common/restrictions_blurb"/> - <p conref="../shared/impala_common.xml#common/decimal_no_stats"/> - <note conref="../shared/impala_common.xml#common/compute_stats_nulls"/> <p conref="../shared/impala_common.xml#common/internals_blurb"/> @@ -507,7 +505,7 @@ show table stats item_partitioned; </p> <p conref="../shared/impala_common.xml#common/permissions_blurb"/> - <p rev="CDH-19187"> + <p rev=""> The user ID that the <cmdname>impalad</cmdname> daemon runs under, typically the <codeph>impala</codeph> user, must have read permission for all affected files in the source directory: http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/topics/impala_create_function.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_create_function.xml b/docs/topics/impala_create_function.xml index 15f8eb2..4afa237 100644 --- a/docs/topics/impala_create_function.xml +++ b/docs/topics/impala_create_function.xml @@ -51,7 +51,7 @@ under the License. multiple functions that compute intermediate results across sets of rows. </p> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> In <keyword keyref="impala25_full"/> and higher, the syntax is also different for creating or dropping scalar Java-based UDFs. The statements for Java UDFs use a new syntax, without any argument types or return type specified. Java-based UDFs created using the new syntax persist across restarts of the Impala catalog server, and can be shared transparently @@ -67,7 +67,7 @@ under the License. LOCATION '<varname>hdfs_path_to_dot_so</varname>' SYMBOL='<varname>symbol_name</varname>'</codeblock> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> To create a persistent Java UDF with <codeph>CREATE FUNCTION</codeph>: <codeblock>CREATE FUNCTION [IF NOT EXISTS] [<varname>db_name</varname>.]<varname>function_name</varname> LOCATION '<varname>hdfs_path_to_jar</varname>' @@ -96,7 +96,7 @@ DROP FUNCTION IF EXISTS bar; [CLOSEFN='<varname>function</varname>] <ph rev="2.0.0">[SERIALIZE_FN='<varname>function</varname>]</ph> [FINALIZE_FN='<varname>function</varname>] - <ph rev="2.3.0 IMPALA-1829 CDH-30572">[INTERMEDIATE <varname>type_spec</varname>]</ph></codeblock> + <ph rev="2.3.0 IMPALA-1829">[INTERMEDIATE <varname>type_spec</varname>]</ph></codeblock> <p conref="../shared/impala_common.xml#common/ddl_blurb"/> @@ -104,8 +104,8 @@ DROP FUNCTION IF EXISTS bar; <b>Varargs notation:</b> </p> - <note rev="CDH-39271 CDH-38572"> - <p rev="CDH-39271 CDH-38572"> + <note rev=""> + <p rev=""> Variable-length argument lists are supported for C++ UDFs, but currently not for Java UDFs. </p> </note> @@ -139,7 +139,7 @@ DROP FUNCTION IF EXISTS bar; </li> </ul> - <p rev="CDH-39271 CDH-38572"> + <p rev=""> See <xref href="impala_udf.xml#udf_varargs"/> for how to code a C++ UDF to accept variable-length argument lists. </p> @@ -244,7 +244,7 @@ DROP FUNCTION IF EXISTS bar; most suited to reusing existing Hive UDFs. (Impala can run Java-based Hive UDFs but not Hive UDAs.) </li> - <li rev="2.5.0 IMPALA-1748 CDH-38369 IMPALA-2843 CDH-39148"> + <li rev="2.5.0 IMPALA-1748 IMPALA-2843"> <keyword keyref="impala25_full"/> introduces UDF improvements to persistence for both C++ and Java UDFs, and better compatibility between Impala and Hive for Java UDFs. See <xref href="impala_udf.xml#udfs"/> for details. @@ -343,19 +343,19 @@ DROP FUNCTION IF EXISTS bar; For additional examples of all kinds of user-defined functions, see <xref href="impala_udf.xml#udfs"/>. </p> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> The following example shows how to take a Java jar file and make all the functions inside one of its classes into UDFs under a single (overloaded) function name in Impala. Each <codeph>CREATE FUNCTION</codeph> or <codeph>DROP FUNCTION</codeph> statement applies to all the overloaded Java functions with the same name. This example uses the signatureless syntax for <codeph>CREATE FUNCTION</codeph> and <codeph>DROP FUNCTION</codeph>, which is available in <keyword keyref="impala25_full"/> and higher. </p> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> At the start, the jar file is in the local filesystem. Then it is copied into HDFS, so that it is available for Impala to reference through the <codeph>CREATE FUNCTION</codeph> statement and queries that refer to the Impala function name. </p> -<codeblock rev="2.5.0 IMPALA-2843 CDH-39148"> +<codeblock rev="2.5.0 IMPALA-2843"> $ jar -tvf udf-examples-cdh570.jar 0 Mon Feb 22 04:06:50 PST 2016 META-INF/ 122 Mon Feb 22 04:06:48 PST 2016 META-INF/MANIFEST.MF @@ -373,13 +373,13 @@ Found 2 items -rw-r--r-- 3 jrussell supergroup 853 2015-10-09 14:05 /user/impala/udfs/hello_world.jar -rw-r--r-- 3 jrussell supergroup 7366 2016-06-08 14:25 /user/impala/udfs/udf-examples-cdh570.jar </codeblock> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> In <cmdname>impala-shell</cmdname>, the <codeph>CREATE FUNCTION</codeph> refers to the HDFS path of the jar file and the fully qualified class name inside the jar. Each of the functions inside the class becomes an Impala function, each one overloaded under the specified Impala function name. </p> -<codeblock rev="2.5.0 IMPALA-2843 CDH-39148"> -[localhost:21000] > create function testudf location '/user/impala/udfs/udf-examples-cdh570.jar' symbol='com.cloudera.impala.TestUdf'; +<codeblock rev="2.5.0 IMPALA-2843"> +[localhost:21000] > create function testudf location '/user/impala/udfs/udf-examples-cdh570.jar' symbol='org.apache.impala.TestUdf'; [localhost:21000] > show functions; +-------------+---------------------------------------+-------------+---------------+ | return type | signature | binary type | is persistent | @@ -407,12 +407,12 @@ Found 2 items | TINYINT | testudf(TINYINT) | JAVA | true | +-------------+---------------------------------------+-------------+---------------+ </codeblock> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> These are all simple functions that return their single arguments, or sum, concatenate, and so on their multiple arguments. Impala determines which overloaded function to use based on the number and types of the arguments. </p> -<codeblock rev="2.5.0 IMPALA-2843 CDH-39148"> +<codeblock rev="2.5.0 IMPALA-2843"> insert into bigint_x values (1), (2), (4), (3); select testudf(x) from bigint_x; +-----------------+ @@ -455,16 +455,16 @@ select testudf(x,x) from string_x; +--------------------+ </codeblock> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> The previous example used the same Impala function name as the name of the class. This example shows how the Impala function name is independent of the underlying Java class or function names. A second <codeph>CREATE FUNCTION</codeph> statement results in a set of overloaded functions all named <codeph>my_func</codeph>, to go along with the overloaded functions all named <codeph>testudf</codeph>. </p> -<codeblock rev="2.5.0 IMPALA-2843 CDH-39148"> +<codeblock rev="2.5.0 IMPALA-2843"> create function my_func location '/user/impala/udfs/udf-examples-cdh570.jar' - symbol='com.cloudera.impala.TestUdf'; + symbol='org.apache.impala.TestUdf'; show functions; +-------------+---------------------------------------+-------------+---------------+ @@ -479,11 +479,11 @@ show functions; | BOOLEAN | testudf(BOOLEAN, BOOLEAN) | JAVA | true | ... </codeblock> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> The corresponding <codeph>DROP FUNCTION</codeph> statement with no signature drops all the overloaded functions with that name. </p> -<codeblock rev="2.5.0 IMPALA-2843 CDH-39148"> +<codeblock rev="2.5.0 IMPALA-2843"> drop function my_func; show functions; +-------------+---------------------------------------+-------------+---------------+ @@ -494,7 +494,7 @@ show functions; | BOOLEAN | testudf(BOOLEAN, BOOLEAN) | JAVA | true | ... </codeblock> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> The signatureless <codeph>CREATE FUNCTION</codeph> syntax for Java UDFs ensures that the functions shown in this example remain available after the Impala service (specifically, the Catalog Server) are restarted. http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/topics/impala_decimal.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_decimal.xml b/docs/topics/impala_decimal.xml index 80e8130..d5c9d70 100644 --- a/docs/topics/impala_decimal.xml +++ b/docs/topics/impala_decimal.xml @@ -798,10 +798,6 @@ SELECT x+y, ROUND(y,1), z/98.6 FROM t1; SELECT CAST(1000.5 AS DECIMAL); </codeblock> - <p conref="../shared/impala_common.xml#common/restrictions_blurb"/> - - <p conref="../shared/impala_common.xml#common/decimal_no_stats"/> - <!-- <p conref="../shared/impala_common.xml#common/partitioning_good"/> --> <p conref="../shared/impala_common.xml#common/hbase_ok"/> http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/topics/impala_describe.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_describe.xml b/docs/topics/impala_describe.xml index 1f7ce9b..67af443 100644 --- a/docs/topics/impala_describe.xml +++ b/docs/topics/impala_describe.xml @@ -577,7 +577,7 @@ Query finished, fetching results ... | | NULL | NULL | | # Detailed Table Information | NULL | NULL | | Database: | describe_formatted | NULL | -| Owner: | cloudera | NULL | +| Owner: | doc_demo | NULL | | CreateTime: | Mon Jul 22 17:03:16 EDT 2013 | NULL | | LastAccessTime: | UNKNOWN | NULL | | Protect Mode: | None | NULL | @@ -625,7 +625,7 @@ Query finished, fetching results ... | | NULL | NULL | | # Detailed Table Information | NULL | NULL | | Database: | describe_formatted | NULL | -| Owner: | cloudera | NULL | +| Owner: | doc_demo | NULL | | CreateTime: | Mon Jul 22 16:56:38 EDT 2013 | NULL | | LastAccessTime: | UNKNOWN | NULL | | Protect Mode: | None | NULL | @@ -648,7 +648,7 @@ Query finished, fetching results ... | View Expanded Text: | SELECT x, upper(s) FROM t1 | NULL | +------------------------------+------------------------------+----------------------+ Returned 28 row(s) in 0.03s -[localhost:21000] > create external table t2 (x int, y int, s string) stored as parquet location '/user/cloudera/sample_data'; +[localhost:21000] > create external table t2 (x int, y int, s string) stored as parquet location '/user/doc_demo/sample_data'; [localhost:21000] > describe formatted t2; Query: describe formatted t2 Query finished, fetching results ... @@ -663,12 +663,12 @@ Query finished, fetching results ... | | NULL | NULL | | # Detailed Table Information | NULL | NULL | | Database: | describe_formatted | NULL | -| Owner: | cloudera | NULL | +| Owner: | doc_demo | NULL | | CreateTime: | Mon Jul 22 17:01:47 EDT 2013 | NULL | | LastAccessTime: | UNKNOWN | NULL | | Protect Mode: | None | NULL | | Retention: | 0 | NULL | -| Location: | hdfs://127.0.0.1:8020/user/cloudera/sample_data | NULL | +| Location: | hdfs://127.0.0.1:8020/user/doc_demo/sample_data | NULL | | Table Type: | EXTERNAL_TABLE | NULL | | Table Parameters: | NULL | NULL | | | EXTERNAL | TRUE | @@ -676,8 +676,8 @@ Query finished, fetching results ... | | NULL | NULL | | # Storage Information | NULL | NULL | | SerDe Library: | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL | -| InputFormat: | com.cloudera.impala.hive.serde.ParquetInputFormat | NULL | -| OutputFormat: | com.cloudera.impala.hive.serde.ParquetOutputFormat | NULL | +| InputFormat: | org.apache.impala.hive.serde.ParquetInputFormat | NULL | +| OutputFormat: | org.apache.impala.hive.serde.ParquetOutputFormat | NULL | | Compressed: | No | NULL | | Num Buckets: | 0 | NULL | | Bucket Columns: | [] | NULL | @@ -688,7 +688,7 @@ Returned 27 row(s) in 0.17s</codeblock> <p conref="../shared/impala_common.xml#common/cancel_blurb_no"/> <p conref="../shared/impala_common.xml#common/permissions_blurb"/> - <p rev="CDH-19187"> + <p rev=""> The user ID that the <cmdname>impalad</cmdname> daemon runs under, typically the <codeph>impala</codeph> user, must have read and execute permissions for all directories that are part of the table. http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/18927ac8/docs/topics/impala_drop_function.xml ---------------------------------------------------------------------- diff --git a/docs/topics/impala_drop_function.xml b/docs/topics/impala_drop_function.xml index 6627211..71f3a32 100644 --- a/docs/topics/impala_drop_function.xml +++ b/docs/topics/impala_drop_function.xml @@ -51,8 +51,8 @@ under the License. <codeblock>DROP [AGGREGATE] FUNCTION [IF EXISTS] [<varname>db_name</varname>.]<varname>function_name</varname>(<varname>type</varname>[, <varname>type</varname>...])</codeblock> - <note rev="2.5.0 IMPALA-2843 CDH-39148"> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <note rev="2.5.0 IMPALA-2843"> + <p rev="2.5.0 IMPALA-2843"> The preceding syntax, which includes the function signature, also applies to Java UDFs that were created using the corresponding <codeph>CREATE FUNCTION</codeph> syntax that includes the argument and return types. After upgrading to <keyword keyref="impala25_full"/> or higher, consider re-creating all Java UDFs with the @@ -61,7 +61,7 @@ under the License. </p> </note> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> To drop Java UDFs (created using the <codeph>CREATE FUNCTION</codeph> syntax with no function signature): </p> @@ -91,7 +91,7 @@ DROP FUNCTION IF EXISTS bar; <p conref="../shared/impala_common.xml#common/cancel_blurb_no"/> <p conref="../shared/impala_common.xml#common/permissions_blurb"/> - <p rev="CDH-19187"> + <p rev=""> The user ID that the <cmdname>impalad</cmdname> daemon runs under, typically the <codeph>impala</codeph> user, does not need any particular HDFS permissions to perform this statement. @@ -100,7 +100,7 @@ DROP FUNCTION IF EXISTS bar; </p> <p conref="../shared/impala_common.xml#common/example_blurb"/> - <p rev="2.5.0 IMPALA-2843 CDH-39148"> + <p rev="2.5.0 IMPALA-2843"> The following example shows how to drop Java functions created with the signatureless <codeph>CREATE FUNCTION</codeph> syntax in <keyword keyref="impala25_full"/> and higher. Issuing <codeph>DROP FUNCTION <varname>function_name</varname></codeph> removes all the @@ -108,9 +108,9 @@ DROP FUNCTION IF EXISTS bar; (See <xref href="impala_create_function.xml#create_function"/> for a longer example showing how to set up such functions in the first place.) </p> -<codeblock rev="2.5.0 IMPALA-2843 CDH-39148"> -create function my_func location '/user/impala/udfs/udf-examples-cdh570.jar' - symbol='com.cloudera.impala.TestUdf'; +<codeblock rev="2.5.0 IMPALA-2843"> +create function my_func location '/user/impala/udfs/udf-examples.jar' + symbol='org.apache.impala.TestUdf'; show functions; +-------------+---------------------------------------+-------------+---------------+
