http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_components.html ---------------------------------------------------------------------- diff --git a/docs/build3x/html/topics/impala_components.html b/docs/build3x/html/topics/impala_components.html new file mode 100644 index 0000000..eb6e0f6 --- /dev/null +++ b/docs/build3x/html/topics/impala_components.html @@ -0,0 +1,227 @@ +<!DOCTYPE html + SYSTEM "about:legacy-compat"> +<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_concepts.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="intro_components"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Components of the Impala Server</title></head><body id="intro_components"><main role="main"><article role="article" aria-labelledby="ariaid-title1"> + + <h1 class="title topictitle1" id="ariaid-title1">Components of the Impala Server</h1> + + + + <div class="body conbody"> + + <p class="p"> + The Impala server is a distributed, massively parallel processing (MPP) database engine. It consists of + different daemon processes that run on specific hosts within your <span class="keyword"></span> cluster. + </p> + + <p class="p toc inpage"></p> + </div> + + <nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_concepts.html">Impala Concepts and Architecture</a></div></div></nav><article class="topic concept nested1" aria-labelledby="ariaid-title2" id="intro_components__intro_impalad"> + + <h2 class="title topictitle2" id="ariaid-title2">The Impala Daemon</h2> + + <div class="body conbody"> + + <p class="p"> + The core Impala component is a daemon process that runs on each DataNode of the cluster, physically represented + by the <code class="ph codeph">impalad</code> process. It reads and writes to data files; accepts queries transmitted + from the <code class="ph codeph">impala-shell</code> command, Hue, JDBC, or ODBC; parallelizes the queries and + distributes work across the cluster; and transmits intermediate query results back to the + central coordinator node. + </p> + + <p class="p"> + You can submit a query to the Impala daemon running on any DataNode, and that instance of the daemon serves as the + <dfn class="term">coordinator node</dfn> for that query. The other nodes transmit partial results back to the + coordinator, which constructs the final result set for a query. When running experiments with functionality + through the <code class="ph codeph">impala-shell</code> command, you might always connect to the same Impala daemon for + convenience. For clusters running production workloads, you might load-balance by + submitting each query to a different Impala daemon in round-robin style, using the JDBC or ODBC interfaces. + </p> + + <p class="p"> + The Impala daemons are in constant communication with the <dfn class="term">statestore</dfn>, to confirm which nodes + are healthy and can accept new work. + </p> + + <p class="p"> + They also receive broadcast messages from the <span class="keyword cmdname">catalogd</span> daemon (introduced in Impala 1.2) + whenever any Impala node in the cluster creates, alters, or drops any type of object, or when an + <code class="ph codeph">INSERT</code> or <code class="ph codeph">LOAD DATA</code> statement is processed through Impala. This + background communication minimizes the need for <code class="ph codeph">REFRESH</code> or <code class="ph codeph">INVALIDATE + METADATA</code> statements that were needed to coordinate metadata across nodes prior to Impala 1.2. + </p> + + <p class="p"> + In <span class="keyword">Impala 2.9</span> and higher, you can control which hosts act as query coordinators + and which act as query executors, to improve scalability for highly concurrent workloads on large clusters. + See <a class="xref" href="impala_scalability.html">Scalability Considerations for Impala</a> for details. + </p> + + <p class="p"> + <strong class="ph b">Related information:</strong> <a class="xref" href="impala_config_options.html#config_options">Modifying Impala Startup Options</a>, + <a class="xref" href="impala_processes.html#processes">Starting Impala</a>, <a class="xref" href="impala_timeouts.html#impalad_timeout">Setting the Idle Query and Idle Session Timeouts for impalad</a>, + <a class="xref" href="impala_ports.html#ports">Ports Used by Impala</a>, <a class="xref" href="impala_proxy.html#proxy">Using Impala through a Proxy for High Availability</a> + </p> + </div> + </article> + + <article class="topic concept nested1" aria-labelledby="ariaid-title3" id="intro_components__intro_statestore"> + + <h2 class="title topictitle2" id="ariaid-title3">The Impala Statestore</h2> + + <div class="body conbody"> + + <p class="p"> + The Impala component known as the <dfn class="term">statestore</dfn> checks on the health of Impala daemons on all the + DataNodes in a cluster, and continuously relays its findings to each of those daemons. It is physically + represented by a daemon process named <code class="ph codeph">statestored</code>; you only need such a process on one + host in the cluster. If an Impala daemon goes offline due to hardware failure, network error, software issue, + or other reason, the statestore informs all the other Impala daemons so that future queries can avoid making + requests to the unreachable node. + </p> + + <p class="p"> + Because the statestore's purpose is to help when things go wrong, it is not critical to the normal + operation of an Impala cluster. If the statestore is not running or becomes unreachable, the Impala daemons + continue running and distributing work among themselves as usual; the cluster just becomes less robust if + other Impala daemons fail while the statestore is offline. When the statestore comes back online, it re-establishes + communication with the Impala daemons and resumes its monitoring function. + </p> + + <p class="p"> + Most considerations for load balancing and high availability apply to the <span class="keyword cmdname">impalad</span> daemon. + The <span class="keyword cmdname">statestored</span> and <span class="keyword cmdname">catalogd</span> daemons do not have special + requirements for high availability, because problems with those daemons do not result in data loss. + If those daemons become unavailable due to an outage on a particular + host, you can stop the Impala service, delete the <span class="ph uicontrol">Impala StateStore</span> and + <span class="ph uicontrol">Impala Catalog Server</span> roles, add the roles on a different host, and restart the + Impala service. + </p> + + <p class="p"> + <strong class="ph b">Related information:</strong> + </p> + + <p class="p"> + <a class="xref" href="impala_scalability.html#statestore_scalability">Scalability Considerations for the Impala Statestore</a>, + <a class="xref" href="impala_config_options.html#config_options">Modifying Impala Startup Options</a>, <a class="xref" href="impala_processes.html#processes">Starting Impala</a>, + <a class="xref" href="impala_timeouts.html#statestore_timeout">Increasing the Statestore Timeout</a>, <a class="xref" href="impala_ports.html#ports">Ports Used by Impala</a> + </p> + </div> + </article> + + <article class="topic concept nested1" aria-labelledby="ariaid-title4" id="intro_components__intro_catalogd"> + + <h2 class="title topictitle2" id="ariaid-title4">The Impala Catalog Service</h2> + + <div class="body conbody"> + + <p class="p"> + The Impala component known as the <dfn class="term">catalog service</dfn> relays the metadata changes from Impala SQL + statements to all the Impala daemons in a cluster. It is physically represented by a daemon process named + <code class="ph codeph">catalogd</code>; you only need such a process on one host in the cluster. Because the requests + are passed through the statestore daemon, it makes sense to run the <span class="keyword cmdname">statestored</span> and + <span class="keyword cmdname">catalogd</span> services on the same host. + </p> + + <p class="p"> + The catalog service avoids the need to issue + <code class="ph codeph">REFRESH</code> and <code class="ph codeph">INVALIDATE METADATA</code> statements when the metadata changes are + performed by statements issued through Impala. When you create a table, load data, and so on through Hive, + you do need to issue <code class="ph codeph">REFRESH</code> or <code class="ph codeph">INVALIDATE METADATA</code> on an Impala node + before executing a query there. + </p> + + <p class="p"> + This feature touches a number of aspects of Impala: + </p> + + + + <ul class="ul" id="intro_catalogd__catalogd_xrefs"> + <li class="li"> + <p class="p"> + See <a class="xref" href="impala_install.html#install">Installing Impala</a>, <a class="xref" href="impala_upgrading.html#upgrading">Upgrading Impala</a> and + <a class="xref" href="impala_processes.html#processes">Starting Impala</a>, for usage information for the + <span class="keyword cmdname">catalogd</span> daemon. + </p> + </li> + + <li class="li"> + <p class="p"> + The <code class="ph codeph">REFRESH</code> and <code class="ph codeph">INVALIDATE METADATA</code> statements are not needed + when the <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">INSERT</code>, or other table-changing or + data-changing operation is performed through Impala. These statements are still needed if such + operations are done through Hive or by manipulating data files directly in HDFS, but in those cases the + statements only need to be issued on one Impala node rather than on all nodes. See + <a class="xref" href="impala_refresh.html#refresh">REFRESH Statement</a> and + <a class="xref" href="impala_invalidate_metadata.html#invalidate_metadata">INVALIDATE METADATA Statement</a> for the latest usage information for + those statements. + </p> + </li> + </ul> + + <div class="p"> + Use <code class="ph codeph">--load_catalog_in_background</code> option to control when + the metadata of a table is loaded. + <ul class="ul"> + <li class="li"> + If set to <code class="ph codeph">false</code>, the metadata of a table is + loaded when it is referenced for the first time. This means that the + first run of a particular query can be slower than subsequent runs. + Starting in Impala 2.2, the default for + <code class="ph codeph">load_catalog_in_background</code> is + <code class="ph codeph">false</code>. + </li> + <li class="li"> + If set to <code class="ph codeph">true</code>, the catalog service attempts to + load metadata for a table even if no query needed that metadata. So + metadata will possibly be already loaded when the first query that + would need it is run. However, for the following reasons, we + recommend not to set the option to <code class="ph codeph">true</code>. + <ul class="ul"> + <li class="li"> + Background load can interfere with query-specific metadata + loading. This can happen on startup or after invalidating + metadata, with a duration depending on the amount of metadata, + and can lead to a seemingly random long running queries that are + difficult to diagnose. + </li> + <li class="li"> + Impala may load metadata for tables that are possibly never + used, potentially increasing catalog size and consequently memory + usage for both catalog service and Impala Daemon. + </li> + </ul> + </li> + </ul> + </div> + + <p class="p"> + Most considerations for load balancing and high availability apply to the <span class="keyword cmdname">impalad</span> daemon. + The <span class="keyword cmdname">statestored</span> and <span class="keyword cmdname">catalogd</span> daemons do not have special + requirements for high availability, because problems with those daemons do not result in data loss. + If those daemons become unavailable due to an outage on a particular + host, you can stop the Impala service, delete the <span class="ph uicontrol">Impala StateStore</span> and + <span class="ph uicontrol">Impala Catalog Server</span> roles, add the roles on a different host, and restart the + Impala service. + </p> + + <div class="note note note_note"><span class="note__title notetitle">Note:</span> + <p class="p"> + In Impala 1.2.4 and higher, you can specify a table name with <code class="ph codeph">INVALIDATE METADATA</code> after + the table is created in Hive, allowing you to make individual tables visible to Impala without doing a full + reload of the catalog metadata. Impala 1.2.4 also includes other changes to make the metadata broadcast + mechanism faster and more responsive, especially during Impala startup. See + <a class="xref" href="../shared/../topics/impala_new_features.html#new_features_124">New Features in Impala 1.2.4</a> for details. + </p> + </div> + + <p class="p"> + <strong class="ph b">Related information:</strong> <a class="xref" href="impala_config_options.html#config_options">Modifying Impala Startup Options</a>, + <a class="xref" href="impala_processes.html#processes">Starting Impala</a>, <a class="xref" href="impala_ports.html#ports">Ports Used by Impala</a> + </p> + </div> + </article> +</article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_compression_codec.html ---------------------------------------------------------------------- diff --git a/docs/build3x/html/topics/impala_compression_codec.html b/docs/build3x/html/topics/impala_compression_codec.html new file mode 100644 index 0000000..5933efa --- /dev/null +++ b/docs/build3x/html/topics/impala_compression_codec.html @@ -0,0 +1,92 @@ +<!DOCTYPE html + SYSTEM "about:legacy-compat"> +<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="compression_codec"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>COMPRESSION_CODEC Query Option (Impala 2.0 or higher only)</title></head><body id="compression_codec"><main role="main"><article role="article" aria-labelledby="ariaid-title1"> + + <h1 class="title topictitle1" id="ariaid-title1">COMPRESSION_CODEC Query Option (<span class="keyword">Impala 2.0</span> or higher only)</h1> + + + + <div class="body conbody"> + + + + + + <p class="p"> + + When Impala writes Parquet data files using the <code class="ph codeph">INSERT</code> statement, the underlying compression + is controlled by the <code class="ph codeph">COMPRESSION_CODEC</code> query option. + </p> + + <div class="note note note_note"><span class="note__title notetitle">Note:</span> + Prior to Impala 2.0, this option was named <code class="ph codeph">PARQUET_COMPRESSION_CODEC</code>. In Impala 2.0 and + later, the <code class="ph codeph">PARQUET_COMPRESSION_CODEC</code> name is not recognized. Use the more general name + <code class="ph codeph">COMPRESSION_CODEC</code> for new code. + </div> + + <p class="p"> + <strong class="ph b">Syntax:</strong> + </p> + +<pre class="pre codeblock"><code>SET COMPRESSION_CODEC=<var class="keyword varname">codec_name</var>;</code></pre> + + <p class="p"> + The allowed values for this query option are <code class="ph codeph">SNAPPY</code> (the default), <code class="ph codeph">GZIP</code>, + and <code class="ph codeph">NONE</code>. + </p> + + <div class="note note note_note"><span class="note__title notetitle">Note:</span> + A Parquet file created with <code class="ph codeph">COMPRESSION_CODEC=NONE</code> is still typically smaller than the + original data, due to encoding schemes such as run-length encoding and dictionary encoding that are applied + separately from compression. + </div> + + <p class="p"></p> + + <p class="p"> + The option value is not case-sensitive. + </p> + + <p class="p"> + If the option is set to an unrecognized value, all kinds of queries will fail due to the invalid option + setting, not just queries involving Parquet tables. (The value <code class="ph codeph">BZIP2</code> is also recognized, but + is not compatible with Parquet tables.) + </p> + + <p class="p"> + <strong class="ph b">Type:</strong> <code class="ph codeph">STRING</code> + </p> + + <p class="p"> + <strong class="ph b">Default:</strong> <code class="ph codeph">SNAPPY</code> + </p> + + + <p class="p"> + <strong class="ph b">Examples:</strong> + </p> + +<pre class="pre codeblock"><code>set compression_codec=gzip; +insert into parquet_table_highly_compressed select * from t1; + +set compression_codec=snappy; +insert into parquet_table_compression_plus_fast_queries select * from t1; + +set compression_codec=none; +insert into parquet_table_no_compression select * from t1; + +set compression_codec=foo; +select * from t1 limit 5; +ERROR: Invalid compression codec: foo +</code></pre> + + <p class="p"> + <strong class="ph b">Related information:</strong> + </p> + + <p class="p"> + For information about how compressing Parquet data files affects query performance, see + <a class="xref" href="impala_parquet.html#parquet_compression">Snappy and GZip Compression for Parquet Data Files</a>. + </p> + </div> +<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html> http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_compute_stats.html ---------------------------------------------------------------------- diff --git a/docs/build3x/html/topics/impala_compute_stats.html b/docs/build3x/html/topics/impala_compute_stats.html new file mode 100644 index 0000000..407ba97 --- /dev/null +++ b/docs/build3x/html/topics/impala_compute_stats.html @@ -0,0 +1,637 @@ +<!DOCTYPE html + SYSTEM "about:legacy-compat"> +<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="compute_stats"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>COMPUTE STATS Statement</title></head><body id="compute_stats"><main role="main"><article role="article" aria-labelledby="ariaid-title1"> + + <h1 class="title topictitle1" id="ariaid-title1">COMPUTE STATS Statement</h1> + + + + <div class="body conbody"> + + <p class="p"> + The + COMPUTE STATS statement gathers information about volume and distribution + of data in a table and all associated columns and partitions. The + information is stored in the metastore database, and used by Impala to + help optimize queries. For example, if Impala can determine that a table + is large or small, or has many or few distinct values it can organize and + parallelize the work appropriately for a join query or insert operation. + For details about the kinds of information gathered by this statement, see + <a class="xref" href="impala_perf_stats.html#perf_stats">Table and Column Statistics</a>. + </p> + + <p class="p"> + <strong class="ph b">Syntax:</strong> + </p> + +<pre class="pre codeblock"><code><span class="ph">COMPUTE STATS [<var class="keyword varname">db_name</var>.]<var class="keyword varname">table_name</var> [ ( <var class="keyword varname">column_list</var> ) ] [TABLESAMPLE SYSTEM(<var class="keyword varname">percentage</var>) [REPEATABLE(<var class="keyword varname">seed</var>)]]</span> + +<var class="keyword varname">column_list</var> ::= <var class="keyword varname">column_name</var> [ , <var class="keyword varname">column_name</var>, ... ] + +COMPUTE INCREMENTAL STATS [<var class="keyword varname">db_name</var>.]<var class="keyword varname">table_name</var> [PARTITION (<var class="keyword varname">partition_spec</var>)] + +<var class="keyword varname">partition_spec</var> ::= <var class="keyword varname">simple_partition_spec</var> | <span class="ph"><var class="keyword varname">complex_partition_spec</var></span> + +<var class="keyword varname">simple_partition_spec</var> ::= <var class="keyword varname">partition_col</var>=<var class="keyword varname">constant_value</var> + +<span class="ph"><var class="keyword varname">complex_partition_spec</var> ::= <var class="keyword varname">comparison_expression_on_partition_col</var></span> +</code></pre> + + <p class="p"> + The <code class="ph codeph">PARTITION</code> clause is only allowed in combination with the <code class="ph codeph">INCREMENTAL</code> + clause. It is optional for <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>, and required for <code class="ph codeph">DROP + INCREMENTAL STATS</code>. Whenever you specify partitions through the <code class="ph codeph">PARTITION + (<var class="keyword varname">partition_spec</var>)</code> clause in a <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> or + <code class="ph codeph">DROP INCREMENTAL STATS</code> statement, you must include all the partitioning columns in the + specification, and specify constant values for all the partition key columns. + </p> + + <p class="p"> + <strong class="ph b">Usage notes:</strong> + </p> + + <p class="p"> + Originally, Impala relied on users to run the Hive <code class="ph codeph">ANALYZE + TABLE</code> statement, but that method of gathering statistics proved + unreliable and difficult to use. The Impala <code class="ph codeph">COMPUTE STATS</code> + statement was built to improve the reliability and user-friendliness of + this operation. <code class="ph codeph">COMPUTE STATS</code> does not require any setup + steps or special configuration. You only run a single Impala + <code class="ph codeph">COMPUTE STATS</code> statement to gather both table and column + statistics, rather than separate Hive <code class="ph codeph">ANALYZE TABLE</code> + statements for each kind of statistics. + </p> + + <p class="p"> + For non-incremental <code class="ph codeph">COMPUTE STATS</code> + statement, the columns for which statistics are computed can be specified + with an optional comma-separate list of columns. + </p> + + <p class="p"> + If no column list is given, the <code class="ph codeph">COMPUTE STATS</code> statement + computes column-level statistics for all columns of the table. This adds + potentially unneeded work for columns whose stats are not needed by + queries. It can be especially costly for very wide tables and unneeded + large string fields. + </p> + <p class="p"> + <code class="ph codeph">COMPUTE STATS</code> returns an error when a specified column + cannot be analyzed, such as when the column does not exist, the column is + of an unsupported type for COMPUTE STATS, e.g. colums of complex types, + or the column is a partitioning column. + + </p> + <p class="p"> + If an empty column list is given, no column is analyzed by <code class="ph codeph">COMPUTE + STATS</code>. + </p> + + <p class="p"> + In <span class="keyword">Impala 2.12</span> and + higher, an optional <code class="ph codeph">TABLESAMPLE</code> clause immediately after + a table reference specifies that the <code class="ph codeph">COMPUTE STATS</code> + operation only processes a specified percentage of the table data. For + tables that are so large that a full <code class="ph codeph">COMPUTE STATS</code> + operation is impractical, you can use <code class="ph codeph">COMPUTE STATS</code> with + a <code class="ph codeph">TABLESAMPLE</code> clause to extrapolate statistics from a + sample of the table data. See <a href="impala_perf_stats.html"><span class="keyword">Table and Column Statistics</span></a>about the + experimental stats extrapolation and sampling features. + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> variation is a shortcut for partitioned tables that works on a + subset of partitions rather than the entire table. The incremental nature makes it suitable for large tables + with many partitions, where a full <code class="ph codeph">COMPUTE STATS</code> operation takes too long to be practical + each time a partition is added or dropped. See <a class="xref" href="impala_perf_stats.html#perf_stats_incremental">impala_perf_stats.html#perf_stats_incremental</a> + for full usage details. + </p> + + <div class="note important note_important"><span class="note__title importanttitle">Important:</span> + <p class="p"> + For a particular table, use either <code class="ph codeph">COMPUTE STATS</code> or + <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>, but never combine the two or + alternate between them. If you switch from <code class="ph codeph">COMPUTE STATS</code> to + <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> during the lifetime of a table, or + vice versa, drop all statistics by running <code class="ph codeph">DROP STATS</code> before + making the switch. + </p> + <p class="p"> + When you run <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> on a table for the first time, + the statistics are computed again from scratch regardless of whether the table already + has statistics. Therefore, expect a one-time resource-intensive operation + for scanning the entire table when running <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> + for the first time on a given table. + </p> + <p class="p"> + For a table with a huge number of partitions and many columns, the approximately 400 bytes + of metadata per column per partition can add up to significant memory overhead, as it must + be cached on the <span class="keyword cmdname">catalogd</span> host and on every <span class="keyword cmdname">impalad</span> host + that is eligible to be a coordinator. If this metadata for all tables combined exceeds 2 GB, + you might experience service downtime. + </p> + </div> + + <p class="p"> + <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> only applies to partitioned tables. If you use the + <code class="ph codeph">INCREMENTAL</code> clause for an unpartitioned table, Impala automatically uses the original + <code class="ph codeph">COMPUTE STATS</code> statement. Such tables display <code class="ph codeph">false</code> under the + <code class="ph codeph">Incremental stats</code> column of the <code class="ph codeph">SHOW TABLE STATS</code> output. + </p> + <div class="note note note_note"><span class="note__title notetitle">Note:</span> + <div class="p"> + Because many of the most performance-critical and resource-intensive + operations rely on table and column statistics to construct accurate and + efficient plans, <code class="ph codeph">COMPUTE STATS</code> is an important step at + the end of your ETL process. Run <code class="ph codeph">COMPUTE STATS</code> on all + tables as your first step during performance tuning for slow queries, or + troubleshooting for out-of-memory conditions: + <ul class="ul"> + <li class="li"> + Accurate statistics help Impala construct an efficient query plan + for join queries, improving performance and reducing memory usage. + </li> + <li class="li"> + Accurate statistics help Impala distribute the work effectively + for insert operations into Parquet tables, improving performance and + reducing memory usage. + </li> + <li class="li"> + Accurate statistics help Impala estimate the memory + required for each query, which is important when you use resource + management features, such as admission control and the YARN resource + management framework. The statistics help Impala to achieve high + concurrency, full utilization of available memory, and avoid + contention with workloads from other Hadoop components. + </li> + <li class="li"> + In <span class="keyword">Impala 2.8</span> and + higher, when you run the <code class="ph codeph">COMPUTE STATS</code> or + <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> statement against a + Parquet table, Impala automatically applies the query option setting + <code class="ph codeph">MT_DOP=4</code> to increase the amount of intra-node + parallelism during this CPU-intensive operation. See <a class="xref" href="impala_mt_dop.html">MT_DOP Query Option</a> for details about what this query option does + and how to use it with CPU-intensive <code class="ph codeph">SELECT</code> + statements. + </li> + </ul> + </div> + </div> + + <p class="p"> + <strong class="ph b">Computing stats for groups of partitions:</strong> + </p> + + <p class="p"> + In <span class="keyword">Impala 2.8</span> and higher, you can run <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> + on multiple partitions, instead of the entire table or one partition at a time. You include + comparison operators other than <code class="ph codeph">=</code> in the <code class="ph codeph">PARTITION</code> clause, + and the <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> statement applies to all partitions that + match the comparison expression. + </p> + + <p class="p"> + For example, the <code class="ph codeph">INT_PARTITIONS</code> table contains 4 partitions. + The following <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> statements affect some but not all + partitions, as indicated by the <code class="ph codeph">Updated <var class="keyword varname">n</var> partition(s)</code> + messages. The partitions that are affected depend on values in the partition key column <code class="ph codeph">X</code> + that match the comparison expression in the <code class="ph codeph">PARTITION</code> clause. + </p> + +<pre class="pre codeblock"><code> +show partitions int_partitions; ++-------+-------+--------+------+--------------+-------------------+---------+... +| x | #Rows | #Files | Size | Bytes Cached | Cache Replication | Format |... ++-------+-------+--------+------+--------------+-------------------+---------+... +| 99 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | PARQUET |... +| 120 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT |... +| 150 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT |... +| 200 | -1 | 0 | 0B | NOT CACHED | NOT CACHED | TEXT |... +| Total | -1 | 0 | 0B | 0B | | |... ++-------+-------+--------+------+--------------+-------------------+---------+... + +compute incremental stats int_partitions partition (x < 100); ++-----------------------------------------+ +| summary | ++-----------------------------------------+ +| Updated 1 partition(s) and 1 column(s). | ++-----------------------------------------+ + +compute incremental stats int_partitions partition (x in (100, 150, 200)); ++-----------------------------------------+ +| summary | ++-----------------------------------------+ +| Updated 2 partition(s) and 1 column(s). | ++-----------------------------------------+ + +compute incremental stats int_partitions partition (x between 100 and 175); ++-----------------------------------------+ +| summary | ++-----------------------------------------+ +| Updated 2 partition(s) and 1 column(s). | ++-----------------------------------------+ + +compute incremental stats int_partitions partition (x in (100, 150, 200) or x < 100); ++-----------------------------------------+ +| summary | ++-----------------------------------------+ +| Updated 3 partition(s) and 1 column(s). | ++-----------------------------------------+ + +compute incremental stats int_partitions partition (x != 150); ++-----------------------------------------+ +| summary | ++-----------------------------------------+ +| Updated 3 partition(s) and 1 column(s). | ++-----------------------------------------+ + +</code></pre> + + <p class="p"> + <strong class="ph b">Complex type considerations:</strong> + </p> + + <p class="p"> + Currently, the statistics created by the <code class="ph codeph">COMPUTE STATS</code> statement do not include + information about complex type columns. The column stats metrics for complex columns are always shown + as -1. For queries involving complex type columns, Impala uses + heuristics to estimate the data distribution within such columns. + </p> + + <p class="p"> + <strong class="ph b">HBase considerations:</strong> + </p> + + <p class="p"> + <code class="ph codeph">COMPUTE STATS</code> works for HBase tables also. The statistics gathered for HBase tables are + somewhat different than for HDFS-backed tables, but that metadata is still used for optimization when HBase + tables are involved in join queries. + </p> + + <p class="p"> + <strong class="ph b">Amazon S3 considerations:</strong> + </p> + + <p class="p"> + <code class="ph codeph">COMPUTE STATS</code> also works for tables where data resides in the Amazon Simple Storage Service (S3). + See <a class="xref" href="impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details. + </p> + + <p class="p"> + <strong class="ph b">Performance considerations:</strong> + </p> + + <p class="p"> + The statistics collected by <code class="ph codeph">COMPUTE STATS</code> are used to optimize join queries + <code class="ph codeph">INSERT</code> operations into Parquet tables, and other resource-intensive kinds of SQL statements. + See <a class="xref" href="impala_perf_stats.html#perf_stats">Table and Column Statistics</a> for details. + </p> + + <p class="p"> + For large tables, the <code class="ph codeph">COMPUTE STATS</code> statement itself might take a long time and you + might need to tune its performance. The <code class="ph codeph">COMPUTE STATS</code> statement does not work with the + <code class="ph codeph">EXPLAIN</code> statement, or the <code class="ph codeph">SUMMARY</code> command in <span class="keyword cmdname">impala-shell</span>. + You can use the <code class="ph codeph">PROFILE</code> statement in <span class="keyword cmdname">impala-shell</span> to examine timing information + for the statement as a whole. If a basic <code class="ph codeph">COMPUTE STATS</code> statement takes a long time for a + partitioned table, consider switching to the <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> syntax so that only + newly added partitions are analyzed each time. + </p> + + <p class="p"> + <strong class="ph b">Examples:</strong> + </p> + + <p class="p"> + This example shows two tables, <code class="ph codeph">T1</code> and <code class="ph codeph">T2</code>, with a small number distinct + values linked by a parent-child relationship between <code class="ph codeph">T1.ID</code> and <code class="ph codeph">T2.PARENT</code>. + <code class="ph codeph">T1</code> is tiny, while <code class="ph codeph">T2</code> has approximately 100K rows. Initially, the statistics + includes physical measurements such as the number of files, the total size, and size measurements for + fixed-length columns such as with the <code class="ph codeph">INT</code> type. Unknown values are represented by -1. After + running <code class="ph codeph">COMPUTE STATS</code> for each table, much more information is available through the + <code class="ph codeph">SHOW STATS</code> statements. If you were running a join query involving both of these tables, you + would need statistics for both tables to get the most effective optimization for the query. + </p> + + + +<pre class="pre codeblock"><code>[localhost:21000] > show table stats t1; +Query: show table stats t1 ++-------+--------+------+--------+ +| #Rows | #Files | Size | Format | ++-------+--------+------+--------+ +| -1 | 1 | 33B | TEXT | ++-------+--------+------+--------+ +Returned 1 row(s) in 0.02s +[localhost:21000] > show table stats t2; +Query: show table stats t2 ++-------+--------+----------+--------+ +| #Rows | #Files | Size | Format | ++-------+--------+----------+--------+ +| -1 | 28 | 960.00KB | TEXT | ++-------+--------+----------+--------+ +Returned 1 row(s) in 0.01s +[localhost:21000] > show column stats t1; +Query: show column stats t1 ++--------+--------+------------------+--------+----------+----------+ +| Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size | ++--------+--------+------------------+--------+----------+----------+ +| id | INT | -1 | -1 | 4 | 4 | +| s | STRING | -1 | -1 | -1 | -1 | ++--------+--------+------------------+--------+----------+----------+ +Returned 2 row(s) in 1.71s +[localhost:21000] > show column stats t2; +Query: show column stats t2 ++--------+--------+------------------+--------+----------+----------+ +| Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size | ++--------+--------+------------------+--------+----------+----------+ +| parent | INT | -1 | -1 | 4 | 4 | +| s | STRING | -1 | -1 | -1 | -1 | ++--------+--------+------------------+--------+----------+----------+ +Returned 2 row(s) in 0.01s +[localhost:21000] > compute stats t1; +Query: compute stats t1 ++-----------------------------------------+ +| summary | ++-----------------------------------------+ +| Updated 1 partition(s) and 2 column(s). | ++-----------------------------------------+ +Returned 1 row(s) in 5.30s +[localhost:21000] > show table stats t1; +Query: show table stats t1 ++-------+--------+------+--------+ +| #Rows | #Files | Size | Format | ++-------+--------+------+--------+ +| 3 | 1 | 33B | TEXT | ++-------+--------+------+--------+ +Returned 1 row(s) in 0.01s +[localhost:21000] > show column stats t1; +Query: show column stats t1 ++--------+--------+------------------+--------+----------+----------+ +| Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size | ++--------+--------+------------------+--------+----------+----------+ +| id | INT | 3 | -1 | 4 | 4 | +| s | STRING | 3 | -1 | -1 | -1 | ++--------+--------+------------------+--------+----------+----------+ +Returned 2 row(s) in 0.02s +[localhost:21000] > compute stats t2; +Query: compute stats t2 ++-----------------------------------------+ +| summary | ++-----------------------------------------+ +| Updated 1 partition(s) and 2 column(s). | ++-----------------------------------------+ +Returned 1 row(s) in 5.70s +[localhost:21000] > show table stats t2; +Query: show table stats t2 ++-------+--------+----------+--------+ +| #Rows | #Files | Size | Format | ++-------+--------+----------+--------+ +| 98304 | 1 | 960.00KB | TEXT | ++-------+--------+----------+--------+ +Returned 1 row(s) in 0.03s +[localhost:21000] > show column stats t2; +Query: show column stats t2 ++--------+--------+------------------+--------+----------+----------+ +| Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size | ++--------+--------+------------------+--------+----------+----------+ +| parent | INT | 3 | -1 | 4 | 4 | +| s | STRING | 6 | -1 | 14 | 9.3 | ++--------+--------+------------------+--------+----------+----------+ +Returned 2 row(s) in 0.01s</code></pre> + + <p class="p"> + The following example shows how to use the <code class="ph codeph">INCREMENTAL</code> clause, available in Impala 2.1.0 and + higher. The <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> syntax lets you collect statistics for newly added or + changed partitions, without rescanning the entire table. + </p> + +<pre class="pre codeblock"><code>-- Initially the table has no incremental stats, as indicated +-- 'false' under Incremental stats. +show table stats item_partitioned; ++-------------+-------+--------+----------+--------------+---------+------------------ +| i_category | #Rows | #Files | Size | Bytes Cached | Format | Incremental stats ++-------------+-------+--------+----------+--------------+---------+------------------ +| Books | -1 | 1 | 223.74KB | NOT CACHED | PARQUET | false +| Children | -1 | 1 | 230.05KB | NOT CACHED | PARQUET | false +| Electronics | -1 | 1 | 232.67KB | NOT CACHED | PARQUET | false +| Home | -1 | 1 | 232.56KB | NOT CACHED | PARQUET | false +| Jewelry | -1 | 1 | 223.72KB | NOT CACHED | PARQUET | false +| Men | -1 | 1 | 231.25KB | NOT CACHED | PARQUET | false +| Music | -1 | 1 | 237.90KB | NOT CACHED | PARQUET | false +| Shoes | -1 | 1 | 234.90KB | NOT CACHED | PARQUET | false +| Sports | -1 | 1 | 227.97KB | NOT CACHED | PARQUET | false +| Women | -1 | 1 | 226.27KB | NOT CACHED | PARQUET | false +| Total | -1 | 10 | 2.25MB | 0B | | ++-------------+-------+--------+----------+--------------+---------+------------------ + +-- After the first COMPUTE INCREMENTAL STATS, +-- all partitions have stats. The first +-- COMPUTE INCREMENTAL STATS scans the whole +-- table, discarding any previous stats from +-- a traditional COMPUTE STATS statement. +compute incremental stats item_partitioned; ++-------------------------------------------+ +| summary | ++-------------------------------------------+ +| Updated 10 partition(s) and 21 column(s). | ++-------------------------------------------+ +show table stats item_partitioned; ++-------------+-------+--------+----------+--------------+---------+------------------ +| i_category | #Rows | #Files | Size | Bytes Cached | Format | Incremental stats ++-------------+-------+--------+----------+--------------+---------+------------------ +| Books | 1733 | 1 | 223.74KB | NOT CACHED | PARQUET | true +| Children | 1786 | 1 | 230.05KB | NOT CACHED | PARQUET | true +| Electronics | 1812 | 1 | 232.67KB | NOT CACHED | PARQUET | true +| Home | 1807 | 1 | 232.56KB | NOT CACHED | PARQUET | true +| Jewelry | 1740 | 1 | 223.72KB | NOT CACHED | PARQUET | true +| Men | 1811 | 1 | 231.25KB | NOT CACHED | PARQUET | true +| Music | 1860 | 1 | 237.90KB | NOT CACHED | PARQUET | true +| Shoes | 1835 | 1 | 234.90KB | NOT CACHED | PARQUET | true +| Sports | 1783 | 1 | 227.97KB | NOT CACHED | PARQUET | true +| Women | 1790 | 1 | 226.27KB | NOT CACHED | PARQUET | true +| Total | 17957 | 10 | 2.25MB | 0B | | ++-------------+-------+--------+----------+--------------+---------+------------------ + +-- Add a new partition... +alter table item_partitioned add partition (i_category='Camping'); +-- Add or replace files in HDFS outside of Impala, +-- rendering the stats for a partition obsolete. +!import_data_into_sports_partition.sh +refresh item_partitioned; +drop incremental stats item_partitioned partition (i_category='Sports'); +-- Now some partitions have incremental stats +-- and some do not. +show table stats item_partitioned; ++-------------+-------+--------+----------+--------------+---------+------------------ +| i_category | #Rows | #Files | Size | Bytes Cached | Format | Incremental stats ++-------------+-------+--------+----------+--------------+---------+------------------ +| Books | 1733 | 1 | 223.74KB | NOT CACHED | PARQUET | true +| Camping | -1 | 1 | 408.02KB | NOT CACHED | PARQUET | false +| Children | 1786 | 1 | 230.05KB | NOT CACHED | PARQUET | true +| Electronics | 1812 | 1 | 232.67KB | NOT CACHED | PARQUET | true +| Home | 1807 | 1 | 232.56KB | NOT CACHED | PARQUET | true +| Jewelry | 1740 | 1 | 223.72KB | NOT CACHED | PARQUET | true +| Men | 1811 | 1 | 231.25KB | NOT CACHED | PARQUET | true +| Music | 1860 | 1 | 237.90KB | NOT CACHED | PARQUET | true +| Shoes | 1835 | 1 | 234.90KB | NOT CACHED | PARQUET | true +| Sports | -1 | 1 | 227.97KB | NOT CACHED | PARQUET | false +| Women | 1790 | 1 | 226.27KB | NOT CACHED | PARQUET | true +| Total | 17957 | 11 | 2.65MB | 0B | | ++-------------+-------+--------+----------+--------------+---------+------------------ + +-- After another COMPUTE INCREMENTAL STATS, +-- all partitions have incremental stats, and only the 2 +-- partitions without incremental stats were scanned. +compute incremental stats item_partitioned; ++------------------------------------------+ +| summary | ++------------------------------------------+ +| Updated 2 partition(s) and 21 column(s). | ++------------------------------------------+ +show table stats item_partitioned; ++-------------+-------+--------+----------+--------------+---------+------------------ +| i_category | #Rows | #Files | Size | Bytes Cached | Format | Incremental stats ++-------------+-------+--------+----------+--------------+---------+------------------ +| Books | 1733 | 1 | 223.74KB | NOT CACHED | PARQUET | true +| Camping | 5328 | 1 | 408.02KB | NOT CACHED | PARQUET | true +| Children | 1786 | 1 | 230.05KB | NOT CACHED | PARQUET | true +| Electronics | 1812 | 1 | 232.67KB | NOT CACHED | PARQUET | true +| Home | 1807 | 1 | 232.56KB | NOT CACHED | PARQUET | true +| Jewelry | 1740 | 1 | 223.72KB | NOT CACHED | PARQUET | true +| Men | 1811 | 1 | 231.25KB | NOT CACHED | PARQUET | true +| Music | 1860 | 1 | 237.90KB | NOT CACHED | PARQUET | true +| Shoes | 1835 | 1 | 234.90KB | NOT CACHED | PARQUET | true +| Sports | 1783 | 1 | 227.97KB | NOT CACHED | PARQUET | true +| Women | 1790 | 1 | 226.27KB | NOT CACHED | PARQUET | true +| Total | 17957 | 11 | 2.65MB | 0B | | ++-------------+-------+--------+----------+--------------+---------+------------------ +</code></pre> + + <p class="p"> + <strong class="ph b">File format considerations:</strong> + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement works with tables created with any of the file formats supported + by Impala. See <a class="xref" href="impala_file_formats.html#file_formats">How Impala Works with Hadoop File Formats</a> for details about working with the + different file formats. The following considerations apply to <code class="ph codeph">COMPUTE STATS</code> depending on the + file format of the table. + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement works with text tables with no restrictions. These tables can be + created through either Impala or Hive. + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement works with Parquet tables. These tables can be created through + either Impala or Hive. + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement works with Avro tables without restriction in <span class="keyword">Impala 2.2</span> + and higher. In earlier releases, <code class="ph codeph">COMPUTE STATS</code> worked only for Avro tables created through Hive, + and required the <code class="ph codeph">CREATE TABLE</code> statement to use SQL-style column names and types rather than an + Avro-style schema specification. + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement works with RCFile tables with no restrictions. These tables can + be created through either Impala or Hive. + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement works with SequenceFile tables with no restrictions. These + tables can be created through either Impala or Hive. + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement works with partitioned tables, whether all the partitions use + the same file format, or some partitions are defined through <code class="ph codeph">ALTER TABLE</code> to use different + file formats. + </p> + + <p class="p"> + <strong class="ph b">Statement type:</strong> DDL + </p> + + <p class="p"> + <strong class="ph b">Cancellation:</strong> Certain multi-stage statements (<code class="ph codeph">CREATE TABLE AS SELECT</code> and + <code class="ph codeph">COMPUTE STATS</code>) can be cancelled during some stages, when running <code class="ph codeph">INSERT</code> + or <code class="ph codeph">SELECT</code> operations internally. To cancel this statement, use Ctrl-C from the + <span class="keyword cmdname">impala-shell</span> interpreter, the <span class="ph uicontrol">Cancel</span> button from the + <span class="ph uicontrol">Watch</span> page in Hue, or <span class="ph uicontrol">Cancel</span> from the list of + in-flight queries (for a particular node) on the <span class="ph uicontrol">Queries</span> tab in the Impala web UI + (port 25000). + </p> + + <p class="p"> + <strong class="ph b">Restrictions:</strong> + </p> + + <div class="note note note_note"><span class="note__title notetitle">Note:</span> Prior to Impala 1.4.0, + <code class="ph codeph">COMPUTE STATS</code> counted the number of + <code class="ph codeph">NULL</code> values in each column and recorded that figure + in the metastore database. Because Impala does not currently use the + <code class="ph codeph">NULL</code> count during query planning, Impala 1.4.0 and + higher speeds up the <code class="ph codeph">COMPUTE STATS</code> statement by + skipping this <code class="ph codeph">NULL</code> counting. </div> + + <p class="p"> + <strong class="ph b">Internal details:</strong> + </p> + <p class="p"> + Behind the scenes, the <code class="ph codeph">COMPUTE STATS</code> statement + executes two statements: one to count the rows of each partition + in the table (or the entire table if unpartitioned) through the + <code class="ph codeph">COUNT(*)</code> function, + and another to count the approximate number of distinct values + in each column through the <code class="ph codeph">NDV()</code> function. + You might see these queries in your monitoring and diagnostic displays. + The same factors that affect the performance, scalability, and + execution of other queries (such as parallel execution, memory usage, + admission control, and timeouts) also apply to the queries run by the + <code class="ph codeph">COMPUTE STATS</code> statement. + </p> + + <p class="p"> + <strong class="ph b">HDFS permissions:</strong> + </p> + <p class="p"> + The user ID that the <span class="keyword cmdname">impalad</span> daemon runs under, + typically the <code class="ph codeph">impala</code> user, must have read + permission for all affected files in the source directory: + all files in the case of an unpartitioned table or + a partitioned table in the case of <code class="ph codeph">COMPUTE STATS</code>; + or all the files in partitions without incremental stats in + the case of <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>. + It must also have read and execute permissions for all + relevant directories holding the data files. + (Essentially, <code class="ph codeph">COMPUTE STATS</code> requires the + same permissions as the underlying <code class="ph codeph">SELECT</code> queries it runs + against the table.) + </p> + + <p class="p"> + <strong class="ph b">Kudu considerations:</strong> + </p> + + <p class="p"> + The <code class="ph codeph">COMPUTE STATS</code> statement applies to Kudu tables. + Impala does not compute the number of rows for each partition for + Kudu tables. Therefore, you do not need to re-run the operation when + you see -1 in the <code class="ph codeph"># Rows</code> column of the output from + <code class="ph codeph">SHOW TABLE STATS</code>. That column always shows -1 for + all Kudu tables. + </p> + + <p class="p"> + <strong class="ph b">Related information:</strong> + </p> + + <p class="p"> + <a class="xref" href="impala_drop_stats.html#drop_stats">DROP STATS Statement</a>, <a class="xref" href="impala_show.html#show_table_stats">SHOW TABLE STATS Statement</a>, + <a class="xref" href="impala_show.html#show_column_stats">SHOW COLUMN STATS Statement</a>, <a class="xref" href="impala_perf_stats.html#perf_stats">Table and Column Statistics</a> + </p> + </div> +<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav></article></main></body></html> http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_compute_stats_min_sample_size.html ---------------------------------------------------------------------- diff --git a/docs/build3x/html/topics/impala_compute_stats_min_sample_size.html b/docs/build3x/html/topics/impala_compute_stats_min_sample_size.html new file mode 100644 index 0000000..03d21e2 --- /dev/null +++ b/docs/build3x/html/topics/impala_compute_stats_min_sample_size.html @@ -0,0 +1,23 @@ +<!DOCTYPE html + SYSTEM "about:legacy-compat"> +<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="compute_stats_sample_min_sample_size"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>COMPUTE_STATS_MIN_SAMPLE_SIZE Query Option</title></head><body id="compute_stats_sample_min_sample_size"><main role="main"><article role="article" aria-labelledby="ariaid-title1"> + <h1 class="title topictitle1" id="ariaid-title1">COMPUTE_STATS_MIN_SAMPLE_SIZE Query Option</h1> + + + <div class="body conbody"> + <p class="p">The <code class="ph codeph">COMPUTE_STATS_MIN_SAMPLE_SIZE</code> query option specifies + the minimum number of bytes that will be scanned in <code class="ph codeph">COMPUTE STATS + TABLESAMPLE</code>, regardless of the user-supplied sampling percent. + This query option prevents sampling for very small tables where accurate + stats can be obtained cheaply without sampling because the minimum sample + size is required to get meaningful stats.</p> + <p class="p"> + <strong class="ph b">Type:</strong> integer + </p> + <p class="p"><strong class="ph b">Default:</strong> 1GB</p> + <p class="p"><strong class="ph b">Added in</strong>: <span class="keyword">Impala 2.12</span></p> + <p class="p"> + <strong class="ph b">Usage notes:</strong> + </p> + </div> +<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html> http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_concepts.html ---------------------------------------------------------------------- diff --git a/docs/build3x/html/topics/impala_concepts.html b/docs/build3x/html/topics/impala_concepts.html new file mode 100644 index 0000000..b98e4ce --- /dev/null +++ b/docs/build3x/html/topics/impala_concepts.html @@ -0,0 +1,48 @@ +<!DOCTYPE html + SYSTEM "about:legacy-compat"> +<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_components.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_development.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_hadoop.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="concepts"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Concepts and Architecture</title></head><body id="concepts"><main role="main"><article role="article" aria-labelledby="ariaid-title1"> + + <h1 class="title topictitle1" id="ariaid-title1">Impala Concepts and Architecture</h1> + + + + <div class="body conbody"> + + <p class="p"> + The following sections provide background information to help you become productive using Impala and + its features. Where appropriate, the explanations include context to help understand how aspects of Impala + relate to other technologies you might already be familiar with, such as relational database management + systems and data warehouses, or other Hadoop components such as Hive, HDFS, and HBase. + </p> + + <p class="p toc"></p> + </div> + + + + + + + + + + + + + + + + + + + + + + + + + + + + +<nav role="navigation" class="related-links"><ul class="ullinks"><li class="link ulchildlink"><strong><a href="../topics/impala_components.html">Components of the Impala Server</a></strong><br></li><li class="link ulchildlink"><strong><a href="../topics/impala_development.html">Developing Impala Applications</a></strong><br></li><li class="link ulchildlink"><strong><a href="../topics/impala_hadoop.html">How Impala Fits Into the Hadoop Ecosystem</a></strong><br></li></ul></nav></article></main></body></html>
