http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/75c46918/docs/build/html/topics/impala_shell_options.html ---------------------------------------------------------------------- diff --git a/docs/build/html/topics/impala_shell_options.html b/docs/build/html/topics/impala_shell_options.html new file mode 100644 index 0000000..be21f0b --- /dev/null +++ b/docs/build/html/topics/impala_shell_options.html @@ -0,0 +1,564 @@ +<!DOCTYPE html + SYSTEM "about:legacy-compat"> +<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2017"><meta name="DC.rights.owner" content="(C) Copyright 2017"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_impala_shell.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="shell_options"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>impala-shell Configuration Options</title></head><body id="shell_options"><main role="main"><article role="article" aria-labelledby="ariaid-title1"> + + <h1 class="title topictitle1" id="ariaid-title1">impala-shell Configuration Options</h1> + + + + <div class="body conbody"> + + <p class="p"> + You can specify the following options when starting the <code class="ph codeph">impala-shell</code> command to change how + shell commands are executed. The table shows the format to use when specifying each option on the command + line, or through the <span class="ph filepath">$HOME/.impalarc</span> configuration file. + </p> + + <div class="note note note_note"><span class="note__title notetitle">Note:</span> + <p class="p"> + These options are different than the configuration options for the <code class="ph codeph">impalad</code> daemon itself. + For the <code class="ph codeph">impalad</code> options, see <a class="xref" href="impala_config_options.html#config_options">Modifying Impala Startup Options</a>. + </p> + </div> + + <p class="p toc inpage"></p> + </div> + + <nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_impala_shell.html">Using the Impala Shell (impala-shell Command)</a></div></div></nav><article class="topic concept nested1" aria-labelledby="ariaid-title2" id="shell_options__shell_option_summary"> + + <h2 class="title topictitle2" id="ariaid-title2">Summary of impala-shell Configuration Options</h2> + + <div class="body conbody"> + + <p class="p"> + The following table shows the names and allowed arguments for the <span class="keyword cmdname">impala-shell</span> + configuration options. You can specify options on the command line, or in a configuration file as described + in <a class="xref" href="impala_shell_options.html#shell_config_file">impala-shell Configuration File</a>. + </p> + + <table class="table"><caption></caption><colgroup><col style="width:25%"><col style="width:25%"><col style="width:50%"></colgroup><thead class="thead"> + <tr class="row"> + <th class="entry nocellnorowborder" id="shell_option_summary__entry__1"> + Command-Line Option + </th> + <th class="entry nocellnorowborder" id="shell_option_summary__entry__2"> + Configuration File Setting + </th> + <th class="entry nocellnorowborder" id="shell_option_summary__entry__3"> + Explanation + </th> + </tr> + </thead><tbody class="tbody"> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -B or --delimited + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + write_delimited=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Causes all query results to be printed in plain format as a delimited text file. Useful for + producing data files to be used with other Hadoop components. Also useful for avoiding the + performance overhead of pretty-printing all output, especially when running benchmark tests using + queries returning large result sets. Specify the delimiter character with the + <code class="ph codeph">--output_delimiter</code> option. Store all query results in a file rather than + printing to the screen with the <code class="ph codeph">-B</code> option. Added in Impala 1.0.1. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + --print_header + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + print_header=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"></p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -o <var class="keyword varname">filename</var> or --output_file <var class="keyword varname">filename</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + output_file=<var class="keyword varname">filename</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Stores all query results in the specified file. Typically used to store the results of a single + query issued from the command line with the <code class="ph codeph">-q</code> option. Also works for + interactive sessions; you see the messages such as number of rows fetched, but not the actual + result set. To suppress these incidental messages when combining the <code class="ph codeph">-q</code> and + <code class="ph codeph">-o</code> options, redirect <code class="ph codeph">stderr</code> to <code class="ph codeph">/dev/null</code>. + Added in Impala 1.0.1. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + --output_delimiter=<var class="keyword varname">character</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + output_delimiter=<var class="keyword varname">character</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Specifies the character to use as a delimiter between fields when query results are printed in + plain format by the <code class="ph codeph">-B</code> option. Defaults to tab (<code class="ph codeph">'\t'</code>). If an + output value contains the delimiter character, that field is quoted, escaped by doubling quotation marks, or both. Added in + Impala 1.0.1. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -p or --show_profiles + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + show_profiles=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Displays the query execution plan (same output as the <code class="ph codeph">EXPLAIN</code> statement) and a + more detailed low-level breakdown of execution steps, for every query executed by the shell. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -h or --help + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + N/A + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Displays help information. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -i <var class="keyword varname">hostname</var> or + --impalad=<var class="keyword varname">hostname</var>[:<var class="keyword varname">portnum</var>] + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + impalad=<var class="keyword varname">hostname</var>[:<var class="keyword varname">portnum</var>] + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Connects to the <code class="ph codeph">impalad</code> daemon on the specified host. The default port of 21000 + is assumed unless you provide another value. You can connect to any host in your cluster that is + running <code class="ph codeph">impalad</code>. If you connect to an instance of <code class="ph codeph">impalad</code> that + was started with an alternate port specified by the <code class="ph codeph">--fe_port</code> flag, provide that + alternative port. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -q <var class="keyword varname">query</var> or --query=<var class="keyword varname">query</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + query=<var class="keyword varname">query</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Passes a query or other <span class="keyword cmdname">impala-shell</span> command from the command line. The + <span class="keyword cmdname">impala-shell</span> interpreter immediately exits after processing the statement. It + is limited to a single statement, which could be a <code class="ph codeph">SELECT</code>, <code class="ph codeph">CREATE + TABLE</code>, <code class="ph codeph">SHOW TABLES</code>, or any other statement recognized in + <code class="ph codeph">impala-shell</code>. Because you cannot pass a <code class="ph codeph">USE</code> statement and + another query, fully qualify the names for any tables outside the <code class="ph codeph">default</code> + database. (Or use the <code class="ph codeph">-f</code> option to pass a file with a <code class="ph codeph">USE</code> + statement followed by other queries.) + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -f <var class="keyword varname">query_file</var> or --query_file=<var class="keyword varname">query_file</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + query_file=<var class="keyword varname">path_to_query_file</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Passes a SQL query from a file. Multiple statements must be semicolon (;) delimited. + <span class="ph">In <span class="keyword">Impala 2.3</span> and higher, you can specify a filename of <code class="ph codeph">-</code> + to represent standard input. This feature makes it convenient to use <span class="keyword cmdname">impala-shell</span> + as part of a Unix pipeline where SQL statements are generated dynamically by other tools.</span> + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -k or --kerberos + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + use_kerberos=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Kerberos authentication is used when the shell connects to <code class="ph codeph">impalad</code>. If Kerberos + is not enabled on the instance of <code class="ph codeph">impalad</code> to which you are connecting, errors + are displayed. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -s <var class="keyword varname">kerberos_service_name</var> or --kerberos_service_name=<var class="keyword varname">name</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + kerberos_service_name=<var class="keyword varname">name</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Instructs <code class="ph codeph">impala-shell</code> to authenticate to a particular <code class="ph codeph">impalad</code> + service principal. If a <var class="keyword varname">kerberos_service_name</var> is not specified, + <code class="ph codeph">impala</code> is used by default. If this option is used in conjunction with a + connection in which Kerberos is not supported, errors are returned. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -V or --verbose + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + verbose=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Enables verbose output. + </p> + </td> + </tr> + <tr class="row"> + + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + --quiet + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + verbose=false + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Disables verbose output. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -v or --version + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + version=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Displays version information. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -c + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + ignore_query_failure=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Continues on query failure. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -r or --refresh_after_connect + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + refresh_after_connect=true + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Updates Impala metadata upon connection. Same as running the + <code class="ph codeph"><a class="xref" href="impala_invalidate_metadata.html#invalidate_metadata">INVALIDATE + METADATA</a></code> statement after connecting. (This option was originally named when the + <code class="ph codeph">REFRESH</code> statement did the extensive metadata updates now performed by + <code class="ph codeph">INVALIDATE METADATA</code>.) + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + <p class="p"> + -d <var class="keyword varname">default_db</var> or --database=<var class="keyword varname">default_db</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + <p class="p"> + default_db=<var class="keyword varname">default_db</var> + </p> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + <p class="p"> + Specifies the database to be used on startup. Same as running the + <code class="ph codeph"><a class="xref" href="impala_use.html#use">USE</a></code> statement after connecting. If not + specified, a database named <code class="ph codeph">DEFAULT</code> is used. + </p> + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + -ssl + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + ssl=true + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + Enables TLS/SSL for <span class="keyword cmdname">impala-shell</span>. + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + --ca_cert=<var class="keyword varname">path_to_certificate</var> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + ca_cert=<var class="keyword varname">path_to_certificate</var> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + The local pathname pointing to the third-party CA certificate, or to a copy of the server + certificate for self-signed server certificates. If <code class="ph codeph">--ca_cert</code> is not set, + <span class="keyword cmdname">impala-shell</span> enables TLS/SSL, but does not validate the server certificate. This is + useful for connecting to a known-good Impala that is only running over TLS/SSL, when a copy of the + certificate is not available (such as when debugging customer installations). + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + -l + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + use_ldap=true + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + Enables LDAP authentication. + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + -u + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + user=<var class="keyword varname">user_name</var> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + Supplies the username, when LDAP authentication is enabled by the <code class="ph codeph">-l</code> option. + (Specify the short username, not the full LDAP distinguished name.) The shell then prompts + interactively for the password. + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + --ldap_password_cmd=<var class="keyword varname">command</var> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + N/A + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + Specifies a command to run to retrieve the LDAP password, + when LDAP authentication is enabled by the <code class="ph codeph">-l</code> option. + If the command includes space-separated arguments, enclose the command and + its arguments in quotation marks. + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 "> + --config_file=<var class="keyword varname">path_to_config_file</var> + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 "> + N/A + </td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + Specifies the path of the file containing <span class="keyword cmdname">impala-shell</span> configuration settings. + The default is <span class="ph filepath">$HOME/.impalarc</span>. This setting can only be specified on the + command line. + </td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 ">--live_progress</td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 ">N/A</td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 ">Prints a progress bar showing roughly the percentage complete for each query. + The information is updated interactively as the query progresses. + See <a class="xref" href="impala_live_progress.html#live_progress">LIVE_PROGRESS Query Option (Impala 2.3 or higher only)</a>.</td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 ">--live_summary</td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 ">N/A</td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 ">Prints a detailed report, similar to the <code class="ph codeph">SUMMARY</code> command, showing progress details for each phase of query execution. + The information is updated interactively as the query progresses. + See <a class="xref" href="impala_live_summary.html#live_summary">LIVE_SUMMARY Query Option (Impala 2.3 or higher only)</a>.</td> + </tr> + <tr class="row"> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__1 ">--var=<var class="keyword varname">variable_name</var>=<var class="keyword varname">value</var></td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__2 ">N/A</td> + <td class="entry nocellnorowborder" headers="shell_option_summary__entry__3 "> + Defines a substitution variable that can be used within the <span class="keyword cmdname">impala-shell</span> session. + The variable can be substituted into statements processed by the <code class="ph codeph">-q</code> or <code class="ph codeph">-f</code> options, + or in an interactive shell session. + Within a SQL statement, you substitute the value by using the notation <code class="ph codeph">${var:<var class="keyword varname">variable_name</var>}</code>. + This feature is available in <span class="keyword">Impala 2.5</span> and higher. + </td> + </tr> + </tbody></table> + </div> + </article> + + <article class="topic concept nested1" aria-labelledby="ariaid-title3" id="shell_options__shell_config_file"> + + <h2 class="title topictitle2" id="ariaid-title3">impala-shell Configuration File</h2> + + <div class="body conbody"> + + <p class="p"> + You can define a set of default options for your <span class="keyword cmdname">impala-shell</span> environment, stored in the + file <span class="ph filepath">$HOME/.impalarc</span>. This file consists of key-value pairs, one option per line. + Everything after a <code class="ph codeph">#</code> character on a line is treated as a comment and ignored. + </p> + + <p class="p"> + The configuration file must contain a header label <code class="ph codeph">[impala]</code>, followed by the options + specific to <span class="keyword cmdname">impala-shell</span>. (This standard convention for configuration files lets you + use a single file to hold configuration options for multiple applications.) + </p> + + <p class="p"> + To specify a different filename or path for the configuration file, specify the argument + <code class="ph codeph">--config_file=<var class="keyword varname">path_to_config_file</var></code> on the + <span class="keyword cmdname">impala-shell</span> command line. + </p> + + <p class="p"> + The names of the options in the configuration file are similar (although not necessarily identical) to the + long-form command-line arguments to the <span class="keyword cmdname">impala-shell</span> command. For the names to use, see + <a class="xref" href="impala_shell_options.html#shell_option_summary">Summary of impala-shell Configuration Options</a>. + </p> + + <p class="p"> + Any options you specify on the <span class="keyword cmdname">impala-shell</span> command line override any corresponding + options within the configuration file. + </p> + + <p class="p"> + The following example shows a configuration file that you might use during benchmarking tests. It sets + verbose mode, so that the output from each SQL query is followed by timing information. + <span class="keyword cmdname">impala-shell</span> starts inside the database containing the tables with the benchmark data, + avoiding the need to issue a <code class="ph codeph">USE</code> statement or use fully qualified table names. + </p> + + <p class="p"> + In this example, the query output is formatted as delimited text rather than enclosed in ASCII art boxes, + and is stored in a file rather than printed to the screen. Those options are appropriate for benchmark + situations, so that the overhead of <span class="keyword cmdname">impala-shell</span> formatting and printing the result set + does not factor into the timing measurements. It also enables the <code class="ph codeph">show_profiles</code> option. + That option prints detailed performance information after each query, which might be valuable in + understanding the performance of benchmark queries. + </p> + +<pre class="pre codeblock"><code>[impala] +verbose=true +default_db=tpc_benchmarking +write_delimited=true +output_delimiter=, +output_file=/home/tester1/benchmark_results.csv +show_profiles=true +</code></pre> + + <p class="p"> + The following example shows a configuration file that connects to a specific remote Impala node, runs a + single query within a particular database, then exits. You would typically use this kind of single-purpose + configuration setting with the <span class="keyword cmdname">impala-shell</span> command-line option + <code class="ph codeph">--config_file=<var class="keyword varname">path_to_config_file</var></code>, to easily select between many + predefined queries that could be run against different databases, hosts, or even different clusters. To run + a sequence of statements instead of a single query, specify the configuration option + <code class="ph codeph">query_file=<var class="keyword varname">path_to_query_file</var></code> instead. + </p> + +<pre class="pre codeblock"><code>[impala] +impalad=impala-test-node1.example.com +default_db=site_stats +# Issue a predefined query and immediately exit. +query=select count(*) from web_traffic where event_date = trunc(now(),'dd') +</code></pre> + </div> + </article> +</article></main></body></html> \ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/75c46918/docs/build/html/topics/impala_shell_running_commands.html ---------------------------------------------------------------------- diff --git a/docs/build/html/topics/impala_shell_running_commands.html b/docs/build/html/topics/impala_shell_running_commands.html new file mode 100644 index 0000000..e0e8880 --- /dev/null +++ b/docs/build/html/topics/impala_shell_running_commands.html @@ -0,0 +1,257 @@ +<!DOCTYPE html + SYSTEM "about:legacy-compat"> +<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2017"><meta name="DC.rights.owner" content="(C) Copyright 2017"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_impala_shell.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="shell_running_commands"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Running Commands and SQL Statements in impala-shell</title></head><body id="shell_running_commands"><main role="main"><article role="article" aria-labelledby="ariaid-title1"> + + <h1 class="title topictitle1" id="ariaid-title1">Running Commands and SQL Statements in impala-shell</h1> + + + + <div class="body conbody"> + + <p class="p"> + For information on available commands, see + <a class="xref" href="impala_shell_commands.html#shell_commands">impala-shell Command Reference</a>. You can see the full set of available + commands by pressing TAB twice, for example: + </p> + +<pre class="pre codeblock"><code>[impalad-host:21000] > +connect describe explain help history insert quit refresh select set shell show use version +[impalad-host:21000] ></code></pre> + + <div class="note note note_note"><span class="note__title notetitle">Note:</span> + Commands must be terminated by a semi-colon. A command can span multiple lines. + </div> + + <p class="p"> + For example: + </p> + +<pre class="pre codeblock"><code>[localhost:21000] > select * + > from t1 + > limit 5; ++---------+-----------+ +| s1 | s2 | ++---------+-----------+ +| hello | world | +| goodbye | cleveland | ++---------+-----------+ +</code></pre> + + <p class="p"> + A comment is considered part of the statement it precedes, so when you enter a <code class="ph codeph">--</code> or + <code class="ph codeph">/* */</code> comment, you get a continuation prompt until you finish entering a statement ending + with a semicolon: + </p> + +<pre class="pre codeblock"><code>[localhost:21000] > -- This is a test comment + > show tables like 't*'; ++--------+ +| name | ++--------+ +| t1 | +| t2 | +| tab1 | +| tab2 | +| tab3 | +| text_t | ++--------+ +</code></pre> + + <p class="p"> + Use the up-arrow and down-arrow keys to cycle through and edit previous commands. + <span class="keyword cmdname">impala-shell</span> uses the <code class="ph codeph">readline</code> library and so supports a standard set of + keyboard shortcuts for editing and cursor movement, such as <code class="ph codeph">Ctrl-A</code> for beginning of line and + <code class="ph codeph">Ctrl-E</code> for end of line. + </p> + + <p class="p"> + In <span class="keyword">Impala 2.5</span> and higher, you can define substitution variables to be used within SQL statements + processed by <span class="keyword cmdname">impala-shell</span>. On the command line, you specify the option + <code class="ph codeph">--var=<var class="keyword varname">variable_name</var>=<var class="keyword varname">value</var></code>. + Within an interactive session or a script file processed by the <code class="ph codeph">-f</code> option, you specify + a <code class="ph codeph">SET</code> command using the notation <code class="ph codeph">SET VAR:<var class="keyword varname">variable_name</var>=<var class="keyword varname">value</var></code>. + Within a SQL statement, you substitute the value by using the notation <code class="ph codeph">${var:<var class="keyword varname">variable_name</var>}</code>. + </p> + + <div class="note note note_note"><span class="note__title notetitle">Note:</span> + Because this feature is part of <span class="keyword cmdname">impala-shell</span> rather than the <span class="keyword cmdname">impalad</span> + backend, make sure the client system you are connecting from has the most recent <span class="keyword cmdname">impala-shell</span>. + You can use this feature with a new <span class="keyword cmdname">impala-shell</span> connecting to an older <span class="keyword cmdname">impalad</span>, + but not the reverse. + </div> + + <p class="p"> + For example, here are some <span class="keyword cmdname">impala-shell</span> commands that define substitution variables and then + use them in SQL statements executed through the <code class="ph codeph">-q</code> and <code class="ph codeph">-f</code> options. + Notice how the <code class="ph codeph">-q</code> argument strings are single-quoted to prevent shell expansion of the + <code class="ph codeph">${var:value}</code> notation, and any string literals within the queries are enclosed by double quotation marks. + </p> + +<pre class="pre codeblock"><code> +$ impala-shell --var=tname=table1 --var=colname=x --var=coltype=string -q 'create table ${var:tname} (${var:colname} ${var:coltype}) stored as parquet' +Starting Impala Shell without Kerberos authentication +Connected to <var class="keyword varname">hostname</var> +Server version: <var class="keyword varname">impalad_version</var> +Query: create table table1 (x string) stored as parquet + +$ NEW_STRING="hello world" +$ impala-shell --var=tname=table1 --var=insert_val="$NEW_STRING" -q 'insert into ${var:tname} values ("${var:insert_val}")' +Starting Impala Shell without Kerberos authentication +Connected to <var class="keyword varname">hostname</var> +Server version: <var class="keyword varname">impalad_version</var> +Query: insert into table1 values ("hello world") +Inserted 1 row(s) in 1.40s + +$ for VAL in foo bar bletch +do + impala-shell --var=tname=table1 --var=insert_val="$VAL" -q 'insert into ${var:tname} values ("${var:insert_val}")' +done +... +Query: insert into table1 values ("foo") +Inserted 1 row(s) in 0.22s +Query: insert into table1 values ("bar") +Inserted 1 row(s) in 0.11s +Query: insert into table1 values ("bletch") +Inserted 1 row(s) in 0.21s + +$ echo "Search for what substring?" ; read answer +Search for what substring? +b +$ impala-shell --var=tname=table1 -q 'select x from ${var:tname} where x like "%${var:answer}%"' +Starting Impala Shell without Kerberos authentication +Connected to <var class="keyword varname">hostname</var> +Server version: <var class="keyword varname">impalad_version</var> +Query: select x from table1 where x like "%b%" ++--------+ +| x | ++--------+ +| bletch | +| bar | ++--------+ +Fetched 2 row(s) in 0.83s +</code></pre> + + <p class="p"> + Here is a substitution variable passed in by the <code class="ph codeph">--var</code> option, + and then referenced by statements issued interactively. Then the variable is + cleared with the <code class="ph codeph">UNSET</code> command, and defined again with the + <code class="ph codeph">SET</code> command. + </p> + +<pre class="pre codeblock"><code> +$ impala-shell --quiet --var=tname=table1 +Starting Impala Shell without Kerberos authentication +*********************************************************************************** +<var class="keyword varname">banner_message</var> +*********************************************************************************** +[<var class="keyword varname">hostname</var>:21000] > select count(*) from ${var:tname}; ++----------+ +| count(*) | ++----------+ +| 4 | ++----------+ +[<var class="keyword varname">hostname</var>:21000] > unset var:tname; +Unsetting variable TNAME +[<var class="keyword varname">hostname</var>:21000] > select count(*) from ${var:tname}; +Error: Unknown variable TNAME +[<var class="keyword varname">hostname</var>:21000] > set var:tname=table1; +[<var class="keyword varname">hostname</var>:21000] > select count(*) from ${var:tname}; ++----------+ +| count(*) | ++----------+ +| 4 | ++----------+ +</code></pre> + + <p class="p"> + The following example shows how the <code class="ph codeph">SOURCE</code> command can execute + a series of statements from a file: + </p> + +<pre class="pre codeblock"><code> +$ cat commands.sql +show databases; +show tables in default; +show functions in _impala_builtins like '*minute*'; + +$ impala-shell -i localhost +... +[localhost:21000] > source commands.sql; +Query: show databases ++------------------+----------------------------------------------+ +| name | comment | ++------------------+----------------------------------------------+ +| _impala_builtins | System database for Impala builtin functions | +| default | Default Hive database | ++------------------+----------------------------------------------+ +Fetched 2 row(s) in 0.06s +Query: show tables in default ++-----------+ +| name | ++-----------+ +| customers | +| sample_07 | +| sample_08 | +| web_logs | ++-----------+ +Fetched 4 row(s) in 0.02s +Query: show functions in _impala_builtins like '*minute*' ++-------------+--------------------------------+-------------+---------------+ +| return type | signature | binary type | is persistent | ++-------------+--------------------------------+-------------+---------------+ +| INT | minute(TIMESTAMP) | BUILTIN | true | +| TIMESTAMP | minutes_add(TIMESTAMP, BIGINT) | BUILTIN | true | +| TIMESTAMP | minutes_add(TIMESTAMP, INT) | BUILTIN | true | +| TIMESTAMP | minutes_sub(TIMESTAMP, BIGINT) | BUILTIN | true | +| TIMESTAMP | minutes_sub(TIMESTAMP, INT) | BUILTIN | true | ++-------------+--------------------------------+-------------+---------------+ +Fetched 5 row(s) in 0.03s +</code></pre> + + <p class="p"> + The following example shows how a file that is run by the <code class="ph codeph">SOURCE</code> command, + or through the <code class="ph codeph">-q</code> or <code class="ph codeph">-f</code> options of <span class="keyword cmdname">impala-shell</span>, + can contain additional <code class="ph codeph">SOURCE</code> commands. + The first file, <span class="ph filepath">nested1.sql</span>, runs an <span class="keyword cmdname">impala-shell</span> command + and then also runs the commands from <span class="ph filepath">nested2.sql</span>. + This ability for scripts to call each other is often useful for code that sets up schemas for applications + or test environments. + </p> + +<pre class="pre codeblock"><code> +$ cat nested1.sql +show functions in _impala_builtins like '*minute*'; +source nested2.sql +$ cat nested2.sql +show functions in _impala_builtins like '*hour*' + +$ impala-shell -i localhost -f nested1.sql +Starting Impala Shell without Kerberos authentication +Connected to localhost:21000 +... +Query: show functions in _impala_builtins like '*minute*' ++-------------+--------------------------------+-------------+---------------+ +| return type | signature | binary type | is persistent | ++-------------+--------------------------------+-------------+---------------+ +| INT | minute(TIMESTAMP) | BUILTIN | true | +| TIMESTAMP | minutes_add(TIMESTAMP, BIGINT) | BUILTIN | true | +| TIMESTAMP | minutes_add(TIMESTAMP, INT) | BUILTIN | true | +| TIMESTAMP | minutes_sub(TIMESTAMP, BIGINT) | BUILTIN | true | +| TIMESTAMP | minutes_sub(TIMESTAMP, INT) | BUILTIN | true | ++-------------+--------------------------------+-------------+---------------+ +Fetched 5 row(s) in 0.01s +Query: show functions in _impala_builtins like '*hour*' ++-------------+------------------------------+-------------+---------------+ +| return type | signature | binary type | is persistent | ++-------------+------------------------------+-------------+---------------+ +| INT | hour(TIMESTAMP) | BUILTIN | true | +| TIMESTAMP | hours_add(TIMESTAMP, BIGINT) | BUILTIN | true | +| TIMESTAMP | hours_add(TIMESTAMP, INT) | BUILTIN | true | +| TIMESTAMP | hours_sub(TIMESTAMP, BIGINT) | BUILTIN | true | +| TIMESTAMP | hours_sub(TIMESTAMP, INT) | BUILTIN | true | ++-------------+------------------------------+-------------+---------------+ +Fetched 5 row(s) in 0.01s +</code></pre> + + </div> +<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_impala_shell.html">Using the Impala Shell (impala-shell Command)</a></div></div></nav></article></main></body></html> \ No newline at end of file
