Added: tajo/site/docs/0.11.1/tsql/single_command.html URL: http://svn.apache.org/viewvc/tajo/site/docs/0.11.1/tsql/single_command.html?rev=1728394&view=auto ============================================================================== --- tajo/site/docs/0.11.1/tsql/single_command.html (added) +++ tajo/site/docs/0.11.1/tsql/single_command.html Thu Feb 4 00:29:05 2016 @@ -0,0 +1,299 @@ + + +<!DOCTYPE html> +<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]--> +<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]--> +<head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + + <title>Executing a single command — Apache Tajo 0.11.0 documentation</title> + + + + + + + <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'> + + + + + + + + + + <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" /> + + + + <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/> + <link rel="up" title="Tajo Shell (TSQL)" href="../tsql.html"/> + <link rel="next" title="Executing Queries from Files" href="execute_file.html"/> + <link rel="prev" title="Introducing to TSQL" href="intro.html"/> + + + <script src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script> + +</head> + +<body class="wy-body-for-nav" role="document"> + + <div class="wy-grid-for-nav"> + + + <nav data-toggle="wy-nav-shift" class="wy-nav-side"> + <div class="wy-side-nav-search"> + <a href="../index.html" class="fa fa-home"> Apache Tajo</a> + <div role="search"> + <form id ="rtd-search-form" class="wy-form" action="../search.html" method="get"> + <input type="text" name="q" placeholder="Search docs" /> + <input type="hidden" name="check_keywords" value="yes" /> + <input type="hidden" name="area" value="default" /> + </form> +</div> + </div> + + <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> + + + <ul class="current"> +<li class="toctree-l1"><a class="reference internal" href="../introduction.html">Introduction</a></li> +<li class="toctree-l1"><a class="reference internal" href="../getting_started.html">Getting Started</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#prerequisites">Prerequisites</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#dowload-and-unpack-the-source-code">Dowload and unpack the source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#build-source-code">Build source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#setting-up-a-local-tajo-cluster">Setting up a local Tajo cluster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#first-query-execution">First query execution</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../configuration.html">Configuration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../configuration/preliminary.html">Preliminary</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/cluster_setup.html">Cluster Setup</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo_master_configuration.html">Tajo Master Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/worker_configuration.html">Worker Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog_configuration.html">Catalog Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/ha_configuration.html">High Availability for TajoMaster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> +</ul> +</li> +<li class="toctree-l1 current"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul class="current"> +<li class="toctree-l2"><a class="reference internal" href="meta_command.html">Meta Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="dfs_command.html">Executing HDFS commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="variables.html">Session Variables</a></li> +<li class="toctree-l2"><a class="reference internal" href="admin_command.html">Administration Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="intro.html">Introducing to TSQL</a></li> +<li class="toctree-l2 current"><a class="current reference internal" href="">Executing a single command</a></li> +<li class="toctree-l2"><a class="reference internal" href="execute_file.html">Executing Queries from Files</a></li> +<li class="toctree-l2"><a class="reference internal" href="background_command.html">Executing as background process</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../sql_language.html">SQL Language</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/data_model.html">Data Model</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/ddl.html">Data Definition Language</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/alter_table.html">ALTER TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/queries.html">Queries</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#server-cluster-time-zone">Server Cluster Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#table-time-zone">Table Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#client-time-zone">Client Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#time-zone-id">Time Zone ID</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../functions.html">Functions</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-scalar-functions">Built-in Scalar Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-aggregation-functions">Built-in Aggregation Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-window-functions">Built-in Window Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#user-defined-functions">User-defined Functions</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_partitioning.html">Table Partitioning</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/intro_to_partitioning.html">Introduction to Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/column_partitioning.html">Column Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/range_partitioning.html">Range Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/hash_partitioning.html">Hash Partitioning</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../storage_plugins.html">Storage Plugin</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/overview.html">Storage Plugin Overview</a></li> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/postgresql.html">PostgreSQL Storage Handler</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../index_overview.html">Index (Experimental Feature)</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../index/types.html">Index Types</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/how_to_use.html">How to use index?</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/future_work.html">Future Works</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../backup_and_restore.html">Backup and Restore</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../backup_and_restore/catalog.html">Backup and Restore Catalog</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../hive_integration.html">Hive Integration</a></li> +<li class="toctree-l1"><a class="reference internal" href="../hbase_integration.html">HBase Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#create-table">CREATE TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#drop-table">DROP TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#usage">Usage</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../swift_integration.html">OpenStack Swift Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#swift-configuration">Swift configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#hadoop-configurations">Hadoop configurations</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#tajo-configuration">Tajo configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#querying-on-swift">Querying on Swift</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../jdbc_driver.html">Tajo JDBC Driver</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC driver</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connecting-to-the-tajo-cluster-instance">Connecting to the Tajo cluster instance</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connection-parameters">Connection Parameters</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#an-example-jdbc-client">An Example JDBC Client</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../tajo_client_api.html">Tajo Client API</a></li> +<li class="toctree-l1"><a class="reference internal" href="../faq.html">FAQ</a></li> +</ul> + + + </div> + + </nav> + + <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> + + + <nav class="wy-nav-top" role="navigation" aria-label="top navigation"> + <i data-toggle="wy-nav-top" class="fa fa-bars"></i> + <a href="../index.html">Apache Tajo</a> + </nav> + + + + <div class="wy-nav-content"> + <div class="rst-content"> + <div role="navigation" aria-label="breadcrumbs navigation"> + <ul class="wy-breadcrumbs"> + <li><a href="../index.html">Docs</a> »</li> + + <li><a href="../tsql.html">Tajo Shell (TSQL)</a> »</li> + + <li>Executing a single command</li> + <li class="wy-breadcrumbs-aside"> + + <a href="../_sources/tsql/single_command.txt" rel="nofollow"> View page source</a> + + </li> + </ul> + <hr/> +</div> + <div role="main"> + + <div class="section" id="executing-a-single-command"> +<h1>Executing a single command<a class="headerlink" href="#executing-a-single-command" title="Permalink to this headline">¶</a></h1> +<p>You may want to run more queries without entering tsql prompt. Tsql provides the <code class="docutils literal"><span class="pre">-c</span></code> argument for above requirement. And Tajo assumes that queries are separated by semicolon as follows:</p> +<div class="highlight-sql"><div class="highlight"><pre>$ bin/tsql -c "select count(*) from table1; select sum(score) from table1;" +Progress: 0%, response time: 0.217 sec +Progress: 0%, response time: 0.218 sec +Progress: 100%, response time: 0.317 sec +?count +------------------------------- +5 +(1 rows, 0.317 sec, 2 B selected) +Progress: 0%, response time: 0.202 sec +Progress: 0%, response time: 0.204 sec +Progress: 100%, response time: 0.345 sec +?sum +------------------------------- +15.0 +(1 rows, 0.345 sec, 5 B selected) +</pre></div> +</div> +</div> + + + </div> + <footer> + + <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation"> + + <a href="execute_file.html" class="btn btn-neutral float-right" title="Executing Queries from Files"/>Next <span class="fa fa-arrow-circle-right"></span></a> + + + <a href="intro.html" class="btn btn-neutral" title="Introducing to TSQL"><span class="fa fa-arrow-circle-left"></span> Previous</a> + + </div> + + + <hr/> + + <div role="contentinfo"> + <p> + © Copyright 2015, Apache Tajo Team. + </p> + </div> + + <a href="https://github.com/snide/sphinx_rtd_theme";>Sphinx theme</a> provided by <a href="https://readthedocs.org";>Read the Docs</a> +</footer> + </div> + </div> + + </section> + + </div> + + + + + + <script type="text/javascript"> + var DOCUMENTATION_OPTIONS = { + URL_ROOT:'../', + VERSION:'0.11.0', + COLLAPSE_INDEX:false, + FILE_SUFFIX:'.html', + HAS_SOURCE: true + }; + </script> + <script type="text/javascript" src="../_static/jquery.js"></script> + <script type="text/javascript" src="../_static/underscore.js"></script> + <script type="text/javascript" src="../_static/doctools.js"></script> + + + + + + <script type="text/javascript" src="../_static/js/theme.js"></script> + + + + + <script type="text/javascript"> + jQuery(function () { + SphinxRtdTheme.StickyNav.enable(); + }); + </script> + + +</body> +</html> \ No newline at end of file
Added: tajo/site/docs/0.11.1/tsql/variables.html URL: http://svn.apache.org/viewvc/tajo/site/docs/0.11.1/tsql/variables.html?rev=1728394&view=auto ============================================================================== --- tajo/site/docs/0.11.1/tsql/variables.html (added) +++ tajo/site/docs/0.11.1/tsql/variables.html Thu Feb 4 00:29:05 2016 @@ -0,0 +1,912 @@ + + +<!DOCTYPE html> +<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]--> +<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]--> +<head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + + <title>Session Variables — Apache Tajo 0.11.0 documentation</title> + + + + + + + <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'> + + + + + + + + + + <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" /> + + + + <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/> + <link rel="up" title="Tajo Shell (TSQL)" href="../tsql.html"/> + <link rel="next" title="Administration Commands" href="admin_command.html"/> + <link rel="prev" title="Executing HDFS commands" href="dfs_command.html"/> + + + <script src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script> + +</head> + +<body class="wy-body-for-nav" role="document"> + + <div class="wy-grid-for-nav"> + + + <nav data-toggle="wy-nav-shift" class="wy-nav-side"> + <div class="wy-side-nav-search"> + <a href="../index.html" class="fa fa-home"> Apache Tajo</a> + <div role="search"> + <form id ="rtd-search-form" class="wy-form" action="../search.html" method="get"> + <input type="text" name="q" placeholder="Search docs" /> + <input type="hidden" name="check_keywords" value="yes" /> + <input type="hidden" name="area" value="default" /> + </form> +</div> + </div> + + <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> + + + <ul class="current"> +<li class="toctree-l1"><a class="reference internal" href="../introduction.html">Introduction</a></li> +<li class="toctree-l1"><a class="reference internal" href="../getting_started.html">Getting Started</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#prerequisites">Prerequisites</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#dowload-and-unpack-the-source-code">Dowload and unpack the source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#build-source-code">Build source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#setting-up-a-local-tajo-cluster">Setting up a local Tajo cluster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#first-query-execution">First query execution</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../configuration.html">Configuration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../configuration/preliminary.html">Preliminary</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/cluster_setup.html">Cluster Setup</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo_master_configuration.html">Tajo Master Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/worker_configuration.html">Worker Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog_configuration.html">Catalog Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/ha_configuration.html">High Availability for TajoMaster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> +</ul> +</li> +<li class="toctree-l1 current"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul class="current"> +<li class="toctree-l2"><a class="reference internal" href="meta_command.html">Meta Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="dfs_command.html">Executing HDFS commands</a></li> +<li class="toctree-l2 current"><a class="current reference internal" href="">Session Variables</a></li> +<li class="toctree-l2"><a class="reference internal" href="admin_command.html">Administration Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="intro.html">Introducing to TSQL</a></li> +<li class="toctree-l2"><a class="reference internal" href="single_command.html">Executing a single command</a></li> +<li class="toctree-l2"><a class="reference internal" href="execute_file.html">Executing Queries from Files</a></li> +<li class="toctree-l2"><a class="reference internal" href="background_command.html">Executing as background process</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../sql_language.html">SQL Language</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/data_model.html">Data Model</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/ddl.html">Data Definition Language</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/alter_table.html">ALTER TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/queries.html">Queries</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#server-cluster-time-zone">Server Cluster Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#table-time-zone">Table Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#client-time-zone">Client Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#time-zone-id">Time Zone ID</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../functions.html">Functions</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-scalar-functions">Built-in Scalar Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-aggregation-functions">Built-in Aggregation Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-window-functions">Built-in Window Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#user-defined-functions">User-defined Functions</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_partitioning.html">Table Partitioning</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/intro_to_partitioning.html">Introduction to Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/column_partitioning.html">Column Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/range_partitioning.html">Range Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/hash_partitioning.html">Hash Partitioning</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../storage_plugins.html">Storage Plugin</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/overview.html">Storage Plugin Overview</a></li> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/postgresql.html">PostgreSQL Storage Handler</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../index_overview.html">Index (Experimental Feature)</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../index/types.html">Index Types</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/how_to_use.html">How to use index?</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/future_work.html">Future Works</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../backup_and_restore.html">Backup and Restore</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../backup_and_restore/catalog.html">Backup and Restore Catalog</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../hive_integration.html">Hive Integration</a></li> +<li class="toctree-l1"><a class="reference internal" href="../hbase_integration.html">HBase Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#create-table">CREATE TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#drop-table">DROP TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#usage">Usage</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../swift_integration.html">OpenStack Swift Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#swift-configuration">Swift configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#hadoop-configurations">Hadoop configurations</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#tajo-configuration">Tajo configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#querying-on-swift">Querying on Swift</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../jdbc_driver.html">Tajo JDBC Driver</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC driver</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connecting-to-the-tajo-cluster-instance">Connecting to the Tajo cluster instance</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connection-parameters">Connection Parameters</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#an-example-jdbc-client">An Example JDBC Client</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../tajo_client_api.html">Tajo Client API</a></li> +<li class="toctree-l1"><a class="reference internal" href="../faq.html">FAQ</a></li> +</ul> + + + </div> + + </nav> + + <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> + + + <nav class="wy-nav-top" role="navigation" aria-label="top navigation"> + <i data-toggle="wy-nav-top" class="fa fa-bars"></i> + <a href="../index.html">Apache Tajo</a> + </nav> + + + + <div class="wy-nav-content"> + <div class="rst-content"> + <div role="navigation" aria-label="breadcrumbs navigation"> + <ul class="wy-breadcrumbs"> + <li><a href="../index.html">Docs</a> »</li> + + <li><a href="../tsql.html">Tajo Shell (TSQL)</a> »</li> + + <li>Session Variables</li> + <li class="wy-breadcrumbs-aside"> + + <a href="../_sources/tsql/variables.txt" rel="nofollow"> View page source</a> + + </li> + </ul> + <hr/> +</div> + <div role="main"> + + <div class="section" id="session-variables"> +<h1>Session Variables<a class="headerlink" href="#session-variables" title="Permalink to this headline">¶</a></h1> +<p>Once a tajo client connects to the Tajo master, it assigns a unique session. This session is kept until the client is disconnected or it is expired.</p> +<p>For the sake of more convenient user configuration, Tajo provides <cite>session variables</cite>. +With session variables, different configurations are allowed for each session.</p> +<p><code class="docutils literal"><span class="pre">tsql</span></code> provides the meta command <code class="docutils literal"><span class="pre">\set</span></code> to manipulate session variables. Just <code class="docutils literal"><span class="pre">\set</span></code> command shows all session variables.</p> +<div class="highlight-python"><div class="highlight"><pre>default> \set +'name1'='val1' +'name2'='val2' +'name3'='val3' + ... +</pre></div> +</div> +<p><code class="docutils literal"><span class="pre">\set</span> <span class="pre">key</span> <span class="pre">val</span></code> will set the session variable named <em>key</em> with the value <em>val</em>.</p> +<div class="highlight-python"><div class="highlight"><pre>default> \set +'CURRENT_DATABASE'='default' + +default> \set key1 val1 + +default> \set +'CURRENT_DATABASE'='default' +'key1'='val1' +</pre></div> +</div> +<p>Also, <code class="docutils literal"><span class="pre">\unset</span> <span class="pre">key</span></code> will unset the session variable named <em>key</em>.</p> +<p>Currently, tajo provides the following session variables.</p> +<dl class="describe"> +<dt> +<code class="descname">BROADCAST_NON_CROSS_JOIN_THRESHOLD</code></dt> +<dd></dd></dl> + +<p>A threshold for non-cross joins. When a non-cross join query is executed with the broadcast join, the whole size of broadcasted tables won’t exceed this threshold.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-broadcast-non-cross-join-threshold-kb"><span>tajo.dist-query.broadcast.non-cross-join.threshold-kb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: KB</li> +<li>Default value: 5120</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et BROADCAST_NON_CROSS_JOIN_THRESHOLD 5120 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">BROADCAST_CROSS_JOIN_THRESHOLD</code></dt> +<dd></dd></dl> + +<p>A threshold for cross joins. When a cross join query is executed, the whole size of broadcasted tables won’t exceed this threshold.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-broadcast-cross-join-threshold-kb"><span>tajo.dist-query.broadcast.cross-join.threshold-kb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: KB</li> +<li>Default value: 1024</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et BROADCAST_CROSS_JOIN_THRESHOLD 1024 +</pre></div> +</div> +<div class="admonition warning"> +<p class="first admonition-title">Warning</p> +<p class="last">In Tajo, the broadcast join is only the way to perform cross joins. Since the cross join is a very expensive operation, this value need to be tuned carefully.</p> +</div> +<dl class="describe"> +<dt> +<code class="descname">JOIN_TASK_INPUT_SIZE</code></dt> +<dd></dd></dl> + +<p>The repartition join is executed in two stages. When a join query is executed with the repartition join, this value indicates the amount of input data processed by each task at the second stage. +As a result, it determines the degree of the parallel processing of the join query.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-join-task-volume-mb"><span>tajo.dist-query.join.task-volume-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 64</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et JOIN_TASK_INPUT_SIZE 64 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">JOIN_PER_SHUFFLE_SIZE</code></dt> +<dd></dd></dl> + +<p>The repartition join is executed in two stages. When a join query is executed with the repartition join, +this value indicates the output size of each task at the first stage, which determines the number of partitions to be shuffled between two stages.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-join-partition-volume-mb"><span>tajo.dist-query.join.partition-volume-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 128</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et JOIN_PER_SHUFFLE_SIZE 128 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">HASH_JOIN_SIZE_LIMIT</code></dt> +<dd></dd></dl> + +<p>This value provides the criterion to decide the algorithm to perform a join in a task. +If the input data is smaller than this value, join is performed with the in-memory hash join. +Otherwise, the sort-merge join is used.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-join-common-in-memory-hash-threshold-mb"><span>tajo.executor.join.common.in-memory-hash-threshold-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 64</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et HASH_JOIN_SIZE_LIMIT 64 +</pre></div> +</div> +<div class="admonition warning"> +<p class="first admonition-title">Warning</p> +<p class="last">This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, +its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. +This value should be tuned carefully.</p> +</div> +<dl class="describe"> +<dt> +<code class="descname">INNER_HASH_JOIN_SIZE_LIMIT</code></dt> +<dd></dd></dl> + +<p>This value provides the criterion to decide the algorithm to perform an inner join in a task. +If the input data is smaller than this value, the inner join is performed with the in-memory hash join. +Otherwise, the sort-merge join is used.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-join-inner-in-memory-hash-threshold-mb"><span>tajo.executor.join.inner.in-memory-hash-threshold-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 64</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et INNER_HASH_JOIN_SIZE_LIMIT 64 +</pre></div> +</div> +<div class="admonition warning"> +<p class="first admonition-title">Warning</p> +<p class="last">This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, +its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. +This value should be tuned carefully.</p> +</div> +<dl class="describe"> +<dt> +<code class="descname">OUTER_HASH_JOIN_SIZE_LIMIT</code></dt> +<dd></dd></dl> + +<p>This value provides the criterion to decide the algorithm to perform an outer join in a task. +If the input data is smaller than this value, the outer join is performed with the in-memory hash join. +Otherwise, the sort-merge join is used.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-join-outer-in-memory-hash-threshold-mb"><span>tajo.executor.join.outer.in-memory-hash-threshold-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 64</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et OUTER_HASH_JOIN_SIZE_LIMIT 64 +</pre></div> +</div> +<div class="admonition warning"> +<p class="first admonition-title">Warning</p> +<p class="last">This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, +its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. +This value should be tuned carefully.</p> +</div> +<dl class="describe"> +<dt> +<code class="descname">JOIN_HASH_TABLE_SIZE</code></dt> +<dd></dd></dl> + +<p>The initial size of hash table for in-memory hash join.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-join-hash-table-size"><span>tajo.executor.join.hash-table.size</span></a></li> +<li>Property value: Integer</li> +<li>Default value: 100000</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et JOIN_HASH_TABLE_SIZE 100000 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">SORT_TASK_INPUT_SIZE</code></dt> +<dd></dd></dl> + +<p>The sort operation is executed in two stages. When a sort query is executed, this value indicates the amount of input data processed by each task at the second stage. +As a result, it determines the degree of the parallel processing of the sort query.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-sort-task-volume-mb"><span>tajo.dist-query.sort.task-volume-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 64</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et SORT_TASK_INPUT_SIZE 64 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">EXTSORT_BUFFER_SIZE</code></dt> +<dd></dd></dl> + +<p>A threshold to choose the sort algorithm. If the input data is larger than this threshold, the external sort algorithm is used.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-external-sort-buffer-mb"><span>tajo.executor.external-sort.buffer-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 200</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et EXTSORT_BUFFER_SIZE 200 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">SORT_LIST_SIZE</code></dt> +<dd></dd></dl> + +<p>The initial size of list for in-memory sort.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-sort-list-size"><span>tajo.executor.sort.list.size</span></a></li> +<li>Property value: Integer</li> +<li>Default value: 100000</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et SORT_LIST_SIZE 100000 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">GROUPBY_MULTI_LEVEL_ENABLED</code></dt> +<dd></dd></dl> + +<p>A flag to enable the multi-level algorithm for distinct aggregation. If this value is set, 3-phase aggregation algorithm is used. +Otherwise, 2-phase aggregation algorithm is used.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-groupby-multi-level-aggr"><span>tajo.dist-query.groupby.multi-level-aggr</span></a></li> +<li>Property value: Boolean</li> +<li>Default value: true</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et GROUPBY_MULTI_LEVEL_ENABLED <span class="nb">true</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">GROUPBY_PER_SHUFFLE_SIZE</code></dt> +<dd></dd></dl> + +<p>The aggregation is executed in two stages. When an aggregation query is executed, +this value indicates the output size of each task at the first stage, which determines the number of partitions to be shuffled between two stages.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-groupby-partition-volume-mb"><span>tajo.dist-query.groupby.partition-volume-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 256</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et GROUPBY_PER_SHUFFLE_SIZE 256 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">GROUPBY_TASK_INPUT_SIZE</code></dt> +<dd></dd></dl> + +<p>The aggregation operation is executed in two stages. When an aggregation query is executed, this value indicates the amount of input data processed by each task at the second stage. +As a result, it determines the degree of the parallel processing of the aggregation query.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-groupby-task-volume-mb"><span>tajo.dist-query.groupby.task-volume-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 64</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et GROUPBY_TASK_INPUT_SIZE 64 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">HASH_GROUPBY_SIZE_LIMIT</code></dt> +<dd></dd></dl> + +<p>This value provides the criterion to decide the algorithm to perform an aggregation in a task. +If the input data is smaller than this value, the aggregation is performed with the in-memory hash aggregation. +Otherwise, the sort-based aggregation is used.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-groupby-in-memory-hash-threshold-mb"><span>tajo.executor.groupby.in-memory-hash-threshold-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 64</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et HASH_GROUPBY_SIZE_LIMIT 64 +</pre></div> +</div> +<div class="admonition warning"> +<p class="first admonition-title">Warning</p> +<p class="last">This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, +its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. +This value should be tuned carefully.</p> +</div> +<dl class="describe"> +<dt> +<code class="descname">AGG_HASH_TABLE_SIZE</code></dt> +<dd></dd></dl> + +<p>The initial size of hash table for in-memory aggregation.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-executor-aggregate-hash-table-size"><span>tajo.executor.aggregate.hash-table.size</span></a></li> +<li>Property value: Integer</li> +<li>Default value: 10000</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et AGG_HASH_TABLE_SIZE 10000 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">TIMEZONE</code></dt> +<dd></dd></dl> + +<p>Refer to <a class="reference internal" href="../time_zone.html"><em>Time Zone</em></a>.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-timezone"><span>tajo.timezone</span></a></li> +<li>Property value: Time zone id</li> +<li>Default value: Default time zone of JVM</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et TIMEZONE GMT+9 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">DATE_ORDER</code></dt> +<dd></dd></dl> + +<p>Date order specification.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-datetime-date-order"><span>tajo.datetime.date-order</span></a></li> +<li>Property value: One of YMD, DMY, MDY.</li> +<li>Default value: YMD</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et DATE_ORDER YMD +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">PARTITION_NO_RESULT_OVERWRITE_ENABLED</code></dt> +<dd></dd></dl> + +<p>If this value is true, a partitioned table is overwritten even if a subquery leads to no result. Otherwise, the table data will be kept if there is no result.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-partition-overwrite-even-if-no-result"><span>tajo.partition.overwrite.even-if-no-result</span></a></li> +<li>Property value: Boolean</li> +<li>Default value: false</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et PARTITION_NO_RESULT_OVERWRITE_ENABLED <span class="nb">false</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">TABLE_PARTITION_PER_SHUFFLE_SIZE</code></dt> +<dd></dd></dl> + +<p>In Tajo, storing a partition table is executed in two stages. +This value indicates the output size of a task of the former stage, which determines the number of partitions to be shuffled between two stages.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-dist-query-table-partition-task-volume-mb"><span>tajo.dist-query.table-partition.task-volume-mb</span></a></li> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 256</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et TABLE_PARTITION_PER_SHUFFLE_SIZE 256 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">ARITHABORT</code></dt> +<dd></dd></dl> + +<p>A flag to indicate how to handle the errors caused by invalid arithmetic operations. If true, a running query will be terminated with an overflow or a divide-by-zero.</p> +<blockquote> +<div><ul class="simple"> +<li>Configuration name: <a class="reference internal" href="../configuration/tajo-site-xml.html#tajo-behavior-arithmetic-abort"><span>tajo.behavior.arithmetic-abort</span></a></li> +<li>Property value: Boolean</li> +<li>Default value: false</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et ARITHABORT <span class="nb">false</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">MAX_OUTPUT_FILE_SIZE</code></dt> +<dd></dd></dl> + +<p>Maximum per-output file size. 0 means infinite.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Integer</li> +<li>Unit: MB</li> +<li>Default value: 0</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et MAX_OUTPUT_FILE_SIZE 0 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">SESSION_EXPIRY_TIME</code></dt> +<dd></dd></dl> + +<p>Session expiry time.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Integer</li> +<li>Unit: seconds</li> +<li>Default value: 3600</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et SESSION_EXPIRY_TIME 3600 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">CLI_COLUMNS</code></dt> +<dd></dd></dl> + +<p>Sets the width for the wrapped format.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Integer</li> +<li>Default value: 120</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et CLI_COLUMNS 120 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">CLI_NULL_CHAR</code></dt> +<dd></dd></dl> + +<p>Sets the string to be printed in place of a null value.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: String</li> +<li>Default value: ‘’</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et CLI_NULL_CHAR <span class="s1">''</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">CLI_PAGE_ROWS</code></dt> +<dd></dd></dl> + +<p>Sets the number of rows for paging.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Integer</li> +<li>Default value: 100</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et CLI_PAGE_ROWS 100 +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">CLI_PAGING_ENABLED</code></dt> +<dd></dd></dl> + +<p>Enable paging of result display.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Boolean</li> +<li>Default value: true</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et CLI_PAGING_ENABLED <span class="nb">true</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">CLI_DISPLAY_ERROR_TRACE</code></dt> +<dd></dd></dl> + +<p>Enable display of error trace.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Boolean</li> +<li>Default value: true</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et CLI_DISPLAY_ERROR_TRACE <span class="nb">true</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">CLI_FORMATTER_CLASS</code></dt> +<dd></dd></dl> + +<p>Sets the output format class to display results.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Class name</li> +<li>Default value: org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et CLI_FORMATTER_CLASS org.apache.tajo.cli.tsql.DefaultTajoCliOutputFormatter +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">ON_ERROR_STOP</code></dt> +<dd></dd></dl> + +<p>tsql will exit if an error occurs.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Boolean</li> +<li>Default value: false</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et ON_ERROR_STOP <span class="nb">false</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">NULL_CHAR</code></dt> +<dd></dd></dl> + +<p>Null char of text file output. This value is used when the table property <cite>text.null</cite> is not specified.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: String</li> +<li>Default value: ‘\N’</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et NULL_CHAR <span class="s1">'\\N'</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">DEBUG_ENABLED</code></dt> +<dd></dd></dl> + +<p>A flag to enable debug mode.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Boolean</li> +<li>Default value: false</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et DEBUG_ENABLED <span class="nb">false</span> +</pre></div> +</div> +<dl class="describe"> +<dt> +<code class="descname">FETCH_ROWNUM</code></dt> +<dd></dd></dl> + +<p>The number of rows to be fetched from Master each time.</p> +<blockquote> +<div><ul class="simple"> +<li>Property value: Integer</li> +<li>Default value: 200</li> +<li>Example</li> +</ul> +</div></blockquote> +<div class="highlight-sh"><div class="highlight"><pre><span class="se">\s</span>et FETCH_ROWNUM 200 +</pre></div> +</div> +</div> + + + </div> + <footer> + + <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation"> + + <a href="admin_command.html" class="btn btn-neutral float-right" title="Administration Commands"/>Next <span class="fa fa-arrow-circle-right"></span></a> + + + <a href="dfs_command.html" class="btn btn-neutral" title="Executing HDFS commands"><span class="fa fa-arrow-circle-left"></span> Previous</a> + + </div> + + + <hr/> + + <div role="contentinfo"> + <p> + © Copyright 2015, Apache Tajo Team. + </p> + </div> + + <a href="https://github.com/snide/sphinx_rtd_theme";>Sphinx theme</a> provided by <a href="https://readthedocs.org";>Read the Docs</a> +</footer> + </div> + </div> + + </section> + + </div> + + + + + + <script type="text/javascript"> + var DOCUMENTATION_OPTIONS = { + URL_ROOT:'../', + VERSION:'0.11.0', + COLLAPSE_INDEX:false, + FILE_SUFFIX:'.html', + HAS_SOURCE: true + }; + </script> + <script type="text/javascript" src="../_static/jquery.js"></script> + <script type="text/javascript" src="../_static/underscore.js"></script> + <script type="text/javascript" src="../_static/doctools.js"></script> + + + + + + <script type="text/javascript" src="../_static/js/theme.js"></script> + + + + + <script type="text/javascript"> + jQuery(function () { + SphinxRtdTheme.StickyNav.enable(); + }); + </script> + + +</body> +</html> \ No newline at end of file Modified: tajo/site/docs/current/_sources/configuration.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/configuration.txt?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/_sources/configuration.txt (original) +++ tajo/site/docs/current/_sources/configuration.txt Thu Feb 4 00:29:05 2016 @@ -1,6 +1,6 @@ -************************ +************* Configuration -************************ +************* .. toctree:: :maxdepth: 1 @@ -13,4 +13,5 @@ Configuration configuration/ha_configuration configuration/service_config_defaults configuration/tajo-site-xml - configuration/catalog-site-xml \ No newline at end of file + configuration/catalog-site-xml + configuration/storage-site-json Added: tajo/site/docs/current/_sources/configuration/storage-site-json.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/configuration/storage-site-json.txt?rev=1728394&view=auto ============================================================================== --- tajo/site/docs/current/_sources/configuration/storage-site-json.txt (added) +++ tajo/site/docs/current/_sources/configuration/storage-site-json.txt Thu Feb 4 00:29:05 2016 @@ -0,0 +1,64 @@ +************************** +The storage-site.json File +************************** + +This file is for configuring :doc:`/table_management/tablespaces`. +The syntax of ``storage-site.json`` is like this basically: + +.. code:: json + + { + "spaces": { + <space config 1>, <space config 2>, ... + }, + "storages": { + <storage config 1>, <storage config 2>, ... + } + } + +------ +Spaces +------ + +This is a section for registering tablespaces. Some space config example is here: + +.. code:: json + + "spaces": { + "jdbc_pgsql": { + "uri": "jdbc:postgresql://127.0.0.1:5740/origin", + "configs": { + "mapped_database":"tajo_pgsql_origin", + "connection_properties": { + "user":"tajouser", + "password":"123456" + } + } + }, + ... + } + +* **space name** : Your own tablespace name which indicates a specific tablespace. Alpha-numeric characters and underscore(_) are permitted. +* **uri** : An URI address of a tablespace +* **configs** : It is optional. You can specify it as JSON object to pass to each tablespace handler. + +After you specify a tablespace, you can use it in `create table statement <../sql_language/ddl.html#create-table>`_. + +-------- +Storages +-------- + +This is for registering storage format and custom storage handler class. +Tajo already supports HDFS, HBase, PostgreSQL, Amazon S3, Openstack Swift, etc, thus in usual cases using mentioned storages, you don't have to add any ``storages`` configuration. +However, if you want to use your custom storage as one of Tajo's data source, you need to add a configuration for your storage. Here is an example of HDFS storage. +See :doc:`storage_plugins/overview` for more information. + +.. code:: json + + "storages": { + "hdfs": { + "handler": "org.apache.tajo.storage.FileTablespace", + "default-format": "text" + }, + ... + } Modified: tajo/site/docs/current/_sources/configuration/tajo-site-xml.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/configuration/tajo-site-xml.txt?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/_sources/configuration/tajo-site-xml.txt (original) +++ tajo/site/docs/current/_sources/configuration/tajo-site-xml.txt Thu Feb 4 00:29:05 2016 @@ -2,23 +2,497 @@ The tajo-site.xml File ********************** -To the ``core-site.xml`` file on every host in your cluster, you must add the following information: +You can add more configurations in the ``tajo-site.xml`` file. Note that you should replicate this file to the whole hosts in your cluster once you edited. +If you are looking for the configurations for the master and the worker, please refer to :doc:`tajo_master_configuration` and :doc:`worker_configuration`. +Also, catalog configurations are found here :doc:`catalog_configuration`. + +========================= +Join Query Settings +========================= + +"""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.join.auto-broadcast` +"""""""""""""""""""""""""""""""""""""" + +A flag to enable or disable the use of broadcast join. + + * Property value type: Boolean + * Default value: true + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.join.auto-broadcast</name> + <value>true</value> + </property> + +.. _tajo.dist-query.broadcast.non-cross-join.threshold-kb: + +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.broadcast.non-cross-join.threshold-kb` +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +A threshold for non-cross joins. When a non-cross join query is executed with the broadcast join, the whole size of broadcasted tables won't exceed this threshold. + + * Property value type: Integer + * Unit: KB + * Default value: 5120 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.broadcast.non-cross-join.threshold-kb</name> + <value>5120</value> + </property> + +.. _tajo.dist-query.broadcast.cross-join.threshold-kb: + +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.broadcast.cross-join.threshold-kb` +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +A threshold for cross joins. When a cross join query is executed, the whole size of broadcasted tables won't exceed this threshold. + + * Property value type: Integer + * Unit: KB + * Default value: 1024 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.broadcast.cross-join.threshold-kb</name> + <value>1024</value> + </property> + +.. warning:: + In Tajo, the broadcast join is only the way to perform cross joins. Since the cross join is a very expensive operation, this value need to be tuned carefully. + +.. _tajo.dist-query.join.task-volume-mb: + +"""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.join.task-volume-mb` +"""""""""""""""""""""""""""""""""""""" + +The repartition join is executed in two stages. When a join query is executed with the repartition join, this value indicates the amount of input data processed by each task at the second stage. +As a result, it determines the degree of the parallel processing of the join query. + + * Property value type: Integer + * Unit: MB + * Default value: 64 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.join.task-volume-mb</name> + <value>64</value> + </property> + +.. _tajo.dist-query.join.partition-volume-mb: + +""""""""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.join.partition-volume-mb` +""""""""""""""""""""""""""""""""""""""""""" + +The repartition join is executed in two stages. When a join query is executed with the repartition join, +this value indicates the output size of each task at the first stage, which determines the number of partitions to be shuffled between two stages. + + * Property value type: Integer + * Unit: MB + * Default value: 128 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.join.partition-volume-mb</name> + <value>128</value> + </property> + +.. _tajo.executor.join.common.in-memory-hash-threshold-mb: + +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.executor.join.common.in-memory-hash-threshold-mb` +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +This value provides the criterion to decide the algorithm to perform a join in a task. +If the input data is smaller than this value, join is performed with the in-memory hash join. +Otherwise, the sort-merge join is used. + + * Property value type: Integer + * Unit: MB + * Default value: 64 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.join.common.in-memory-hash-threshold-mb</name> + <value>64</value> + </property> + +.. warning:: + This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, + its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. + This value should be tuned carefully. + +.. _tajo.executor.join.inner.in-memory-hash-threshold-mb: + +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.executor.join.inner.in-memory-hash-threshold-mb` +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +This value provides the criterion to decide the algorithm to perform an inner join in a task. +If the input data is smaller than this value, the inner join is performed with the in-memory hash join. +Otherwise, the sort-merge join is used. + + * Property value type: Integer + * Unit: MB + * Default value: 64 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.join.inner.in-memory-hash-threshold-mb</name> + <value>64</value> + </property> + +.. warning:: + This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, + its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. + This value should be tuned carefully. + +.. _tajo.executor.join.outer.in-memory-hash-threshold-mb: + +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.executor.join.outer.in-memory-hash-threshold-mb` +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +This value provides the criterion to decide the algorithm to perform an outer join in a task. +If the input data is smaller than this value, the outer join is performed with the in-memory hash join. +Otherwise, the sort-merge join is used. + + * Property value type: Integer + * Unit: MB + * Default value: 64 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.join.outer.in-memory-hash-threshold-mb</name> + <value>64</value> + </property> + +.. warning:: + This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, + its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. + This value should be tuned carefully. + +.. _tajo.executor.join.hash-table.size: + +""""""""""""""""""""""""""""""""""""" +`tajo.executor.join.hash-table.size` +""""""""""""""""""""""""""""""""""""" + +The initial size of hash table for in-memory hash join. + + * Property value type: Integer + * Default value: 100000 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.join.hash-table.size</name> + <value>100000</value> + </property> ====================== -System Config +Sort Query Settings ====================== +.. _tajo.dist-query.sort.task-volume-mb: + +"""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.sort.task-volume-mb` +"""""""""""""""""""""""""""""""""""""" + +The sort operation is executed in two stages. When a sort query is executed, this value indicates the amount of input data processed by each task at the second stage. +As a result, it determines the degree of the parallel processing of the sort query. + + * Property value type: Integer + * Unit: MB + * Default value: 64 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.sort.task-volume-mb</name> + <value>64</value> + </property> + +.. _tajo.executor.external-sort.buffer-mb: + +"""""""""""""""""""""""""""""""""""""""" +`tajo.executor.external-sort.buffer-mb` +"""""""""""""""""""""""""""""""""""""""" + +A threshold to choose the sort algorithm. If the input data is larger than this threshold, the external sort algorithm is used. + + * Property value type: Integer + * Unit: MB + * Default value: 200 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.external-sort.buffer-mb</name> + <value>200</value> + </property> + +.. _tajo.executor.sort.list.size: +"""""""""""""""""""""""""""""""""""""" +`tajo.executor.sort.list.size` +"""""""""""""""""""""""""""""""""""""" + +The initial size of list for in-memory sort. + + * Property value type: Integer + * Default value: 100000 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.sort.list.size</name> + <value>100000</value> + </property> + +========================= +Group by Query Settings +========================= + +.. _tajo.dist-query.groupby.multi-level-aggr: + +"""""""""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.groupby.multi-level-aggr` +"""""""""""""""""""""""""""""""""""""""""""" + +A flag to enable the multi-level algorithm for distinct aggregation. If this value is set, 3-phase aggregation algorithm is used. +Otherwise, 2-phase aggregation algorithm is used. + + * Property value type: Boolean + * Default value: true + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.groupby.multi-level-aggr</name> + <value>true</value> + </property> + +.. _tajo.dist-query.groupby.partition-volume-mb: + +"""""""""""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.groupby.partition-volume-mb` +"""""""""""""""""""""""""""""""""""""""""""""" + +The aggregation is executed in two stages. When an aggregation query is executed, +this value indicates the output size of each task at the first stage, which determines the number of partitions to be shuffled between two stages. + + * Property value type: Integer + * Unit: MB + * Default value: 256 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.groupby.partition-volume-mb</name> + <value>256</value> + </property> + +.. _tajo.dist-query.groupby.task-volume-mb: + +"""""""""""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.groupby.task-volume-mb` +"""""""""""""""""""""""""""""""""""""""""""""" + +The aggregation operation is executed in two stages. When an aggregation query is executed, this value indicates the amount of input data processed by each task at the second stage. +As a result, it determines the degree of the parallel processing of the aggregation query. + + * Property value type: Integer + * Unit: MB + * Default value: 64 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.groupby.task-volume-mb</name> + <value>64</value> + </property> + +.. _tajo.executor.groupby.in-memory-hash-threshold-mb: + +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.executor.groupby.in-memory-hash-threshold-mb` +"""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +This value provides the criterion to decide the algorithm to perform an aggregation in a task. +If the input data is smaller than this value, the aggregation is performed with the in-memory hash aggregation. +Otherwise, the sort-based aggregation is used. + + * Property value type: Integer + * Unit: MB + * Default value: 64 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.groupby.in-memory-hash-threshold-mb</name> + <value>64</value> + </property> + +.. warning:: + This value is the size of the input stored on file systems. So, when the input data is loaded into JVM heap, + its actual size is usually much larger than the configured value, which means that too large threshold can cause unexpected OutOfMemory errors. + This value should be tuned carefully. + +.. _tajo.executor.aggregate.hash-table.size: + +"""""""""""""""""""""""""""""""""""""""""" +`tajo.executor.aggregate.hash-table.size` +"""""""""""""""""""""""""""""""""""""""""" + +The initial size of hash table for in-memory aggregation. + + * Property value type: Integer + * Default value: 10000 + * Example + +.. code-block:: xml + + <property> + <name>tajo.executor.aggregate.hash-table.size</name> + <value>10000</value> + </property> ====================== Date/Time Settings ====================== -+--------------------------+----------------+--------------------------------------------------------+ -| Property Name | Property Value | Descriptions | -+==========================+================+========================================================+ -| tajo.timezone | Time zone id | Refer to :doc:`/time_zone` | -+--------------------------+----------------+--------------------------------------------------------+ -| tajo.datetime.date-order | Date order | Determine date order. It should be one of YMD, DMY, MDY| -+--------------------------+----------------+--------------------------------------------------------+ +.. _tajo.timezone: + +""""""""""""""""""" +`tajo.timezone` +""""""""""""""""""" + +Refer to :doc:`/time_zone`. + + * Property value type: Time zone id + * Default value: Default time zone of JVM + * Example + +.. code-block:: xml + + <property> + <name>tajo.timezone</name> + <value>GMT+9</value> + </property> + +.. _tajo.datetime.date-order: + +""""""""""""""""""""""""""" +`tajo.datetime.date-order` +""""""""""""""""""""""""""" + +Date order specification. + + * Property value type: One of YMD, DMY, MDY. + * Default value: YMD + * Example + +.. code-block:: xml + + <property> + <name>tajo.datetime.date-order</name> + <value>YMD</value> + </property> + +====================== +Table partitions +====================== + +.. _tajo.partition.overwrite.even-if-no-result: + +"""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.partition.overwrite.even-if-no-result` +"""""""""""""""""""""""""""""""""""""""""""""""""""" + +If this value is true, a partitioned table is overwritten even if a subquery leads to no result. Otherwise, the table data will be kept if there is no result. + + * Property value type: Boolean + * Default value: false + * Example + +.. code-block:: xml + + <property> + <name>tajo.partition.overwrite.even-if-no-result</name> + <value>false</value> + </property> + +.. _tajo.dist-query.table-partition.task-volume-mb: + +"""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.dist-query.table-partition.task-volume-mb` +"""""""""""""""""""""""""""""""""""""""""""""""""""" + +In Tajo, storing a partition table is executed in two stages. +This value indicates the output size of a task of the former stage, which determines the number of partitions to be shuffled between two stages. + + * Property value type: Integer + * Unit: MB + * Default value: 256 + * Example + +.. code-block:: xml + + <property> + <name>tajo.dist-query.table-partition.task-volume-mb</name> + <value>256</value> + </property> + +====================== +Arithmetic Settings +====================== + +.. _tajo.behavior.arithmetic-abort: + +"""""""""""""""""""""""""""""""""""""""""""""""""""" +`tajo.behavior.arithmetic-abort` +"""""""""""""""""""""""""""""""""""""""""""""""""""" + +A flag to indicate how to handle the errors caused by invalid arithmetic operations. If true, a running query will be terminated with an overflow or a divide-by-zero. + + * Property value type: Boolean + * Default value: false + * Example + +.. code-block:: xml + <property> + <name>tajo.behavior.arithmetic-abort</name> + <value>false</value> + </property> \ No newline at end of file Modified: tajo/site/docs/current/_sources/configuration/worker_configuration.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/configuration/worker_configuration.txt?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/_sources/configuration/worker_configuration.txt (original) +++ tajo/site/docs/current/_sources/configuration/worker_configuration.txt Thu Feb 4 00:29:05 2016 @@ -14,7 +14,7 @@ If you want to adjust heap memory size, TAJO_WORKER_HEAPSIZE=8000 -The default size is 1000 (1GB). +The default size is 5000 (5GB). ======================== Temporary Data Directory @@ -54,8 +54,8 @@ In order to specify the resource capacit =================================== ============= ====================== ================================= tajo.worker.resource.cpu-cores Integer available cpu-cores the number of cpu cores tajo.worker.resource.memory-mb Integer available jvm heap memory size (MB) - tajo.worker.resource.disks Integer 1 the number of disks - tajo.task.resource.min.memory-mb Integer 500 minimum allocatable memory per task + tajo.worker.resource.disks Integer 2 the number of disks + tajo.task.resource.min.memory-mb Integer 1000 minimum allocatable memory per task tajo.qm.resource.min.memory-mb Integer 500 minimum allocatable memory per query =================================== ============= ====================== ================================= @@ -73,7 +73,7 @@ In order to specify the resource capacit Example ------------ -Assume that you want to give 15GB Jvm heap, 1GB memory per task, 4 disks, and 12 cores on each worker. The example configuration is as follows: +Assume that you want to give 15GB Jvm heap, 2GB memory per task, 4 disks, and 12 cores on each worker. The example configuration is as follows: ``tajo-env.sh`` @@ -87,13 +87,13 @@ Assume that you want to give 15GB Jvm he .. code-block:: xml <property> - <name>tajo.worker.resource.tajo.worker.resource.cpu-cores</name> + <name>tajo.worker.resource.cpu-cores</name> <value>12</value> </property> <property> <name>tajo.task.resource.min.memory-mb</name> - <value>1000</value> + <value>2000</value> </property> <property> @@ -117,7 +117,7 @@ Assume that you want to give 15GB Jvm he <property> <name>tajo.task.resource.min.memory-mb</name> - <value>1000</value> + <value>2000</value> </property> <property> @@ -141,7 +141,7 @@ Assume that you want to give 15GB Jvm he <property> <name>tajo.task.resource.min.memory-mb</name> - <value>1000</value> + <value>2000</value> </property> <property> Modified: tajo/site/docs/current/_sources/hbase_integration.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/hbase_integration.txt?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/_sources/hbase_integration.txt (original) +++ tajo/site/docs/current/_sources/hbase_integration.txt Thu Feb 4 00:29:05 2016 @@ -9,12 +9,14 @@ In order to use this feature, you need t This section describes how to setup HBase integration. -First, you need to set your HBase home directory to the environment variable ``HBASE_HOME`` in conf/tajo-env.sh as follows: :: +First, you need to set your HBase home directory to the environment variable ``HBASE_HOME`` in ``conf/tajo-env.sh`` as follows: :: export HBASE_HOME=/path/to/your/hbase/directory If you set the directory, Tajo will add HBase library file to classpath. +Next, you must configure tablespace about HBase. Please see :doc:`/table_management/tablespaces` if you want to know more information about it. + ======================== @@ -30,8 +32,12 @@ CREATE TABLE WITH ('table'='<hbase_table_name>' , 'columns'=':key,<column_family_name>:<qualifier_name>, ...' , 'hbase.zookeeper.quorum'='<zookeeper_address>' - , 'hbase.zookeeper.property.clientPort'='<zookeeper_client_port>' - ) + , 'hbase.zookeeper.property.clientPort'='<zookeeper_client_port>') + [LOCATION 'hbase:zk://<hostname>:<port>/'] ; + +``IF NOT EXISTS`` allows ``CREATE [EXTERNAL] TABLE`` statement to avoid an error which occurs when the table does not exist. + +If you want to create ``EXTERNAL TABLE``, You must write ``LOCATION`` statement. Options @@ -40,7 +46,6 @@ Options * ``hbase.zookeeper.quorum`` : Set zookeeper quorum address. You can use different zookeeper cluster on the same Tajo database. If you don't set the zookeeper address, Tajo will refer the property of hbase-site.xml file. * ``hbase.zookeeper.property.clientPort`` : Set zookeeper client port. If you don't set the port, Tajo will refer the property of hbase-site.xml file. -``IF NOT EXISTS`` allows ``CREATE [EXTERNAL] TABLE`` statement to avoid an error which occurs when the table does not exist. @@ -94,7 +99,8 @@ In order to create a new HBase table whi CREATE EXTERNAL TABLE blog (rowkey text, author text, register_date text, title text) USING hbase WITH ( 'table'='blog' - , 'columns'=':key,info:author,info:date,content:title'); + , 'columns'=':key,info:author,info:date,content:title') + LOCATION 'hbase:zk://<hostname>:<port>/'; After executing the command above, you should be able to see the new table in the HBase shell: Modified: tajo/site/docs/current/_sources/hive_integration.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/hive_integration.txt?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/_sources/hive_integration.txt (original) +++ tajo/site/docs/current/_sources/hive_integration.txt Thu Feb 4 00:29:05 2016 @@ -1,6 +1,6 @@ -************************************* +**************** Hive Integration -************************************* +**************** Apache Tajo⢠catalog supports HiveCatalogStore to integrate with Apache Hiveâ¢. This integration allows Tajo to access all tables used in Apache Hive. @@ -12,16 +12,22 @@ and then add some configs into ``conf/ta This section describes how to setup HiveMetaStore integration. This instruction would take no more than five minutes. -You need to set your Hive home directory to the environment variable ``HIVE_HOME`` in conf/tajo-env.sh as follows: :: +You need to set your Hive home directory to the environment variable **HIVE_HOME** in ``conf/tajo-env.sh`` as follows: + +.. code-block:: sh export HIVE_HOME=/path/to/your/hive/directory If you need to use jdbc to connect HiveMetaStore, you have to prepare MySQL jdbc driver. -Next, you should set the path of MySQL JDBC driver jar file to the environment variable HIVE_JDBC_DRIVER_DIR in conf/tajo-env.sh as follows: :: +Next, you should set the path of MySQL JDBC driver jar file to the environment variable **HIVE_JDBC_DRIVER_DIR** in ``conf/tajo-env.sh`` as follows: + +.. code-block:: sh - export HIVE_JDBC_DRIVER_DIR==/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar + export HIVE_JDBC_DRIVER_DIR=/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar -Finally, you should specify HiveCatalogStore as Tajo catalog driver class in ``conf/catalog-site.xml`` as follows: :: +Finally, you should specify HiveCatalogStore as Tajo catalog driver class in ``conf/catalog-site.xml`` as follows: + +.. code-block:: xml <property> <name>tajo.catalog.store.class</name> @@ -30,13 +36,17 @@ Finally, you should specify HiveCatalogS .. note:: - Hive stores a list of partitions for each table in its metastore. If new partitions are - directly added to HDFS, HiveMetastore will not able aware of these partitions unless the user + Hive stores a list of partitions for each table in its metastore. When new partitions are + added directly to HDFS, HiveMetastore can't recognize these partitions until the user executes ``ALTER TABLE table_name ADD PARTITION`` commands on each of the newly added partitions or - ``MSCK REPAIR TABLE table_name`` command. + ``MSCK REPAIR TABLE table_name`` command. + + But current Tajo doesn't provide ``ADD PARTITION`` command and Hive doesn't provide an api for + responding to ``MSK REPAIR TABLE`` command. Thus, if you insert data to Hive partitioned + table and you want to scan the updated partitions through Tajo, you must run following command on Hive + (see `Hive doc <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RecoverPartitions(MSCKREPAIRTABLE)>`_ + for more details of the command): - But current tajo doesn't provide ``ADD PARTITION`` command and hive doesn't provide an api for - responding to ``MSK REPAIR TABLE`` command. Thus, if you insert data to hive partitioned - table and you want to scan the updated partitions through Tajo, you must run following command on hive :: + .. code-block:: sql - $ MSCK REPAIR TABLE [table_name]; + MSCK REPAIR TABLE [table_name]; Modified: tajo/site/docs/current/_sources/index.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/index.txt?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/_sources/index.txt (original) +++ tajo/site/docs/current/_sources/index.txt Thu Feb 4 00:29:05 2016 @@ -15,7 +15,7 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Apache Tajo⢠(0.11.0 Release) - User documentation +Apache Tajo⢠(0.11.1 Release) - User documentation =========================================================================== Table of Contents: Modified: tajo/site/docs/current/_sources/index/future_work.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/index/future_work.txt?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/_sources/index/future_work.txt (original) +++ tajo/site/docs/current/_sources/index/future_work.txt Thu Feb 4 00:29:05 2016 @@ -2,7 +2,6 @@ Future Works ************************************* -* Providing more index types, such as bitmap and HBase index +* Supporting more index types, such as bitmap and HBase index * Supporting index on partitioned tables -* Supporting the backup and restore feature * Cost-based query optimization by estimating the query selectivity \ No newline at end of file
