Added: tajo/site/docs/0.11.0/tsql/single_command.html URL: http://svn.apache.org/viewvc/tajo/site/docs/0.11.0/tsql/single_command.html?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/0.11.0/tsql/single_command.html (added) +++ tajo/site/docs/0.11.0/tsql/single_command.html Tue Oct 27 11:04:33 2015 @@ -0,0 +1,297 @@ + + +<!DOCTYPE html> +<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]--> +<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]--> +<head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + + <title>Executing a single command — Apache Tajo 0.11.0 documentation</title> + + + + + + + <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'> + + + + + + + + + + <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" /> + + + + <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/> + <link rel="up" title="Tajo Shell (TSQL)" href="../tsql.html"/> + <link rel="next" title="Executing Queries from Files" href="execute_file.html"/> + <link rel="prev" title="Introducing to TSQL" href="intro.html"/> + + + <script src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script> + +</head> + +<body class="wy-body-for-nav" role="document"> + + <div class="wy-grid-for-nav"> + + + <nav data-toggle="wy-nav-shift" class="wy-nav-side"> + <div class="wy-side-nav-search"> + <a href="../index.html" class="fa fa-home"> Apache Tajo</a> + <div role="search"> + <form id ="rtd-search-form" class="wy-form" action="../search.html" method="get"> + <input type="text" name="q" placeholder="Search docs" /> + <input type="hidden" name="check_keywords" value="yes" /> + <input type="hidden" name="area" value="default" /> + </form> +</div> + </div> + + <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> + + + <ul class="current"> +<li class="toctree-l1"><a class="reference internal" href="../introduction.html">Introduction</a></li> +<li class="toctree-l1"><a class="reference internal" href="../getting_started.html">Getting Started</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#prerequisites">Prerequisites</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#dowload-and-unpack-the-source-code">Dowload and unpack the source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#build-source-code">Build source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#setting-up-a-local-tajo-cluster">Setting up a local Tajo cluster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#first-query-execution">First query execution</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../configuration.html">Configuration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../configuration/preliminary.html">Preliminary</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/cluster_setup.html">Cluster Setup</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo_master_configuration.html">Tajo Master Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/worker_configuration.html">Worker Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog_configuration.html">Catalog Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/ha_configuration.html">High Availability for TajoMaster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +</ul> +</li> +<li class="toctree-l1 current"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul class="current"> +<li class="toctree-l2"><a class="reference internal" href="meta_command.html">Meta Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="dfs_command.html">Executing HDFS commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="variables.html">Session Variables</a></li> +<li class="toctree-l2"><a class="reference internal" href="admin_command.html">Administration Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="intro.html">Introducing to TSQL</a></li> +<li class="toctree-l2 current"><a class="current reference internal" href="">Executing a single command</a></li> +<li class="toctree-l2"><a class="reference internal" href="execute_file.html">Executing Queries from Files</a></li> +<li class="toctree-l2"><a class="reference internal" href="background_command.html">Executing as background process</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../sql_language.html">SQL Language</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/data_model.html">Data Model</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/ddl.html">Data Definition Language</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/alter_table.html">ALTER TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/queries.html">Queries</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#server-cluster-time-zone">Server Cluster Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#table-time-zone">Table Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#client-time-zone">Client Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#time-zone-id">Time Zone ID</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../functions.html">Functions</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-scalar-functions">Built-in Scalar Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-aggregation-functions">Built-in Aggregation Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-window-functions">Built-in Window Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#user-defined-functions">User-defined Functions</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_partitioning.html">Table Partitioning</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/intro_to_partitioning.html">Introduction to Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/column_partitioning.html">Column Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/range_partitioning.html">Range Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/hash_partitioning.html">Hash Partitioning</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../storage_plugins.html">Storage Plugin</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/overview.html">Storage Plugin Overview</a></li> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/postgresql.html">PostgreSQL Storage Handler</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../index_overview.html">Index (Experimental Feature)</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../index/types.html">Index Types</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/how_to_use.html">How to use index?</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/future_work.html">Future Works</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../backup_and_restore.html">Backup and Restore</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../backup_and_restore/catalog.html">Backup and Restore Catalog</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../hive_integration.html">Hive Integration</a></li> +<li class="toctree-l1"><a class="reference internal" href="../hbase_integration.html">HBase Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#create-table">CREATE TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#drop-table">DROP TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#usage">Usage</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../swift_integration.html">OpenStack Swift Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#swift-configuration">Swift configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#hadoop-configurations">Hadoop configurations</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#tajo-configuration">Tajo configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#querying-on-swift">Querying on Swift</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../jdbc_driver.html">Tajo JDBC Driver</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC driver</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connecting-to-the-tajo-cluster-instance">Connecting to the Tajo cluster instance</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connection-parameters">Connection Parameters</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#an-example-jdbc-client">An Example JDBC Client</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../tajo_client_api.html">Tajo Client API</a></li> +<li class="toctree-l1"><a class="reference internal" href="../faq.html">FAQ</a></li> +</ul> + + + </div> + + </nav> + + <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> + + + <nav class="wy-nav-top" role="navigation" aria-label="top navigation"> + <i data-toggle="wy-nav-top" class="fa fa-bars"></i> + <a href="../index.html">Apache Tajo</a> + </nav> + + + + <div class="wy-nav-content"> + <div class="rst-content"> + <div role="navigation" aria-label="breadcrumbs navigation"> + <ul class="wy-breadcrumbs"> + <li><a href="../index.html">Docs</a> »</li> + + <li><a href="../tsql.html">Tajo Shell (TSQL)</a> »</li> + + <li>Executing a single command</li> + <li class="wy-breadcrumbs-aside"> + + <a href="../_sources/tsql/single_command.txt" rel="nofollow"> View page source</a> + + </li> + </ul> + <hr/> +</div> + <div role="main"> + + <div class="section" id="executing-a-single-command"> +<h1>Executing a single command<a class="headerlink" href="#executing-a-single-command" title="Permalink to this headline">¶</a></h1> +<p>You may want to run more queries without entering tsql prompt. Tsql provides the <code class="docutils literal"><span class="pre">-c</span></code> argument for above requirement. And Tajo assumes that queries are separated by semicolon as follows:</p> +<div class="highlight-sql"><div class="highlight"><pre>$ bin/tsql -c "select count(*) from table1; select sum(score) from table1;" +Progress: 0%, response time: 0.217 sec +Progress: 0%, response time: 0.218 sec +Progress: 100%, response time: 0.317 sec +?count +------------------------------- +5 +(1 rows, 0.317 sec, 2 B selected) +Progress: 0%, response time: 0.202 sec +Progress: 0%, response time: 0.204 sec +Progress: 100%, response time: 0.345 sec +?sum +------------------------------- +15.0 +(1 rows, 0.345 sec, 5 B selected) +</pre></div> +</div> +</div> + + + </div> + <footer> + + <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation"> + + <a href="execute_file.html" class="btn btn-neutral float-right" title="Executing Queries from Files"/>Next <span class="fa fa-arrow-circle-right"></span></a> + + + <a href="intro.html" class="btn btn-neutral" title="Introducing to TSQL"><span class="fa fa-arrow-circle-left"></span> Previous</a> + + </div> + + + <hr/> + + <div role="contentinfo"> + <p> + © Copyright 2014, Apache Tajo Team. + </p> + </div> + + <a href="https://github.com/snide/sphinx_rtd_theme";>Sphinx theme</a> provided by <a href="https://readthedocs.org";>Read the Docs</a> +</footer> + </div> + </div> + + </section> + + </div> + + + + + + <script type="text/javascript"> + var DOCUMENTATION_OPTIONS = { + URL_ROOT:'../', + VERSION:'0.11.0', + COLLAPSE_INDEX:false, + FILE_SUFFIX:'.html', + HAS_SOURCE: true + }; + </script> + <script type="text/javascript" src="../_static/jquery.js"></script> + <script type="text/javascript" src="../_static/underscore.js"></script> + <script type="text/javascript" src="../_static/doctools.js"></script> + + + + + + <script type="text/javascript" src="../_static/js/theme.js"></script> + + + + + <script type="text/javascript"> + jQuery(function () { + SphinxRtdTheme.StickyNav.enable(); + }); + </script> + + +</body> +</html> \ No newline at end of file
Added: tajo/site/docs/0.11.0/tsql/variables.html URL: http://svn.apache.org/viewvc/tajo/site/docs/0.11.0/tsql/variables.html?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/0.11.0/tsql/variables.html (added) +++ tajo/site/docs/0.11.0/tsql/variables.html Tue Oct 27 11:04:33 2015 @@ -0,0 +1,331 @@ + + +<!DOCTYPE html> +<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]--> +<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]--> +<head> + <meta charset="utf-8"> + <meta name="viewport" content="width=device-width, initial-scale=1.0"> + + <title>Session Variables — Apache Tajo 0.11.0 documentation</title> + + + + + + + <link href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700' rel='stylesheet' type='text/css'> + + + + + + + + + + <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" /> + + + + <link rel="top" title="Apache Tajo 0.11.0 documentation" href="../index.html"/> + <link rel="up" title="Tajo Shell (TSQL)" href="../tsql.html"/> + <link rel="next" title="Administration Commands" href="admin_command.html"/> + <link rel="prev" title="Executing HDFS commands" href="dfs_command.html"/> + + + <script src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script> + +</head> + +<body class="wy-body-for-nav" role="document"> + + <div class="wy-grid-for-nav"> + + + <nav data-toggle="wy-nav-shift" class="wy-nav-side"> + <div class="wy-side-nav-search"> + <a href="../index.html" class="fa fa-home"> Apache Tajo</a> + <div role="search"> + <form id ="rtd-search-form" class="wy-form" action="../search.html" method="get"> + <input type="text" name="q" placeholder="Search docs" /> + <input type="hidden" name="check_keywords" value="yes" /> + <input type="hidden" name="area" value="default" /> + </form> +</div> + </div> + + <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation"> + + + <ul class="current"> +<li class="toctree-l1"><a class="reference internal" href="../introduction.html">Introduction</a></li> +<li class="toctree-l1"><a class="reference internal" href="../getting_started.html">Getting Started</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#prerequisites">Prerequisites</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#dowload-and-unpack-the-source-code">Dowload and unpack the source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#build-source-code">Build source code</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#setting-up-a-local-tajo-cluster">Setting up a local Tajo cluster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../getting_started.html#first-query-execution">First query execution</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../configuration.html">Configuration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../configuration/preliminary.html">Preliminary</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/cluster_setup.html">Cluster Setup</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo_master_configuration.html">Tajo Master Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/worker_configuration.html">Worker Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog_configuration.html">Catalog Configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/ha_configuration.html">High Availability for TajoMaster</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +</ul> +</li> +<li class="toctree-l1 current"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul class="current"> +<li class="toctree-l2"><a class="reference internal" href="meta_command.html">Meta Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="dfs_command.html">Executing HDFS commands</a></li> +<li class="toctree-l2 current"><a class="current reference internal" href="">Session Variables</a></li> +<li class="toctree-l2"><a class="reference internal" href="admin_command.html">Administration Commands</a></li> +<li class="toctree-l2"><a class="reference internal" href="intro.html">Introducing to TSQL</a></li> +<li class="toctree-l2"><a class="reference internal" href="single_command.html">Executing a single command</a></li> +<li class="toctree-l2"><a class="reference internal" href="execute_file.html">Executing Queries from Files</a></li> +<li class="toctree-l2"><a class="reference internal" href="background_command.html">Executing as background process</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../sql_language.html">SQL Language</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/data_model.html">Data Model</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/ddl.html">Data Definition Language</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/alter_table.html">ALTER TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/queries.html">Queries</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#server-cluster-time-zone">Server Cluster Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#table-time-zone">Table Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#client-time-zone">Client Time Zone</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#time-zone-id">Time Zone ID</a></li> +<li class="toctree-l2"><a class="reference internal" href="../time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../functions.html">Functions</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-scalar-functions">Built-in Scalar Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-aggregation-functions">Built-in Aggregation Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#built-in-window-functions">Built-in Window Functions</a></li> +<li class="toctree-l2"><a class="reference internal" href="../functions.html#user-defined-functions">User-defined Functions</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../table_partitioning.html">Table Partitioning</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/intro_to_partitioning.html">Introduction to Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/column_partitioning.html">Column Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/range_partitioning.html">Range Partitioning</a></li> +<li class="toctree-l2"><a class="reference internal" href="../partitioning/hash_partitioning.html">Hash Partitioning</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../storage_plugins.html">Storage Plugin</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/overview.html">Storage Plugin Overview</a></li> +<li class="toctree-l2"><a class="reference internal" href="../storage_plugins/postgresql.html">PostgreSQL Storage Handler</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../index_overview.html">Index (Experimental Feature)</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../index/types.html">Index Types</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/how_to_use.html">How to use index?</a></li> +<li class="toctree-l2"><a class="reference internal" href="../index/future_work.html">Future Works</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../backup_and_restore.html">Backup and Restore</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../backup_and_restore/catalog.html">Backup and Restore Catalog</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../hive_integration.html">Hive Integration</a></li> +<li class="toctree-l1"><a class="reference internal" href="../hbase_integration.html">HBase Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#create-table">CREATE TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#drop-table">DROP TABLE</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE) INTO</a></li> +<li class="toctree-l2"><a class="reference internal" href="../hbase_integration.html#usage">Usage</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../swift_integration.html">OpenStack Swift Integration</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#swift-configuration">Swift configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#hadoop-configurations">Hadoop configurations</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#tajo-configuration">Tajo configuration</a></li> +<li class="toctree-l2"><a class="reference internal" href="../swift_integration.html#querying-on-swift">Querying on Swift</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../jdbc_driver.html">Tajo JDBC Driver</a><ul> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC driver</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connecting-to-the-tajo-cluster-instance">Connecting to the Tajo cluster instance</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#connection-parameters">Connection Parameters</a></li> +<li class="toctree-l2"><a class="reference internal" href="../jdbc_driver.html#an-example-jdbc-client">An Example JDBC Client</a></li> +</ul> +</li> +<li class="toctree-l1"><a class="reference internal" href="../tajo_client_api.html">Tajo Client API</a></li> +<li class="toctree-l1"><a class="reference internal" href="../faq.html">FAQ</a></li> +</ul> + + + </div> + + </nav> + + <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> + + + <nav class="wy-nav-top" role="navigation" aria-label="top navigation"> + <i data-toggle="wy-nav-top" class="fa fa-bars"></i> + <a href="../index.html">Apache Tajo</a> + </nav> + + + + <div class="wy-nav-content"> + <div class="rst-content"> + <div role="navigation" aria-label="breadcrumbs navigation"> + <ul class="wy-breadcrumbs"> + <li><a href="../index.html">Docs</a> »</li> + + <li><a href="../tsql.html">Tajo Shell (TSQL)</a> »</li> + + <li>Session Variables</li> + <li class="wy-breadcrumbs-aside"> + + <a href="../_sources/tsql/variables.txt" rel="nofollow"> View page source</a> + + </li> + </ul> + <hr/> +</div> + <div role="main"> + + <div class="section" id="session-variables"> +<h1>Session Variables<a class="headerlink" href="#session-variables" title="Permalink to this headline">¶</a></h1> +<p>Each client connection to TajoMaster creates a unique session, and the client and TajoMaster uses the session until disconnect. A session provides session variables which are used for various configs per session.</p> +<p><code class="docutils literal"><span class="pre">tsql</span></code> provides the meta command <code class="docutils literal"><span class="pre">\set</span></code> to manipulate session variables. Just <code class="docutils literal"><span class="pre">\set</span></code> command shows all session variables.</p> +<div class="highlight-python"><div class="highlight"><pre>default> \set +'name1'='val1' +'name2'='val2' +'name3'='val3' + ... +</pre></div> +</div> +<p><code class="docutils literal"><span class="pre">\set</span> <span class="pre">key</span> <span class="pre">val</span></code> will set the session variable named <em>key</em> with the value <em>val</em>.</p> +<div class="highlight-python"><div class="highlight"><pre>default> \set +'CURRENT_DATABASE'='default' + +default> \set key1 val1 + +default> \set +'CURRENT_DATABASE'='default' +'key1'='val1' +</pre></div> +</div> +<p>Also, <code class="docutils literal"><span class="pre">\unset</span> <span class="pre">key</span></code> will unset the session variable named <em>key</em>.</p> +<p>Now, tajo provides the following session variables.</p> +<ul class="simple"> +<li><code class="docutils literal"><span class="pre">DIST_QUERY_BROADCAST_JOIN_THRESHOLD</span></code></li> +<li><code class="docutils literal"><span class="pre">DIST_QUERY_JOIN_TASK_VOLUME</span></code></li> +<li><code class="docutils literal"><span class="pre">DIST_QUERY_SORT_TASK_VOLUME</span></code></li> +<li><code class="docutils literal"><span class="pre">DIST_QUERY_GROUPBY_TASK_VOLUME</span></code></li> +<li><code class="docutils literal"><span class="pre">DIST_QUERY_JOIN_PARTITION_VOLUME</span></code></li> +<li><code class="docutils literal"><span class="pre">DIST_QUERY_GROUPBY_PARTITION_VOLUME</span></code></li> +<li><code class="docutils literal"><span class="pre">DIST_QUERY_TABLE_PARTITION_VOLUME</span></code></li> +<li><code class="docutils literal"><span class="pre">EXECUTOR_EXTERNAL_SORT_BUFFER_SIZE</span></code></li> +<li><code class="docutils literal"><span class="pre">EXECUTOR_HASH_JOIN_SIZE_THRESHOLD</span></code></li> +<li><code class="docutils literal"><span class="pre">EXECUTOR_INNER_HASH_JOIN_SIZE_THRESHOLD</span></code></li> +<li><code class="docutils literal"><span class="pre">EXECUTOR_OUTER_HASH_JOIN_SIZE_THRESHOLD</span></code></li> +<li><code class="docutils literal"><span class="pre">EXECUTOR_GROUPBY_INMEMORY_HASH_THRESHOLD</span></code></li> +<li><code class="docutils literal"><span class="pre">MAX_OUTPUT_FILE_SIZE</span></code></li> +<li><code class="docutils literal"><span class="pre">CODEGEN</span></code></li> +<li><code class="docutils literal"><span class="pre">CLIENT_SESSION_EXPIRY_TIME</span></code></li> +<li><code class="docutils literal"><span class="pre">CLI_MAX_COLUMN</span></code></li> +<li><code class="docutils literal"><span class="pre">CLI_NULL_CHAR</span></code></li> +<li><code class="docutils literal"><span class="pre">CLI_PRINT_PAUSE_NUM_RECORDS</span></code></li> +<li><code class="docutils literal"><span class="pre">CLI_PRINT_PAUSE</span></code></li> +<li><code class="docutils literal"><span class="pre">CLI_PRINT_ERROR_TRACE</span></code></li> +<li><code class="docutils literal"><span class="pre">CLI_OUTPUT_FORMATTER_CLASS</span></code></li> +<li><code class="docutils literal"><span class="pre">CLI_ERROR_STOP</span></code></li> +<li><code class="docutils literal"><span class="pre">TIMEZONE</span></code></li> +<li><code class="docutils literal"><span class="pre">DATE_ORDER</span></code></li> +<li><code class="docutils literal"><span class="pre">TEXT_NULL</span></code></li> +<li><code class="docutils literal"><span class="pre">DEBUG_ENABLED</span></code></li> +<li><code class="docutils literal"><span class="pre">BEHAVIOR_ARITHMETIC_ABORT</span></code></li> +<li><code class="docutils literal"><span class="pre">RESULT_SET_FETCH_ROWNUM</span></code></li> +</ul> +</div> + + + </div> + <footer> + + <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation"> + + <a href="admin_command.html" class="btn btn-neutral float-right" title="Administration Commands"/>Next <span class="fa fa-arrow-circle-right"></span></a> + + + <a href="dfs_command.html" class="btn btn-neutral" title="Executing HDFS commands"><span class="fa fa-arrow-circle-left"></span> Previous</a> + + </div> + + + <hr/> + + <div role="contentinfo"> + <p> + © Copyright 2014, Apache Tajo Team. + </p> + </div> + + <a href="https://github.com/snide/sphinx_rtd_theme";>Sphinx theme</a> provided by <a href="https://readthedocs.org";>Read the Docs</a> +</footer> + </div> + </div> + + </section> + + </div> + + + + + + <script type="text/javascript"> + var DOCUMENTATION_OPTIONS = { + URL_ROOT:'../', + VERSION:'0.11.0', + COLLAPSE_INDEX:false, + FILE_SUFFIX:'.html', + HAS_SOURCE: true + }; + </script> + <script type="text/javascript" src="../_static/jquery.js"></script> + <script type="text/javascript" src="../_static/underscore.js"></script> + <script type="text/javascript" src="../_static/doctools.js"></script> + + + + + + <script type="text/javascript" src="../_static/js/theme.js"></script> + + + + + <script type="text/javascript"> + jQuery(function () { + SphinxRtdTheme.StickyNav.enable(); + }); + </script> + + +</body> +</html> \ No newline at end of file Added: tajo/site/docs/current/_sources/functions/agg_func.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/functions/agg_func.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/functions/agg_func.txt (added) +++ tajo/site/docs/current/_sources/functions/agg_func.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,91 @@ +************************************ +Aggregation Functions +************************************ + +.. function:: avg (expression) + + Returns the average of all input values. + + :param expression: + :type expression: int | float + :rtype: float8 + +.. function:: corr (expression1, expression2) + + Returns the coefficient of correlation between a set of number pairs. + + :param expression1: + :param expression2: + :type expression1: int | float + :type expression2: int | float + :rtype: float8 + +.. function:: count() + + Returns the number of input rows. + + :rtype: int8 + +.. function:: last_value(expression) + + Returns the last value of expression. + + :param expression: + :type expression: int | float | date | time | timestamp | text + :rtype: same as parameter data type + +.. function:: max(expression) + + Returns the maximum value of expression. + + :param expression: + :type expression: int | float | date | time | timestamp | text + :rtype: same as parameter data type + +.. function:: min(expression) + + Returns the minimum value of expression. + + :param expression: + :type expression: int | float | date | time | timestamp | text + :rtype: same as parameter data type + +.. function:: stddev_pop(expression) + + Returns the population standard deviation of a set of numbers. + + :param expression: + :type expression: int | float + :rtype: float8 + +.. function:: stddev_samp(expression) + + Returns the sample standard deviation of a set of numbers. + + :param expression: + :type expression: int | float + :rtype: float8 + +.. function:: sum(expression) + + Returns the sum of a set of numbers. + + :param expression: + :type expression: int | float + :rtype: same as parameter data type + +.. function:: var_pop(expression) + + Returns the variance of a set of numbers. + + :param expression: + :type expression: int | float + :rtype: float8 + +.. function:: var_samp(expression) + + Returns the unbiased sample variance of a set of numbers. + + :param expression: + :type expression: int | float + :rtype: float8 \ No newline at end of file Added: tajo/site/docs/current/_sources/functions/data_type_func_and_operators.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/functions/data_type_func_and_operators.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/functions/data_type_func_and_operators.txt (added) +++ tajo/site/docs/current/_sources/functions/data_type_func_and_operators.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,43 @@ +************************************ +Data Type Functions and Operators +************************************ + +.. function:: to_bin(source int4) + + Returns the binary representation of integer. + + :param source: source value + :rtype: text + :example: + + .. code-block:: sql + + select to_bin(22); + > 10110 + +.. function:: to_char(source int8, format text) + + Convert integer to string. + + :param source: source value + :param format: format + :rtype: text + :example: + + .. code-block:: sql + + select to_char(125, '00999'); + > 00125 + +.. function:: to_hex(source int4) + + Convert the argument to hexadecimal. + + :param source: source value + :rtype: text + :example: + + .. code-block:: sql + + select to_hex(15); + > F Added: tajo/site/docs/current/_sources/functions/python.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/functions/python.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/functions/python.txt (added) +++ tajo/site/docs/current/_sources/functions/python.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,159 @@ +****************************** +Python Functions +****************************** + +======================= +User-defined Functions +======================= + +----------------------- +Function registration +----------------------- + +To register Python UDFs, you must install script files in all cluster nodes. +After that, you can register your functions by specifying the paths to those script files in ``tajo-site.xml``. Here is an example of the configuration. + +.. code-block:: xml + + <property> + <name>tajo.function.python.code-dir</name> + <value>/path/to/script1.py,/path/to/script2.py</value> + </property> + +Please note that you can specify multiple paths with ``','`` as a delimiter. Each file can contain multiple functions. Here is a typical example of a script file. + +.. code-block:: python + + # /path/to/udf1.py + + @output_type('int4') + def return_one(): + return 1 + + @output_type("text") + def helloworld(): + return 'Hello, World' + + # No decorator - blob + def concat_py(str): + return str+str + + @output_type('int4') + def sum_py(a,b): + return a+b + +If the configuration is set properly, every function in the script files are registered when the Tajo cluster starts up. + +----------------------- +Decorators and types +----------------------- + +By default, every function has a return type of ``BLOB``. +You can use Python decorators to define output types for the script functions. Tajo can figure out return types from the annotations of the Python script. + +* ``output_type``: Defines the return data type for a script UDF in a format that Tajo can understand. The defined type must be one of the types supported by Tajo. For supported types, please refer to :doc:`/sql_language/data_model`. + +----------------------- +Query example +----------------------- + +Once the Python UDFs are successfully registered, you can use them as other built-in functions. + +.. code-block:: sql + + default> select concat_py(n_name)::text from nation where sum_py(n_regionkey,1) > 2; + +============================================== +User-defined Aggregation Functions +============================================== + +----------------------- +Function registration +----------------------- + +To define your Python aggregation functions, you should write Python classes for each function. +Followings are typical examples of Python UDAFs. + +.. code-block:: python + + # /path/to/udaf1.py + + class AvgPy: + sum = 0 + cnt = 0 + + def __init__(self): + self.reset() + + def reset(self): + self.sum = 0 + self.cnt = 0 + + # eval at the first stage + def eval(self, item): + self.sum += item + self.cnt += 1 + + # get intermediate result + def get_partial_result(self): + return [self.sum, self.cnt] + + # merge intermediate results + def merge(self, list): + self.sum += list[0] + self.cnt += list[1] + + # get final result + @output_type('float8') + def get_final_result(self): + return self.sum / float(self.cnt) + + + class CountPy: + cnt = 0 + + def __init__(self): + self.reset() + + def reset(self): + self.cnt = 0 + + # eval at the first stage + def eval(self): + self.cnt += 1 + + # get intermediate result + def get_partial_result(self): + return self.cnt + + # merge intermediate results + def merge(self, cnt): + self.cnt += cnt + + # get final result + @output_type('int4') + def get_final_result(self): + return self.cnt + + +These classes must provide ``reset()``, ``eval()``, ``merge()``, ``get_partial_result()``, and ``get_final_result()`` functions. + +* ``reset()`` resets the aggregation state. +* ``eval()`` evaluates input tuples in the first stage. +* ``merge()`` merges intermediate results of the first stage. +* ``get_partial_result()`` returns intermediate results of the first stage. Output type must be same with the input type of ``merge()``. +* ``get_final_result()`` returns the final aggregation result. + +----------------------- +Query example +----------------------- + +Once the Python UDAFs are successfully registered, you can use them as other built-in aggregation functions. + +.. code-block:: sql + + default> select avgpy(n_nationkey), countpy() from nation; + +.. warning:: + + Currently, Python UDAFs cannot be used as window functions. \ No newline at end of file Added: tajo/site/docs/current/_sources/functions/window_func.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/functions/window_func.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/functions/window_func.txt (added) +++ tajo/site/docs/current/_sources/functions/window_func.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,47 @@ +************************************ +Window Functions +************************************ + +.. function:: first_value (value any) + + Returns the first value of input rows. + + :param value: input value + :rtype: same as parameter data type + +.. function:: last_value (value any) + + Returns the last value of input rows. + + :param value: input value + :rtype: same as parameter data type + +.. function:: lag (value any [, offset integer [, default any ]]) + + Returns value evaluated at the row that is offset rows before the current row within the partition. If there is no such row, instead return default. Both offset and default are evaluated with respect to the current row. If omitted, offset defaults to 1 and default to null. + + :param value: input value + :param offset: offset + :param default: default value + :rtype: same as parameter data type + +.. function:: lead (value any [, offset integer [, default any ]]) + + Returns value evaluated at the row that is offset rows after the current row within the partition. If there is no such row, instead return default. Both offset and default are evaluated with respect to the current row. If omitted, offset defaults to 1 and default to null. + + :param value: input value + :param offset: offset + :param default: default value + :rtype: same as parameter data type + +.. function:: rank () + + Returns rank of the current row with gaps. + + :rtype: int8 + +.. function:: row_number () + + Returns the current row within its partition, counting from 1. + + :rtype: int8 \ No newline at end of file Added: tajo/site/docs/current/_sources/hive_integration.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/hive_integration.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/hive_integration.txt (added) +++ tajo/site/docs/current/_sources/hive_integration.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,42 @@ +************************************* +Hive Integration +************************************* + +Apache Tajo⢠catalog supports HiveCatalogStore to integrate with Apache Hiveâ¢. +This integration allows Tajo to access all tables used in Apache Hive. +Depending on your purpose, you can execute either SQL queries or HiveQL queries on the +same tables managed in Apache Hive. + +In order to use this feature, you need to build Tajo with a specified maven profile +and then add some configs into ``conf/tajo-env.sh`` and ``conf/catalog-site.xml``. +This section describes how to setup HiveMetaStore integration. +This instruction would take no more than five minutes. + +You need to set your Hive home directory to the environment variable ``HIVE_HOME`` in conf/tajo-env.sh as follows: :: + + export HIVE_HOME=/path/to/your/hive/directory + +If you need to use jdbc to connect HiveMetaStore, you have to prepare MySQL jdbc driver. +Next, you should set the path of MySQL JDBC driver jar file to the environment variable HIVE_JDBC_DRIVER_DIR in conf/tajo-env.sh as follows: :: + + export HIVE_JDBC_DRIVER_DIR==/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar + +Finally, you should specify HiveCatalogStore as Tajo catalog driver class in ``conf/catalog-site.xml`` as follows: :: + + <property> + <name>tajo.catalog.store.class</name> + <value>org.apache.tajo.catalog.store.HiveCatalogStore</value> + </property> + +.. note:: + + Hive stores a list of partitions for each table in its metastore. If new partitions are + directly added to HDFS, HiveMetastore will not able aware of these partitions unless the user + ``ALTER TABLE table_name ADD PARTITION`` commands on each of the newly added partitions or + ``MSCK REPAIR TABLE table_name`` command. + + But current tajo doesn't provide ``ADD PARTITION`` command and hive doesn't provide an api for + responding to ``MSK REPAIR TABLE`` command. Thus, if you insert data to hive partitioned + table and you want to scan the updated partitions through Tajo, you must run following command on hive :: + + $ MSCK REPAIR TABLE [table_name]; Added: tajo/site/docs/current/_sources/sql_language/alter_table.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/sql_language/alter_table.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/sql_language/alter_table.txt (added) +++ tajo/site/docs/current/_sources/sql_language/alter_table.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,100 @@ +************************ +ALTER TABLE +************************ + +======================== +RENAME TABLE +======================== + +*Synopsis* + +.. code-block:: sql + + ALTER TABLE <table_name> RENAME TO <new_table_name> + + For example: + ALTER TABLE table1 RENAME TO table2; + +This statement lets you change the name of a table to a different name. + +======================== +RENAME COLUMN +======================== + +*Synopsis* + +.. code-block:: sql + + ALTER TABLE <table_name> RENAME COLUMN <column_name> TO <new_column_name> + + For example: + ALTER TABLE table1 RENAME COLUMN id TO id2; + +This statement will allow users to change a column's name. + +======================== +ADD COLUMN +======================== + +*Synopsis* + +.. code-block:: sql + + ALTER TABLE <table_name> ADD COLUMN <column_name> <data_type> + + For example: + ALTER TABLE table1 ADD COLUMN id text; + +This statement lets you add new columns to the end of the existing column. + +======================== +SET PROPERTY +======================== + +*Synopsis* + +.. code-block:: sql + + ALTER TABLE <table_name> SET PROPERTY (<key> = <value>, ...) + + For example: + ALTER TABLE table1 SET PROPERTY 'timezone' = 'GMT-7' + ALTER TABLE table1 SET PROPERTY 'text.delimiter' = '&' + ALTER TABLE table1 SET PROPERTY 'compression.type'='RECORD','compression.codec'='org.apache.hadoop.io.compress.SnappyCodec' + + +This statement will allow users to change a table property. + +======================== + DROP PARTITION +======================== + +*Synopsis* + +.. code-block:: sql + + ALTER TABLE <table_name> [IF EXISTS] DROP PARTITION (<partition column> = <partition value>, ...) [PURGE] + + For example: + ALTER TABLE table1 DROP PARTITION (col1 = 1 , col2 = 2) + ALTER TABLE table1 DROP PARTITION (col1 = '2015' , col2 = '01', col3 = '11' ) + ALTER TABLE table1 DROP PARTITION (col1 = 'TAJO' ) PURGE + +You can use ``ALTER TABLE DROP PARTITION`` to drop a partition for a table. This doesn't remove the data for a table. But if ``PURGE`` is specified, the partition data will be removed. The metadata is completely lost in all cases. An error is thrown if the partition for the table doesn't exist. You can use ``IF EXISTS`` to skip the error. + +======================== +REPAIR PARTITION +======================== + +Tajo stores a list of partitions for each table in its catalog. If partitions are manually added to the distributed file system, the metastore is not aware of these partitions. Running the ``ALTER TABLE REPAIR PARTITION`` statement ensures that the tables are properly populated. + +*Synopsis* + +.. code-block:: sql + + ALTER TABLE <table_name> REPAIR PARTITION + +.. note:: + + Even though an information of a partition is stored in the catalog, Tajo does not recover it when its partition directory doesn't exist in the file system. + Added: tajo/site/docs/current/_sources/sql_language/joins.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/sql_language/joins.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/sql_language/joins.txt (added) +++ tajo/site/docs/current/_sources/sql_language/joins.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,160 @@ +************************** +Joins +************************** + +===================== +Overview +===================== + +In Tajo, a single query can accesses multiple rows of the same or different relations at one time. This query is called *join*. +Currently, Tajo supports cross, inner, and outer joins. + +A join query can involve multiple relations in the ``FROM`` clause according to the following rule. + +.. code-block:: sql + + FROM joined_table [, joined_table [, ...] ] + +, where ``joined_table`` is: + +.. code-block:: sql + + table_reference join_type table_reference [ ON join_condition ] + +``join_type`` can be one of the followings. + +.. code-block:: sql + + CROSS JOIN + [ NATURAL ] [ INNER ] JOIN + { LEFT | RIGHT | FULL } OUTER JOIN + +``join_condition`` can be specified in the ``WHERE`` clause as well as the ``ON`` clause. + +For more information, please refer to :doc:`/sql_language/predicates`. + +.. note:: + + Currently, Tajo cannot natively support non-equality conditions. It means that inner joins with non-equality conditions will be executed with cross joins. Outer joins with non-equality conditions cannot be executed yet. + +===================== +Examples +===================== + +* For inner and outer joins, only equality conditions are allowed as follows. For inner joins, implicit join notation is allowed. + +.. code-block:: sql + + SELECT a.* FROM a, b WHERE a.id = b.id + + SELECT a.* FROM a JOIN b ON (a.id = b.id) + + SELECT a.* FROM a LEFT OUTER JOIN b ON (a.id = b.id AND a.type = b.type) + +However, the following query will be executed with CROSS join, thereby taking a very long time. + +.. code-block:: sql + + SELECT a.* FROM a JOIN b ON (a.id <> b.id) + +In addition, the following query is not allowed. + +.. code-block:: sql + + SELECT a.* FROM a LEFT OUTER JOIN b ON (a.id > b.id) + +* You can join more than 2 tables in a query with multiple join types. + +.. code-block:: sql + + SELECT a.* FROM a, b, c WHERE a.id = b.id AND b.id2 = c.id2 + + SELECT a.* FROM a INNER JOIN b ON a.id = b.id FULL OUTER JOIN c ON b.id2 = c.id2 + +When a query involves three or more tables, there may be a lot of possible join orders. Tajo automatically finds the best join order regardless of the input order. For example, suppose that relation ``b`` is larger than relation ``a``, and in turn, the relation ``c`` is larger than relation ``b``. The query + +.. code-block:: sql + + SELECT a.* FROM c INNER JOIN b ON b.id2 = c.id2 INNER JOIN a ON a.id = b.id + +is rewritten to + +.. code-block:: sql + + SELECT a.* FROM a INNER JOIN b ON a.id = b.id INNER JOIN c ON b.id2 = c.id2 + +because early join of small relations accelerates the query speed. + +* Tajo also supports natural join. When relations have a common column name, they are joined with an equality condition on that column even though it is not explicitly declared in the query. For example, + +.. code-block:: sql + + SELECT a.* FROM a JOIN b + +is rewritten to + +.. code-block:: sql + + SELECT a.* FROM a INNER JOIN b ON a.id = b.id + +===================== +Join Optimization +===================== + +Join is one of the most expensive operations in relational world. +Tajo adopts several optimization techniques to improve its join performance. + +--------------------- +Join ordering +--------------------- + +Join ordering is one of the important techniques for join performance improvement. +Basically, joining multiple relations is left-associative. However, query performance can be significantly changed according to which order is chosen for the join execution. + +To find the best join order, Tajo's cost-based optimizer considers join conditions, join types, and the size of input relations. +In addition, it considers the computation cost of consecutive joins so that the shape of query plan forms a bushy tree. + +For example, suppose that there are 4 relations ``a`` (10), ``b`` (20), ``c`` (30), and ``d`` (40) where the numbers within brackets represent the relation size. +The following query + +.. code-block:: sql + + SELECT + * + FROM + a, b, c, d + WHERE + a.id1 = b.id1 AND + a.id4 = d.id4 AND + b.id2 = c.id2 AND + c.id3 = d.id3 + +is rewritten into + +.. code-block:: sql + + SELECT + * + FROM + (a INNER JOIN d ON a.id4 = d.id4) + INNER JOIN + (b INNER JOIN c ON b.id2 = c.id2) + ON a.id1 = b.id1 AND c.id3 = d.id3 + + +--------------------- +Broadcast join +--------------------- + +In Tajo, a join query is executed in two stages. The first stage is responsible for scanning input data and performing local join, while the second stage is responsible for performing global join and returning the result. +To perform join globally in the second stage, intermediate result of the first stage is exchanged according to join keys, i.e., *shuffled*, among Tajo workers. +Here, the cost of shuffle is expensive especially when the input relation size is very small. + +Broadcast join is a good solution to handle this problem. In broadcast join, the small relations are replicated to every worker who participates in the join computation. +Thus, they can perform join without expensive data shuffle. + +Tajo provides a session variable for broadcast join configuration. (For more detailed information of session variables, please refer to :doc:`/tsql/variables`.) + +* ``DIST_QUERY_BROADCAST_JOIN_THRESHOLD`` is a threshold for broadcast join. Only the relations who are larger than this value can be broadcasted. + +You can also apply this configuration system widely by setting ``tajo.dist-query.join.broadcast.threshold-bytes`` in ``${TAJO_HOME}/conf/tajo-site.xml``. \ No newline at end of file Added: tajo/site/docs/current/_sources/storage_plugin.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/storage_plugin.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/storage_plugin.txt (added) +++ tajo/site/docs/current/_sources/storage_plugin.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,47 @@ +************************************* +Storage Plugin +************************************* + +Overview +======== + +Tajo supports various storage systems, such as HDFS, Amazon S3, Openstack Swift, and HBase. Also, we have a plan to support RDBMS storages like Oracle, MySQL, PostgreSQL. Tajo already embeds HDFS, S3, Openstack, and HBase, and also Tajo allows users to register custom storages and data formats to Tajo cluster instances. This section describes how you register custom storages and data types. + +Register custom storage +======================= + +First of all, your storage implementation should be packed as a jar file. Then, please copy the jar file into ``tajo/extlib`` directory. Next, you should copy ``conf/storage-site.json.template`` into ``conf/storage-site.json`` and modify the file like the below. + +Configuration +============= + +Tajo has a default configuration for builtin storages, such as HDFS, local file system, and Amazon S3. it also allows users to add custom storage plugins + +``conf/storage-site.json`` file has the following struct: + +.. code-block:: json + + { + "storages": { + "${scheme}": { + "handler": "${class name}" + } + } + } + +Each storage instance (i.e., :doc:`/table_management/tablespaces`) is identified by an URI. The scheme of URI plays a role to identify storage type. For example, ``hdfs://`` is used for Hdfs storage, ``jdbc://`` is used for JDBC-based storage, and ``hbase://`` is used for HBase storage. + +You should substitute a scheme name without ``://`` for ``${scheme}``. + +See an example for HBase storage. + +.. code-block:: json + + { + "storages": { + "hbase": { + "handler": "org.apache.tajo.storage.hbase.HBaseTablespace", + "default-format": "hbase" + } + } + } \ No newline at end of file Added: tajo/site/docs/current/_sources/storage_plugins.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/storage_plugins.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/storage_plugins.txt (added) +++ tajo/site/docs/current/_sources/storage_plugins.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,11 @@ +************************************* +Storage Plugin +************************************* + +This section describes the storage plugins available in Tajo to access datasets from different data sources. + +.. toctree:: + :maxdepth: 1 + + storage_plugins/overview + storage_plugins/postgresql \ No newline at end of file Added: tajo/site/docs/current/_sources/storage_plugins/overview.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/storage_plugins/overview.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/storage_plugins/overview.txt (added) +++ tajo/site/docs/current/_sources/storage_plugins/overview.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,47 @@ +************************************* +Storage Plugin Overview +************************************* + +Overview +======== + +Tajo supports various storage systems, such as HDFS, Amazon S3, Openstack Swift, HBase, and RDBMS. Tajo already embeds HDFS, S3, Openstack, HBase, RDBMS storage plugins, and also Tajo allows users to register custom storages and data formats to Tajo cluster instances. This section describes how you register custom storages and data types. + +Register custom storage +======================= + +First of all, your storage implementation should be packed as a jar file. Then, please copy the jar file into ``tajo/extlib`` directory. Next, you should copy ``conf/storage-site.json.template`` into ``conf/storage-site.json`` and modify the file like the below. + +Configuration +============= + +Tajo has a default configuration for builtin storages, such as HDFS, local file system, and Amazon S3. it also allows users to add custom storage plugins + +``conf/storage-site.json`` file has the following struct: + +.. code-block:: json + + { + "storages": { + "${scheme}": { + "handler": "${class name}" + } + } + } + +Each storage instance (i.e., :doc:`/table_management/tablespaces`) is identified by an URI. The scheme of URI plays a role to identify storage type. For example, ``hdfs://`` is used for Hdfs storage, ``jdbc://`` is used for JDBC-based storage, and ``hbase://`` is used for HBase storage. + +You should substitute a scheme name without ``://`` for ``${scheme}``. + +See an example for HBase storage. + +.. code-block:: json + + { + "storages": { + "hbase": { + "handler": "org.apache.tajo.storage.hbase.HBaseTablespace", + "default-format": "hbase" + } + } + } \ No newline at end of file Added: tajo/site/docs/current/_sources/storage_plugins/postgresql.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/storage_plugins/postgresql.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/storage_plugins/postgresql.txt (added) +++ tajo/site/docs/current/_sources/storage_plugins/postgresql.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,40 @@ +************************************* +PostgreSQL Storage Handler +************************************* + +Overview +======== + +PostgreSQL storage handler is available by default in Tajo. It enables users' queries to access database objects in PostgreSQL. Tables in PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are available in Tajo via this storage handles. Its main advantages is to allow federated query processing among tables in stored HDFS and PostgreSQL. + +Configuration +============= + +PostgreSQL storage handler is a builtin storage handler. So, you can eaisly register PostgreSQL databases to a Tajo cluster if you just add the following line to ``conf/storage-site.json`` file. If you want to know more information about ``storage-site.json``, please refer to :doc:`/table_management/tablespaces`. + +.. code-block:: json + + { + "spaces": { + "pgsql_db1": { + "uri": "jdbc:postgresql://hostname:port/db1" + + "configs": { + "mapped_database": "tajo_db1" + "connection_properties": { + "user": "tajo", + "password": "xxxx" + } + } + } + } + } + +``configs`` allows users to specific additional configurations. +``mapped_database`` specifies a database name shown in Tajo. In the example, the database ``db1`` in PostgreSQL +will be mapped to the database ``tajo_db1`` in Tajo. +``connection_properties`` allows users to set JDBC connection parameters. +Please refer to https://jdbc.postgresql.org/documentation/head/connect.html in order to know the details of +PostgreSQL connection parameters. + +The storage-site.json will be effective after you restart a tajo cluster. \ No newline at end of file Added: tajo/site/docs/current/_sources/table_management/json.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management/json.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/table_management/json.txt (added) +++ tajo/site/docs/current/_sources/table_management/json.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,100 @@ +**** +JSON +**** + +JSON(JavaScript Object Notation) is an open standard format for data (de)serialization. Since it is simple and human-readable, it is popularly used in many fields. +Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them. + +============================ +How to Create a JSON Table ? +============================ + +You can create a JSON table using the ``CREATE TABLE`` statement. (For more information, please refer to :doc:`/sql_language/ddl`.) +For example, please consider an example data as follows: + +.. code-block:: bash + + $ hdfs dfs -cat /table1/table.json + { "title" : "Hand of the King", "name" : { "first_name": "Eddard", "last_name": "Stark"}} + { "title" : "Assassin", "name" : { "first_name": "Arya", "last_name": "Stark"}} + { "title" : "Dancing Master", "name" : { "first_name": "Syrio", "last_name": "Forel"}} + +Tajo provides two ways to create a table for this data. First is a traditional way to create tables. Here is an example. + +.. code-block:: sql + + CREATE EXTERNAL TABLE table1 ( + title TEXT, + name RECORD ( + first_name TEXT, + last_name TEXT + ) + ) USING JSON LOCATION '/table1/table.json'; + +With this way, you need to specify every column which they want to use. This will be a tedious work, and not appropriate for flexible JSON schema. +Second is a simpler alternative to alleviate this problem. When you create an external table of JSON format, you can simply omit the column specification as follows: + +.. code-block:: sql + + CREATE EXTERNAL TABLE table1 (*) USING JSON LOCATION '/table1/table.json'; + +No matter which way you choose, you can submit any queries on this table. + +.. code-block:: sql + + > SELECT title, name.last_name from table1 where name.first_name = 'Arya'; + title,name/last_name + ------------------------------- + Assassin,Stark + +.. warning:: + + If you create a table with the second way, every column is assumed as the ``TEXT`` type. + So, you need to perform type casting if you want to handle them as other types. + +=================== +Physical Properties +=================== + +Some table storage formats provide parameters for enabling or disabling features and adjusting physical parameters. +The ``WITH`` clause in the CREATE TABLE statement allows users to set those parameters. + +The JSON format provides the following physical properties. + +* ``text.delimiter``: delimiter character. ``|`` or ``\u0001`` is usually used, and the default field delimiter is ``|``. +* ``text.null``: ``NULL`` character. The default ``NULL`` character is an empty string ``''``. Hive's default ``NULL`` character is ``'\\N'``. +* ``compression.codec``: Compression codec. You can enable compression feature and set specified compression algorithm. The compression algorithm used to compress files. The compression codec name should be the fully qualified class name inherited from `org.apache.hadoop.io.compress.CompressionCodec <https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html>`_. By default, compression is disabled. +* ``timezone``: the time zone that the table uses for writting. When table rows are read or written, ```timestamp``` and ```time``` column values are adjusted by this timezone if it is set. Time zone can be an abbreviation form like 'PST' or 'DST'. Also, it accepts an offset-based form like 'UTC+9' or a location-based form like 'Asia/Seoul'. +* ``text.error-tolerance.max-num``: the maximum number of permissible parsing errors. This value should be an integer value. By default, ``text.error-tolerance.max-num`` is ``0``. According to the value, parsing errors will be handled in different ways. + + * If ``text.error-tolerance.max-num < 0``, all parsing errors are ignored. + * If ``text.error-tolerance.max-num == 0``, any parsing error is not allowed. If any error occurs, the query will be failed. (default) + * If ``text.error-tolerance.max-num > 0``, the given number of parsing errors in each task will be pemissible. + +* ``text.skip.headerlines``: Number of header lines to be skipped. Some text files often have a header which has a kind of metadata(e.g.: column names), thus this option can be useful. + +The following example is to set a custom field delimiter, ``NULL`` character, and compression codec: + +.. code-block:: sql + + CREATE TABLE table1 ( + id int, + name text, + score float, + type text + ) USING JSON WITH('text.delimiter'='\u0001', + 'text.null'='\\N', + 'compression.codec'='org.apache.hadoop.io.compress.SnappyCodec'); + +.. warning:: + + Be careful when using ``\n`` as the field delimiter because *TEXT* format tables use ``\n`` as the line delimiter. + At the moment, Tajo does not provide a way to specify the line delimiter. + +========================== +Null Value Handling Issues +========================== +In default, ``NULL`` character in *TEXT* format is an empty string ``''``. +In other words, an empty field is basically recognized as a ``NULL`` value in Tajo. +If a field domain is ``TEXT``, an empty field is recognized as a string value ``''`` instead of ``NULL`` value. +Besides, You can also use your own ``NULL`` character by specifying a physical property ``text.null``. Added: tajo/site/docs/current/_sources/table_management/orc.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management/orc.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/table_management/orc.txt (added) +++ tajo/site/docs/current/_sources/table_management/orc.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,47 @@ +*** +ORC +*** + +**ORC(Optimized Row Columnar)** is a columnar storage format from Hive. ORC improves performance for reading, +writing, and processing data. +For more details, please refer to `ORC Files <https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC>`_ at Hive wiki. + +=========================== +How to Create an ORC Table? +=========================== + +If you are not familiar with ``CREATE TABLE`` statement, please refer to Data Definition Language :doc:`/sql_language/ddl`. + +In order to specify a certain file format for your table, you need to use the ``USING`` clause in your ``CREATE TABLE`` +statement. Below is an example statement for creating a table using orc files. + +.. code-block:: sql + + CREATE TABLE table1 ( + id int, + name text, + score float, + type text + ) USING orc; + +=================== +Physical Properties +=================== + +Some table storage formats provide parameters for enabling or disabling features and adjusting physical parameters. +The ``WITH`` clause in the CREATE TABLE statement allows users to set those parameters. + +Now, ORC file provides the following physical properties. + +* ``orc.max.merge.distance``: When ORC file is read, if stripes are too closer and the distance is lower than this value, they are merged and read at once. Default is 1MB. +* ``orc.stripe.size``: It decides size of each stripe. Default is 64MB. +* ``orc.compression.kind``: It means the compression algorithm used to compress and write data. It should be one of ``none``, ``snappy``, ``zlib``. Default is ``none``. +* ``orc.buffer.size``: It decides size of writing buffer. Default is 256KB. +* ``orc.rowindex.stride``: Define the default ORC index stride in number of rows. (Stride is the number of rows an index entry represents.) Default is 10000. + +====================================== +Compatibility Issues with Apache Hive⢠+====================================== + +At the moment, Tajo only supports flat relational tables. +We are currently working on adding support for nested schemas and non-scalar types (`TAJO-710 <https://issues.apache.org/jira/browse/TAJO-710>`_). \ No newline at end of file Added: tajo/site/docs/current/_sources/table_management/tablespaces.txt URL: http://svn.apache.org/viewvc/tajo/site/docs/current/_sources/table_management/tablespaces.txt?rev=1710773&view=auto ============================================================================== --- tajo/site/docs/current/_sources/table_management/tablespaces.txt (added) +++ tajo/site/docs/current/_sources/table_management/tablespaces.txt Tue Oct 27 11:04:33 2015 @@ -0,0 +1,45 @@ +************************************* +Tablespaces +************************************* + +Tablespaces in Tajo allow users to define locations in the storage system where the files or data objects representing database objects can be stored. Once defined, a tablespace can be referred to by name when creating a database or a table. Especially, it is very useful when a Tajo cluster instance should use heterogeneous storage systems such as HDFS, MySQL, and Oracle. + +Configuration +============= + +By default, Tajo use in ``${tajo.rootdir}/warehouse`` in ``conf/tajo-site.xml`` as a default tablespace. It also allows users to register additional tablespaces. + +``conf/storage-site.json`` file. + +The configuration file has the following struct: + +.. code-block:: json + + { + "spaces": { + "${table_space_name}": { + "uri": "hbase://quorum1:port,quorum2:port/" + } + } + } + +The following is an example for two tablespaces for hbase and hdfs: + +.. code-block:: json + + { + "spaces": { + "hbase-cluster1": { + "uri": "hbase://quorum1:port,quorum2:port/" + }, + + "ssd": { + "uri": "hdfs://host:port/data/ssd" + } + } + } + + +.. note:: + + Also, each tablespace can use different storage type. Please see :doc:`/storage_plugins` if you want to know more information about it. \ No newline at end of file
