Modified: tajo/site/docs/current/getting_started.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/getting_started.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/getting_started.html (original) +++ tajo/site/docs/current/getting_started.html Thu Feb 4 00:29:05 2016 @@ -77,6 +77,7 @@ <li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -99,6 +100,7 @@ <li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -119,7 +121,7 @@ <li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> </ul> </li> @@ -244,9 +246,9 @@ Download the file that ends in .tar.gz t <p>You prepare the prerequisites and the source code, you can build the source code now.</p> <p>The first step of the installation procedure is to configure the source tree for your system and choose the options you would like. This is done by running the configure script. For a default installation simply enter:</p> <p>You can compile source code and get a binary archive as follows:</p> -<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span><span class="nb">cd </span>tajo-x.y.z -<span class="nv">$ </span>mvn clean install -DskipTests -Pdist -Dtar -Dhadoop.version<span class="o">=</span>2.X.X -<span class="nv">$ </span>ls tajo-dist/target/tajo-x.y.z-SNAPSHOT.tar.gz +<div class="highlight-bash"><div class="highlight"><pre>$ <span class="nb">cd</span> tajo-x.y.z +$ mvn clean install -DskipTests -Pdist -Dtar -Dhadoop.version<span class="o">=</span>2.X.X +$ ls tajo-dist/target/tajo-x.y.z-SNAPSHOT.tar.gz </pre></div> </div> <div class="admonition note"> @@ -257,8 +259,8 @@ Download the file that ends in .tar.gz t <div>$ mvn clean install -DskipTests -Pdist -Dtar -Dhadoop.version=2.5.1</div></blockquote> </div> <p>Then, after you move some proper directory, discompress the tar.gz file as follows:</p> -<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span><span class="nb">cd</span> <span class="o">[</span>a directory to be parent of tajo binary<span class="o">]</span> -<span class="nv">$ </span>tar xzvf <span class="si">${</span><span class="nv">TAJO_SRC</span><span class="si">}</span>/tajo-dist/target/tajo-x.y.z-SNAPSHOT.tar.gz +<div class="highlight-bash"><div class="highlight"><pre>$ <span class="nb">cd</span> <span class="o">[</span>a directory to be parent of tajo binary<span class="o">]</span> +$ tar xzvf <span class="si">${</span><span class="nv">TAJO_SRC</span><span class="si">}</span>/tajo-dist/target/tajo-x.y.z-SNAPSHOT.tar.gz </pre></div> </div> </div> @@ -266,15 +268,15 @@ Download the file that ends in .tar.gz t <h2>Setting up a local Tajo cluster<a class="headerlink" href="#setting-up-a-local-tajo-cluster" title="Permalink to this headline">¶</a></h2> <p>Apache Tajo⢠provides two run modes: local mode and fully distributed mode. Here, we explain only the local mode where a Tajo instance runs on a local file system. A local mode Tajo instance can start up with very simple configurations.</p> <p>First of all, you need to add the environment variables to conf/tajo-env.sh.</p> -<div class="highlight-bash"><div class="highlight"><pre><span class="c"># Hadoop home. Required</span> -<span class="nb">export </span><span class="nv">HADOOP_HOME</span><span class="o">=</span> ... +<div class="highlight-bash"><div class="highlight"><pre><span class="c1"># Hadoop home. Required</span> +<span class="nb">export</span> <span class="nv">HADOOP_HOME</span><span class="o">=</span> ... -<span class="c"># The java implementation to use. Required.</span> -<span class="nb">export </span><span class="nv">JAVA_HOME</span><span class="o">=</span> ... +<span class="c1"># The java implementation to use. Required.</span> +<span class="nb">export</span> <span class="nv">JAVA_HOME</span><span class="o">=</span> ... </pre></div> </div> <p>To launch the tajo master, execute start-tajo.sh.</p> -<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ $TAJO_HOME</span>/bin/start-tajo.sh +<div class="highlight-bash"><div class="highlight"><pre>$ $TAJO_HOME/bin/start-tajo.sh </pre></div> </div> <div class="admonition note"> @@ -289,9 +291,9 @@ Download the file that ends in .tar.gz t <div class="section" id="first-query-execution"> <h2>First query execution<a class="headerlink" href="#first-query-execution" title="Permalink to this headline">¶</a></h2> <p>First of all, we need to prepare some table for query execution. For example, you can make a simple text-based table as follows:</p> -<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>mkdir /home/x/table1 -<span class="nv">$ </span><span class="nb">cd</span> /home/x/table1 -<span class="nv">$ </span>cat > data.csv +<div class="highlight-bash"><div class="highlight"><pre>$ mkdir /home/x/table1 +$ <span class="nb">cd</span> /home/x/table1 +$ cat > data.csv 1<span class="p">|</span>abc<span class="p">|</span>1.1<span class="p">|</span>a 2<span class="p">|</span>def<span class="p">|</span>2.3<span class="p">|</span>b 3<span class="p">|</span>ghi<span class="p">|</span>3.4<span class="p">|</span>c @@ -378,7 +380,7 @@ If you want to know more explanation for <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div>
Modified: tajo/site/docs/current/hbase_integration.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/hbase_integration.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/hbase_integration.html (original) +++ tajo/site/docs/current/hbase_integration.html Thu Feb 4 00:29:05 2016 @@ -77,6 +77,7 @@ <li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -99,6 +100,7 @@ <li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -119,7 +121,7 @@ <li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> </ul> </li> @@ -210,11 +212,12 @@ This integration allows Tajo to access all tables used in Apache HBase.</p> <p>In order to use this feature, you need to build add some configs into <code class="docutils literal"><span class="pre">conf/tajo-env.sh</span></code> and then add some properties into a table create statement.</p> <p>This section describes how to setup HBase integration.</p> -<p>First, you need to set your HBase home directory to the environment variable <code class="docutils literal"><span class="pre">HBASE_HOME</span></code> in conf/tajo-env.sh as follows:</p> +<p>First, you need to set your HBase home directory to the environment variable <code class="docutils literal"><span class="pre">HBASE_HOME</span></code> in <code class="docutils literal"><span class="pre">conf/tajo-env.sh</span></code> as follows:</p> <div class="highlight-python"><div class="highlight"><pre>export HBASE_HOME=/path/to/your/hbase/directory </pre></div> </div> <p>If you set the directory, Tajo will add HBase library file to classpath.</p> +<p>Next, you must configure tablespace about HBase. Please see <a class="reference internal" href="table_management/tablespaces.html"><em>Tablespaces</em></a> if you want to know more information about it.</p> <div class="section" id="create-table"> <h2>CREATE TABLE<a class="headerlink" href="#create-table" title="Permalink to this headline">¶</a></h2> <p><em>Synopsis</em></p> @@ -223,10 +226,12 @@ This integration allows Tajo to access a <span class="k">WITH</span> <span class="p">(</span><span class="s1">'table'</span><span class="o">=</span><span class="s1">'<hbase_table_name>'</span> <span class="p">,</span> <span class="s1">'columns'</span><span class="o">=</span><span class="s1">':key,<column_family_name>:<qualifier_name>, ...'</span> <span class="p">,</span> <span class="s1">'hbase.zookeeper.quorum'</span><span class="o">=</span><span class="s1">'<zookeeper_address>'</span> -<span class="p">,</span> <span class="s1">'hbase.zookeeper.property.clientPort'</span><span class="o">=</span><span class="s1">'<zookeeper_client_port>'</span> -<span class="p">)</span> +<span class="p">,</span> <span class="s1">'hbase.zookeeper.property.clientPort'</span><span class="o">=</span><span class="s1">'<zookeeper_client_port>'</span><span class="p">)</span> +<span class="p">[</span><span class="k">LOCATION</span> <span class="s1">'hbase:zk://<hostname>:<port>/'</span><span class="p">]</span> <span class="p">;</span> </pre></div> </div> +<p><code class="docutils literal"><span class="pre">IF</span> <span class="pre">NOT</span> <span class="pre">EXISTS</span></code> allows <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">[EXTERNAL]</span> <span class="pre">TABLE</span></code> statement to avoid an error which occurs when the table does not exist.</p> +<p>If you want to create <code class="docutils literal"><span class="pre">EXTERNAL</span> <span class="pre">TABLE</span></code>, You must write <code class="docutils literal"><span class="pre">LOCATION</span></code> statement.</p> <p>Options</p> <ul class="simple"> <li><code class="docutils literal"><span class="pre">table</span></code> : Set hbase origin table name. If you want to create an external table, the table must exists on HBase. The other way, if you want to create a managed table, the table must doesn’t exist on HBase.</li> @@ -234,7 +239,6 @@ This integration allows Tajo to access a <li><code class="docutils literal"><span class="pre">hbase.zookeeper.quorum</span></code> : Set zookeeper quorum address. You can use different zookeeper cluster on the same Tajo database. If you don’t set the zookeeper address, Tajo will refer the property of hbase-site.xml file.</li> <li><code class="docutils literal"><span class="pre">hbase.zookeeper.property.clientPort</span></code> : Set zookeeper client port. If you don’t set the port, Tajo will refer the property of hbase-site.xml file.</li> </ul> -<p><code class="docutils literal"><span class="pre">IF</span> <span class="pre">NOT</span> <span class="pre">EXISTS</span></code> allows <code class="docutils literal"><span class="pre">CREATE</span> <span class="pre">[EXTERNAL]</span> <span class="pre">TABLE</span></code> statement to avoid an error which occurs when the table does not exist.</p> </div> <div class="section" id="drop-table"> <h2>DROP TABLE<a class="headerlink" href="#drop-table" title="Permalink to this headline">¶</a></h2> @@ -270,7 +274,8 @@ This integration allows Tajo to access a <div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">EXTERNAL</span> <span class="k">TABLE</span> <span class="n">blog</span> <span class="p">(</span><span class="n">rowkey</span> <span class="nb">text</span><span class="p">,</span> <span class="n">author</span> <span class="nb">text</span><span class="p">,</span> <span class="n">register_date</span> <span class="nb">text</span><span class="p">,</span> <span class="n">title</span> <span class="nb">text</span><span class="p">)</span> <span class="k">USING</span> <span class="n">hbase</span> <span class="k">WITH</span> <span class="p">(</span> <span class="s1">'table'</span><span class="o">=</span><span class="s1">'blog'</span> - <span class="p">,</span> <span class="s1">'columns'</span><span class="o">=</span><span class="s1">':key,info:author,info:date,content:title'</span><span class="p">);</span> + <span class="p">,</span> <span class="s1">'columns'</span><span class="o">=</span><span class="s1">':key,info:author,info:date,content:title'</span><span class="p">)</span> +<span class="k">LOCATION</span> <span class="s1">'hbase:zk://<hostname>:<port>/'</span><span class="p">;</span> </pre></div> </div> <p>After executing the command above, you should be able to see the new table in the HBase shell:</p> @@ -365,7 +370,7 @@ blrunner-01, Jaehwa Jung, 2014-10-31, <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/hive_integration.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/hive_integration.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/hive_integration.html (original) +++ tajo/site/docs/current/hive_integration.html Thu Feb 4 00:29:05 2016 @@ -77,6 +77,7 @@ <li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -99,6 +100,7 @@ <li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -119,7 +121,7 @@ <li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> </ul> </li> @@ -214,32 +216,34 @@ same tables managed in Apache Hive.</p> and then add some configs into <code class="docutils literal"><span class="pre">conf/tajo-env.sh</span></code> and <code class="docutils literal"><span class="pre">conf/catalog-site.xml</span></code>. This section describes how to setup HiveMetaStore integration. This instruction would take no more than five minutes.</p> -<p>You need to set your Hive home directory to the environment variable <code class="docutils literal"><span class="pre">HIVE_HOME</span></code> in conf/tajo-env.sh as follows:</p> -<div class="highlight-python"><div class="highlight"><pre>export HIVE_HOME=/path/to/your/hive/directory +<p>You need to set your Hive home directory to the environment variable <strong>HIVE_HOME</strong> in <code class="docutils literal"><span class="pre">conf/tajo-env.sh</span></code> as follows:</p> +<div class="highlight-sh"><div class="highlight"><pre><span class="nb">export</span> <span class="nv">HIVE_HOME</span><span class="o">=</span>/path/to/your/hive/directory </pre></div> </div> <p>If you need to use jdbc to connect HiveMetaStore, you have to prepare MySQL jdbc driver. -Next, you should set the path of MySQL JDBC driver jar file to the environment variable HIVE_JDBC_DRIVER_DIR in conf/tajo-env.sh as follows:</p> -<div class="highlight-python"><div class="highlight"><pre>export HIVE_JDBC_DRIVER_DIR==/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar +Next, you should set the path of MySQL JDBC driver jar file to the environment variable <strong>HIVE_JDBC_DRIVER_DIR</strong> in <code class="docutils literal"><span class="pre">conf/tajo-env.sh</span></code> as follows:</p> +<div class="highlight-sh"><div class="highlight"><pre><span class="nb">export</span> <span class="nv">HIVE_JDBC_DRIVER_DIR</span><span class="o">=</span>/path/to/your/mysql_jdbc_driver/mysql-connector-java-x.x.x-bin.jar </pre></div> </div> <p>Finally, you should specify HiveCatalogStore as Tajo catalog driver class in <code class="docutils literal"><span class="pre">conf/catalog-site.xml</span></code> as follows:</p> -<div class="highlight-python"><div class="highlight"><pre><property> - <name>tajo.catalog.store.class</name> - <value>org.apache.tajo.catalog.store.HiveCatalogStore</value> -</property> +<div class="highlight-xml"><div class="highlight"><pre><span class="nt"><property></span> + <span class="nt"><name></span>tajo.catalog.store.class<span class="nt"></name></span> + <span class="nt"><value></span>org.apache.tajo.catalog.store.HiveCatalogStore<span class="nt"></value></span> +<span class="nt"></property></span> </pre></div> </div> <div class="admonition note"> <p class="first admonition-title">Note</p> -<p>Hive stores a list of partitions for each table in its metastore. If new partitions are -directly added to HDFS, HiveMetastore will not able aware of these partitions unless the user +<p>Hive stores a list of partitions for each table in its metastore. When new partitions are +added directly to HDFS, HiveMetastore can’t recognize these partitions until the user executes <code class="docutils literal"><span class="pre">ALTER</span> <span class="pre">TABLE</span> <span class="pre">table_name</span> <span class="pre">ADD</span> <span class="pre">PARTITION</span></code> commands on each of the newly added partitions or -<code class="docutils literal"><span class="pre">MSCK</span> <span class="pre">REPAIR</span> <span class="pre">TABLE</span> <span class="pre">table_name</span></code> command.</p> -<p>But current tajo doesn’t provide <code class="docutils literal"><span class="pre">ADD</span> <span class="pre">PARTITION</span></code> command and hive doesn’t provide an api for -responding to <code class="docutils literal"><span class="pre">MSK</span> <span class="pre">REPAIR</span> <span class="pre">TABLE</span></code> command. Thus, if you insert data to hive partitioned -table and you want to scan the updated partitions through Tajo, you must run following command on hive</p> -<div class="last highlight-python"><div class="highlight"><pre>$ MSCK REPAIR TABLE [table_name]; +<code class="docutils literal"><span class="pre">MSCK</span> <span class="pre">REPAIR</span> <span class="pre">TABLE</span> <span class="pre">table_name</span></code> command.</p> +<p>But current Tajo doesn’t provide <code class="docutils literal"><span class="pre">ADD</span> <span class="pre">PARTITION</span></code> command and Hive doesn’t provide an api for +responding to <code class="docutils literal"><span class="pre">MSK</span> <span class="pre">REPAIR</span> <span class="pre">TABLE</span></code> command. Thus, if you insert data to Hive partitioned +table and you want to scan the updated partitions through Tajo, you must run following command on Hive +(see <a class="reference external" href="https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-RecoverPartitions(MSCKREPAIRTABLE)">Hive doc</a> +for more details of the command):</p> +<div class="last highlight-sql"><div class="highlight"><pre><span class="n">MSCK</span> <span class="n">REPAIR</span> <span class="k">TABLE</span> <span class="p">[</span><span class="k">table_name</span><span class="p">];</span> </pre></div> </div> </div> @@ -263,7 +267,7 @@ table and you want to scan the updated p <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/index.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/index.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/index.html (original) +++ tajo/site/docs/current/index.html Thu Feb 4 00:29:05 2016 @@ -7,7 +7,7 @@ <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> - <title>Apache Tajo⢠(0.11.0 Release) - User documentation — Apache Tajo 0.11.0 documentation</title> + <title>Apache Tajo⢠(0.11.1 Release) - User documentation — Apache Tajo 0.11.0 documentation</title> @@ -76,6 +76,7 @@ <li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -98,6 +99,7 @@ <li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -118,7 +120,7 @@ <li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> </ul> </li> @@ -192,7 +194,7 @@ <ul class="wy-breadcrumbs"> <li><a href="#">Docs</a> »</li> - <li>Apache Tajo⢠(0.11.0 Release) - User documentation</li> + <li>Apache Tajo⢠(0.11.1 Release) - User documentation</li> <li class="wy-breadcrumbs-aside"> <a href="_sources/index.txt" rel="nofollow"> View page source</a> @@ -203,8 +205,8 @@ </div> <div role="main"> - <div class="section" id="apache-tajo-0-11-0-release-user-documentation"> -<h1>Apache Tajo⢠(0.11.0 Release) - User documentation<a class="headerlink" href="#apache-tajo-0-11-0-release-user-documentation" title="Permalink to this headline">¶</a></h1> + <div class="section" id="apache-tajo-0-11-1-release-user-documentation"> +<h1>Apache Tajo⢠(0.11.1 Release) - User documentation<a class="headerlink" href="#apache-tajo-0-11-1-release-user-documentation" title="Permalink to this headline">¶</a></h1> <p>Table of Contents:</p> <div class="toctree-wrapper compound"> <ul> @@ -263,11 +265,20 @@ </ul> </li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a><ul> -<li class="toctree-l3"><a class="reference internal" href="configuration/tajo-site-xml.html#system-config">System Config</a></li> +<li class="toctree-l3"><a class="reference internal" href="configuration/tajo-site-xml.html#join-query-settings">Join Query Settings</a></li> +<li class="toctree-l3"><a class="reference internal" href="configuration/tajo-site-xml.html#sort-query-settings">Sort Query Settings</a></li> +<li class="toctree-l3"><a class="reference internal" href="configuration/tajo-site-xml.html#group-by-query-settings">Group by Query Settings</a></li> <li class="toctree-l3"><a class="reference internal" href="configuration/tajo-site-xml.html#date-time-settings">Date/Time Settings</a></li> +<li class="toctree-l3"><a class="reference internal" href="configuration/tajo-site-xml.html#table-partitions">Table partitions</a></li> +<li class="toctree-l3"><a class="reference internal" href="configuration/tajo-site-xml.html#arithmetic-settings">Arithmetic Settings</a></li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a><ul> +<li class="toctree-l3"><a class="reference internal" href="configuration/storage-site-json.html#spaces">Spaces</a></li> +<li class="toctree-l3"><a class="reference internal" href="configuration/storage-site-json.html#storages">Storages</a></li> +</ul> +</li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -347,6 +358,7 @@ <li class="toctree-l3"><a class="reference internal" href="sql_language/predicates.html#string-pattern-matching-predicates">String Pattern Matching Predicates</a></li> </ul> </li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -387,15 +399,14 @@ <li class="toctree-l3"><a class="reference internal" href="table_management/table_overview.html#tablespaces">Tablespaces</a></li> <li class="toctree-l3"><a class="reference internal" href="table_management/table_overview.html#managed-table">Managed Table</a></li> <li class="toctree-l3"><a class="reference internal" href="table_management/table_overview.html#external-table">External Table</a></li> -<li class="toctree-l3"><a class="reference internal" href="table_management/table_overview.html#table-properties">Table Properties</a></li> -<li class="toctree-l3"><a class="reference internal" href="table_management/table_overview.html#common-table-properties">Common Table Properties</a></li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a><ul> +<li class="toctree-l3"><a class="reference internal" href="table_management/tablespaces.html#external-table-managed-table-and-tablespace">External Table, Managed Table and Tablespace</a></li> <li class="toctree-l3"><a class="reference internal" href="table_management/tablespaces.html#configuration">Configuration</a></li> </ul> </li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a><ul> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a><ul> <li class="toctree-l3"><a class="reference internal" href="table_management/text.html">TEXT</a></li> <li class="toctree-l3"><a class="reference internal" href="table_management/json.html">JSON</a></li> <li class="toctree-l3"><a class="reference internal" href="table_management/rcfile.html">RCFile</a></li> @@ -404,7 +415,10 @@ <li class="toctree-l3"><a class="reference internal" href="table_management/sequencefile.html">SequenceFile</a></li> </ul> </li> -<li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a><ul> +<li class="toctree-l3"><a class="reference internal" href="table_management/compression.html#compression-properties-for-each-data-format">Compression Properties for each Data Format</a></li> +</ul> +</li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="table_partitioning.html">Table Partitioning</a><ul> @@ -414,7 +428,8 @@ </li> <li class="toctree-l2"><a class="reference internal" href="partitioning/column_partitioning.html">Column Partitioning</a><ul> <li class="toctree-l3"><a class="reference internal" href="partitioning/column_partitioning.html#how-to-create-a-column-partitioned-table">How to Create a Column Partitioned Table</a></li> -<li class="toctree-l3"><a class="reference internal" href="partitioning/column_partitioning.html#partition-pruning-on-column-partitioned-tables">Partition Pruning on Column Partitioned Tables</a></li> +<li class="toctree-l3"><a class="reference internal" href="partitioning/column_partitioning.html#querying-partitioned-tables">Querying Partitioned Tables</a></li> +<li class="toctree-l3"><a class="reference internal" href="partitioning/column_partitioning.html#add-data-to-partition-table">Add data to Partition Table</a></li> <li class="toctree-l3"><a class="reference internal" href="partitioning/column_partitioning.html#compatibility-issues-with-apache-hive">Compatibility Issues with Apache Hiveâ¢</a></li> </ul> </li> @@ -441,6 +456,7 @@ <li class="toctree-l2"><a class="reference internal" href="index/how_to_use.html">How to use index?</a><ul> <li class="toctree-l3"><a class="reference internal" href="index/how_to_use.html#create-index">1. Create index</a></li> <li class="toctree-l3"><a class="reference internal" href="index/how_to_use.html#enable-disable-index-scans">2. Enable/disable index scans</a></li> +<li class="toctree-l3"><a class="reference internal" href="index/how_to_use.html#index-backup-and-restore">3. Index backup and restore</a></li> </ul> </li> <li class="toctree-l2"><a class="reference internal" href="index/future_work.html">Future Works</a></li> @@ -516,7 +532,7 @@ <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/index/future_work.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/index/future_work.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/index/future_work.html (original) +++ tajo/site/docs/current/index/future_work.html Thu Feb 4 00:29:05 2016 @@ -78,6 +78,7 @@ <li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul> @@ -100,6 +101,7 @@ <li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> @@ -120,7 +122,7 @@ <li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> </ul> </li> @@ -210,9 +212,8 @@ <div class="section" id="future-works"> <h1>Future Works<a class="headerlink" href="#future-works" title="Permalink to this headline">¶</a></h1> <ul class="simple"> -<li>Providing more index types, such as bitmap and HBase index</li> +<li>Supporting more index types, such as bitmap and HBase index</li> <li>Supporting index on partitioned tables</li> -<li>Supporting the backup and restore feature</li> <li>Cost-based query optimization by estimating the query selectivity</li> </ul> </div> @@ -235,7 +236,7 @@ <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/index/how_to_use.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/index/how_to_use.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/index/how_to_use.html (original) +++ tajo/site/docs/current/index/how_to_use.html Thu Feb 4 00:29:05 2016 @@ -78,6 +78,7 @@ <li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul> @@ -100,6 +101,7 @@ <li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> @@ -120,7 +122,7 @@ <li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> </ul> </li> @@ -211,11 +213,12 @@ <h1>How to use index?<a class="headerlink" href="#how-to-use-index" title="Permalink to this headline">¶</a></h1> <div class="section" id="create-index"> <h2>1. Create index<a class="headerlink" href="#create-index" title="Permalink to this headline">¶</a></h2> -<p>The first step for utilizing index is index creation. You can create index using SQL (<a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>) or Tajo API (<a class="reference internal" href="../tajo_client_api.html"><em>Tajo Client API</em></a>). For example, you can create a BST index on the lineitem table by submitting the following SQL to Tajo.</p> +<p>The first step for utilizing index is to create an index. You can create an index using SQL (<a class="reference internal" href="../sql_language/ddl.html"><em>Data Definition Language</em></a>) or Tajo API (<a class="reference internal" href="../tajo_client_api.html"><em>Tajo Client API</em></a>). +For example, the following SQL statement will create a BST index on the lineitem table.</p> <div class="highlight-sql"><div class="highlight"><pre><span class="k">default</span><span class="o">></span> <span class="k">create</span> <span class="k">index</span> <span class="n">l_orderkey_idx</span> <span class="k">on</span> <span class="n">lineitem</span> <span class="p">(</span><span class="n">l_orderkey</span><span class="p">);</span> </pre></div> </div> -<p>If the index is created successfully, you can see the information about that index as follows:</p> +<p>If the index is created successfully, you can see the index information as follows:</p> <div class="highlight-python"><div class="highlight"><pre>default> \d lineitem table name: default.lineitem @@ -253,19 +256,26 @@ Indexes: </div> <div class="section" id="enable-disable-index-scans"> <h2>2. Enable/disable index scans<a class="headerlink" href="#enable-disable-index-scans" title="Permalink to this headline">¶</a></h2> -<p>When an index is successfully created, you must enable the index scan feature as follows:</p> +<p>Reading data using index is disabled by default. +So, exploiting the created index, you need a further step, enabling ‘index scan’ as following:</p> <div class="highlight-sql"><div class="highlight"><pre>default> \set INDEX_ENABLED true </pre></div> </div> -<p>If you don’t want to use the index scan feature anymore, you can simply disable it as follows:</p> +<p>If you don’t want to use index scan anymore, you can simply disable it as follows:</p> <div class="highlight-sql"><div class="highlight"><pre>default> \set INDEX_ENABLED false </pre></div> </div> <div class="admonition note"> <p class="first admonition-title">Note</p> -<p class="last">Once the index scan feature is enabled, Tajo currently always performs the index scan regardless of its efficiency. You should set this option when the expected number of retrieved tuples is sufficiently small.</p> +<p class="last">Once index scan is enabled, Tajo will perform ‘index scan’ if possible. In some cases, it may cause performance +degradation. If you always want to get better performance, you should either enable or disable ‘index scan’ +according to selectivity. Usually, the performance gain of index will increase when the selectivity is low.</p> </div> </div> +<div class="section" id="index-backup-and-restore"> +<h2>3. Index backup and restore<a class="headerlink" href="#index-backup-and-restore" title="Permalink to this headline">¶</a></h2> +<p>Tajo currently provides only the catalog backup and restore for index. Please refer to <a class="reference internal" href="../backup_and_restore/catalog.html"><em>Backup and Restore Catalog</em></a> for more information about catalog backup and restore.</p> +</div> </div> @@ -286,7 +296,7 @@ Indexes: <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/index/types.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/index/types.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/index/types.html (original) +++ tajo/site/docs/current/index/types.html Thu Feb 4 00:29:05 2016 @@ -78,6 +78,7 @@ <li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul> @@ -100,6 +101,7 @@ <li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> @@ -120,7 +122,7 @@ <li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> </ul> </li> @@ -209,8 +211,8 @@ <div class="section" id="index-types"> <h1>Index Types<a class="headerlink" href="#index-types" title="Permalink to this headline">¶</a></h1> -<p>Currently, Tajo supports only one type of index, <code class="docutils literal"><span class="pre">TWO_LEVEL_BIN_TREE</span></code>, shortly <code class="docutils literal"><span class="pre">BST</span></code>. The BST index is a kind of binary search tree which is extended to be permanently stored on disk. It consists of two levels of nodes; a leaf node indexes the keys with the positions of data in an HDFS block and a root node indexes the keys with the leaf node indices.</p> -<p>When an index scan is started, the query engine first reads the root node and finds the search key. If it finds a leaf node corresponding to the search key, it subsequently finds the search key in that leaf node. Finally, it directly reads a tuple corresponding to the search key from HDFS.</p> +<p>Currently, Tajo supports only one type of index, <code class="docutils literal"><span class="pre">TWO_LEVEL_BIN_TREE</span></code>, shortly <code class="docutils literal"><span class="pre">BST</span></code>. The BST index is a kind of binary search tree which is extended to be permanently stored on disk. It consists of two levels of nodes; a leaf node indexes the keys with the offsets to data stored on HDFS, and a root node indexes the keys with the offsets to the leaf nodes.</p> +<p>When an index scan is started, the query engine first reads the root node and finds the search key. If it successfully finds a leaf node corresponding to the search key, it subsequently finds the search key in that leaf node. Finally, it directly reads a tuple corresponding to the search key from HDFS.</p> </div> @@ -231,7 +233,7 @@ <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/index_overview.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/index_overview.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/index_overview.html (original) +++ tajo/site/docs/current/index_overview.html Thu Feb 4 00:29:05 2016 @@ -77,6 +77,7 @@ <li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -99,6 +100,7 @@ <li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -119,7 +121,7 @@ <li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> </ul> </li> @@ -207,11 +209,6 @@ <div class="section" id="index-experimental-feature"> <h1>Index (Experimental Feature)<a class="headerlink" href="#index-experimental-feature" title="Permalink to this headline">¶</a></h1> <p>An index is a data structure that is used for efficient query processing. Using an index, the Tajo query engine can directly retrieve search values.</p> -<p>This is still an experimental feature. In order to use indexes, you must check out the source code of the <code class="docutils literal"><span class="pre">index_support</span></code> branch:</p> -<div class="highlight-python"><div class="highlight"><pre>git clone -b index_support https://git-wip-us.apache.org/repos/asf/tajo.git tajo-index -</pre></div> -</div> -<p>For the source code build, please refer to <a class="reference internal" href="getting_started.html"><em>Getting Started</em></a>.</p> <p>The following sections describe the supported index types, the query execution with an index, and the future works.</p> <div class="toctree-wrapper compound"> <ul> @@ -240,7 +237,7 @@ <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/introduction.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/introduction.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/introduction.html (original) +++ tajo/site/docs/current/introduction.html Thu Feb 4 00:29:05 2016 @@ -30,7 +30,7 @@ <link rel="top" title="Apache Tajo 0.11.0 documentation" href="index.html"/> <link rel="next" title="Getting Started" href="getting_started.html"/> - <link rel="prev" title="Apache Tajo⢠(0.11.0 Release) - User documentation" href="index.html"/> + <link rel="prev" title="Apache Tajo⢠(0.11.1 Release) - User documentation" href="index.html"/> <script src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script> @@ -77,6 +77,7 @@ <li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -99,6 +100,7 @@ <li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -119,7 +121,7 @@ <li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> </ul> </li> @@ -226,7 +228,7 @@ queries in order to avoid the worst quer <a href="getting_started.html" class="btn btn-neutral float-right" title="Getting Started"/>Next <span class="fa fa-arrow-circle-right"></span></a> - <a href="index.html" class="btn btn-neutral" title="Apache Tajo⢠(0.11.0 Release) - User documentation"><span class="fa fa-arrow-circle-left"></span> Previous</a> + <a href="index.html" class="btn btn-neutral" title="Apache Tajo⢠(0.11.1 Release) - User documentation"><span class="fa fa-arrow-circle-left"></span> Previous</a> </div> @@ -235,7 +237,7 @@ queries in order to avoid the worst quer <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/jdbc_driver.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/jdbc_driver.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/jdbc_driver.html (original) +++ tajo/site/docs/current/jdbc_driver.html Thu Feb 4 00:29:05 2016 @@ -77,6 +77,7 @@ <li class="toctree-l2"><a class="reference internal" href="configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo Shell (TSQL)</a><ul> @@ -99,6 +100,7 @@ <li class="toctree-l2"><a class="reference internal" href="sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="time_zone.html">Time Zone</a><ul> @@ -119,7 +121,7 @@ <li class="toctree-l1"><a class="reference internal" href="table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="table_management/compression.html">Compression</a></li> </ul> </li> @@ -222,9 +224,9 @@ In this section, we explain how to get J <div class="section" id="from-building-source-code"> <h3>From Building Source Code<a class="headerlink" href="#from-building-source-code" title="Permalink to this headline">¶</a></h3> <p>You can build Tajo from the source code and then get JAR files as follows:</p> -<div class="highlight-bash"><div class="highlight"><pre><span class="nv">$ </span>tar xzvf tajo-x.y.z-src.tar.gz -<span class="nv">$ </span>mvn clean package -DskipTests -Pdist -Dtar -<span class="nv">$ </span>ls -l tajo-dist/target/tajo-x.y.z/share/jdbc-dist/tajo-jdbc-x.y.z.jar +<div class="highlight-bash"><div class="highlight"><pre>$ tar xzvf tajo-x.y.z-src.tar.gz +$ mvn clean package -DskipTests -Pdist -Dtar +$ ls -l tajo-dist/target/tajo-x.y.z/share/jdbc-dist/tajo-jdbc-x.y.z.jar </pre></div> </div> </div> @@ -232,7 +234,7 @@ In this section, we explain how to get J <div class="section" id="setting-the-classpath"> <h2>Setting the CLASSPATH<a class="headerlink" href="#setting-the-classpath" title="Permalink to this headline">¶</a></h2> <p>In order to use the JDBC driver, you should add <code class="docutils literal"><span class="pre">tajo-jdbc-x.y.z.jar</span></code> in your <code class="docutils literal"><span class="pre">CLASSPATH</span></code>.</p> -<div class="highlight-bash"><div class="highlight"><pre><span class="nv">CLASSPATH</span><span class="o">=</span>path/to/tajo-jdbc-x.y.z.jar:<span class="nv">$CLASSPATH</span> +<div class="highlight-bash"><div class="highlight"><pre><span class="nv">CLASSPATH</span><span class="o">=</span>path/to/tajo-jdbc-x.y.z.jar:<span class="nv">$C</span>LASSPATH </pre></div> </div> </div> @@ -351,7 +353,7 @@ All tables are contained in <code class= <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/objects.inv URL: http://svn.apache.org/viewvc/tajo/site/docs/current/objects.inv?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== Binary files - no diff available. Modified: tajo/site/docs/current/partitioning/column_partitioning.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/partitioning/column_partitioning.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/partitioning/column_partitioning.html (original) +++ tajo/site/docs/current/partitioning/column_partitioning.html Thu Feb 4 00:29:05 2016 @@ -78,6 +78,7 @@ <li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul> @@ -100,6 +101,7 @@ <li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> @@ -120,7 +122,7 @@ <li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> </ul> </li> @@ -214,26 +216,49 @@ <h2>How to Create a Column Partitioned Table<a class="headerlink" href="#how-to-create-a-column-partitioned-table" title="Permalink to this headline">¶</a></h2> <p>You can create a partitioned table by using the <code class="docutils literal"><span class="pre">PARTITION</span> <span class="pre">BY</span></code> clause. For a column partitioned table, you should use the <code class="docutils literal"><span class="pre">PARTITION</span> <span class="pre">BY</span> <span class="pre">COLUMN</span></code> clause with partition keys.</p> -<p>For example, assume there is a table <code class="docutils literal"><span class="pre">orders</span></code> composed of the following schema.</p> -<div class="highlight-python"><div class="highlight"><pre>id INT, -item_name TEXT, -price FLOAT -</pre></div> -</div> -<p>Also, assume that you want to use <code class="docutils literal"><span class="pre">order_date</span> <span class="pre">TEXT</span></code> and <code class="docutils literal"><span class="pre">ship_date</span> <span class="pre">TEXT</span></code> as the partition keys. -Then, you should create a table as follows:</p> -<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">orders</span> <span class="p">(</span> - <span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> - <span class="n">item_name</span> <span class="nb">TEXT</span><span class="p">,</span> - <span class="n">price</span> -<span class="p">)</span> <span class="n">PARTITION</span> <span class="k">BY</span> <span class="k">COLUMN</span> <span class="p">(</span><span class="n">order_date</span> <span class="nb">TEXT</span><span class="p">,</span> <span class="n">ship_date</span> <span class="nb">TEXT</span><span class="p">);</span> +<p>For example, assume a table with the following schema.</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> +<span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> +<span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> +<span class="n">grade</span> <span class="nb">TEXT</span><span class="p">,</span> +<span class="n">country</span> <span class="nb">TEXT</span><span class="p">,</span> +<span class="n">city</span> <span class="nb">TEXT</span><span class="p">,</span> +<span class="n">phone</span> <span class="nb">TEXT</span> +<span class="p">);</span> </pre></div> </div> +<p>If you want to make country as partitioned column, your Tajo definition would be this:</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">student</span> <span class="p">(</span> + <span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> + <span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> + <span class="n">grade</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">city</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">phone</span> <span class="nb">TEXT</span> +<span class="p">)</span> <span class="n">PARTITION</span> <span class="k">BY</span> <span class="k">COLUMN</span> <span class="p">(</span><span class="n">country</span> <span class="nb">TEXT</span><span class="p">);</span> +</pre></div> +</div> +<p>Let us assume you want to use more partition columns and parquet file format. Here’s an example statement to create a table:</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">student</span> <span class="p">(</span> + <span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> + <span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> + <span class="n">grade</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">phone</span> <span class="nb">TEXT</span> +<span class="p">)</span> <span class="k">USING</span> <span class="n">PARQUET</span> +<span class="n">PARTITION</span> <span class="k">BY</span> <span class="k">COLUMN</span> <span class="p">(</span><span class="n">country</span> <span class="nb">TEXT</span><span class="p">,</span> <span class="n">city</span> <span class="nb">TEXT</span><span class="p">);</span> +</pre></div> +</div> +<p>The statement above creates the student table with id, name, grade, etc. The table is also partitioned and data is stored in parquet files.</p> +<p>You might have noticed that while the partitioning key columns are a part of the table DDL, theyâre only listed in the <code class="docutils literal"><span class="pre">PARTITION</span> <span class="pre">BY</span></code> clause. In Tajo, as data is written to disk, each partition of data will be automatically split out into different folders, e.g. country=USA/city=NEWYORK. During a read operation, Tajo will use the folder structure to quickly locate the right partitions and also return the partitioning columns as columns in the result set.</p> </div> -<div class="section" id="partition-pruning-on-column-partitioned-tables"> -<h2>Partition Pruning on Column Partitioned Tables<a class="headerlink" href="#partition-pruning-on-column-partitioned-tables" title="Permalink to this headline">¶</a></h2> -<p>The following predicates in the <code class="docutils literal"><span class="pre">WHERE</span></code> clause can be used to prune unqualified column partitions without processing -during query planning phase.</p> +<div class="section" id="querying-partitioned-tables"> +<h2>Querying Partitioned Tables<a class="headerlink" href="#querying-partitioned-tables" title="Permalink to this headline">¶</a></h2> +<p>If a table created using the <code class="docutils literal"><span class="pre">PARTITION</span> <span class="pre">BY</span></code> clause, a query can do partition pruning and scan only a fraction of the table relevant to the partitions specified by the query. Tajo currently does partition pruning if the partition predicates are specified in the WHERE clause. For example, if table student is partitioned on column country and column city, the following query retrieves rows in <code class="docutils literal"><span class="pre">country=KOREA\city=SEOUL</span></code> directory.</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="k">SELECT</span> <span class="o">*</span> <span class="k">FROM</span> <span class="n">student</span> <span class="k">WHERE</span> <span class="n">country</span> <span class="o">=</span> <span class="s1">'KOREA'</span> <span class="k">AND</span> <span class="n">city</span> <span class="o">=</span> <span class="s1">'SEOUL'</span><span class="p">;</span> +</pre></div> +</div> +<p>The following predicates in the <code class="docutils literal"><span class="pre">WHERE</span></code> clause can be used to prune column partitions during query planning phase.</p> <ul class="simple"> <li><code class="docutils literal"><span class="pre">=</span></code></li> <li><code class="docutils literal"><span class="pre"><></span></code></li> @@ -245,10 +270,134 @@ during query planning phase.</p> <li>IN list predicates</li> </ul> </div> +<div class="section" id="add-data-to-partition-table"> +<h2>Add data to Partition Table<a class="headerlink" href="#add-data-to-partition-table" title="Permalink to this headline">¶</a></h2> +<p>Tajo provides a very useful feature of dynamic partitioning. You don’t need to use any syntax with both <code class="docutils literal"><span class="pre">INSERT</span> <span class="pre">INTO</span> <span class="pre">...</span> <span class="pre">SELECT</span></code> and <code class="docutils literal"><span class="pre">Create</span> <span class="pre">Table</span> <span class="pre">As</span> <span class="pre">Select(CTAS)</span></code> statments for dynamic partitioning. Tajo will automatically filter the data, create directories, move filtered data to appropriate directory and create partition over it.</p> +<p>For example, assume there are both <code class="docutils literal"><span class="pre">student_source</span></code> and <code class="docutils literal"><span class="pre">student</span></code> tables composed of the following schema.</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">student_source</span> <span class="p">(</span> + <span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> + <span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> + <span class="n">grade</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">country</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">city</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">phone</span> <span class="nb">TEXT</span> +<span class="p">);</span> + +<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">student</span> <span class="p">(</span> + <span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> + <span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> + <span class="n">grade</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">phone</span> <span class="nb">TEXT</span> +<span class="p">)</span> <span class="n">PARTITION</span> <span class="k">BY</span> <span class="k">COLUMN</span> <span class="p">(</span><span class="n">country</span> <span class="nb">TEXT</span><span class="p">,</span> <span class="n">city</span> <span class="nb">TEXT</span><span class="p">);</span> +</pre></div> +</div> +<div class="section" id="how-to-insert-dynamically-to-partition-table"> +<h3>How to INSERT dynamically to partition table<a class="headerlink" href="#how-to-insert-dynamically-to-partition-table" title="Permalink to this headline">¶</a></h3> +<p>If you want to load an entire country or an entire city in one fell swoop:</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="k">INSERT</span> <span class="n">OVERWRITE</span> <span class="k">INTO</span> <span class="n">student</span> +<span class="k">SELECT</span> <span class="n">id</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">gender</span><span class="p">,</span> <span class="n">grade</span><span class="p">,</span> <span class="n">phone</span><span class="p">,</span> <span class="n">country</span><span class="p">,</span> <span class="n">city</span> +<span class="k">FROM</span> <span class="n">student_source</span><span class="p">;</span> +</pre></div> +</div> +</div> +<div class="section" id="how-to-ctas-dynamically-to-partition-table"> +<h3>How to CTAS dynamically to partition table<a class="headerlink" href="#how-to-ctas-dynamically-to-partition-table" title="Permalink to this headline">¶</a></h3> +<p>when a partition table is created:</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="k">DROP</span> <span class="k">TABLE</span> <span class="n">if</span> <span class="k">exists</span> <span class="n">student</span><span class="p">;</span> + +<span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">student</span> <span class="p">(</span> + <span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> + <span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> + <span class="n">grade</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">phone</span> <span class="nb">TEXT</span> +<span class="p">)</span> <span class="n">PARTITION</span> <span class="k">BY</span> <span class="k">COLUMN</span> <span class="p">(</span><span class="n">country</span> <span class="nb">TEXT</span><span class="p">,</span> <span class="n">city</span> <span class="nb">TEXT</span><span class="p">)</span> +<span class="k">AS</span> <span class="k">SELECT</span> <span class="n">id</span><span class="p">,</span> <span class="n">name</span><span class="p">,</span> <span class="n">gender</span><span class="p">,</span> <span class="n">grade</span><span class="p">,</span> <span class="n">phone</span><span class="p">,</span> <span class="n">country</span><span class="p">,</span> <span class="n">city</span> +<span class="k">FROM</span> <span class="n">student_source</span><span class="p">;</span> +</pre></div> +</div> +<div class="admonition note"> +<p class="first admonition-title">Note</p> +<p class="last">When loading data into a partition, itâs necessary to include the partition columns as the last columns in the query. The column names in the source query donât need to match the partition column names.</p> +</div> +</div> +</div> <div class="section" id="compatibility-issues-with-apache-hive"> <h2>Compatibility Issues with Apache Hiveâ¢<a class="headerlink" href="#compatibility-issues-with-apache-hive" title="Permalink to this headline">¶</a></h2> -<p>If partitioned tables of Hive are created as external tables in Tajo, Tajo can process the Hive partitioned tables directly. -There haven’t known compatibility issues yet.</p> +<p>If partitioned tables of Hive are created as external tables in Tajo, Tajo can process the Hive partitioned tables directly.</p> +<div class="section" id="how-to-create-partition-table"> +<h3>How to create partition table<a class="headerlink" href="#how-to-create-partition-table" title="Permalink to this headline">¶</a></h3> +<p>If you create a partition table as follows in Tajo:</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="k">default</span><span class="o">></span> <span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">student</span> <span class="p">(</span> + <span class="n">id</span> <span class="nb">INT</span><span class="p">,</span> + <span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> + <span class="n">grade</span> <span class="nb">TEXT</span><span class="p">,</span> + <span class="n">phone</span> <span class="nb">TEXT</span> +<span class="p">)</span> <span class="n">PARTITION</span> <span class="k">BY</span> <span class="k">COLUMN</span> <span class="p">(</span><span class="n">country</span> <span class="nb">TEXT</span><span class="p">,</span> <span class="n">city</span> <span class="nb">TEXT</span><span class="p">);</span> +</pre></div> +</div> +<p>And then you can get table information in Hive:</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="n">hive</span><span class="o">></span> <span class="k">desc</span> <span class="n">student</span><span class="p">;</span> +<span class="n">OK</span> +<span class="n">id</span> <span class="nb">int</span> +<span class="n">name</span> <span class="n">string</span> +<span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span> +<span class="n">grade</span> <span class="n">string</span> +<span class="n">phone</span> <span class="n">string</span> +<span class="n">country</span> <span class="n">string</span> +<span class="n">city</span> <span class="n">string</span> + +<span class="o">#</span> <span class="n">Partition</span> <span class="n">Information</span> +<span class="o">#</span> <span class="n">col_name</span> <span class="n">data_type</span> <span class="k">comment</span> + +<span class="n">country</span> <span class="n">string</span> +<span class="n">city</span> <span class="n">string</span> +</pre></div> +</div> +<p>Or as you create the table in Hive:</p> +<div class="highlight-sql"><div class="highlight"><pre><span class="n">hive</span> <span class="o">></span> <span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">student</span> <span class="p">(</span> + <span class="n">id</span> <span class="nb">int</span><span class="p">,</span> + <span class="n">name</span> <span class="n">string</span><span class="p">,</span> + <span class="n">gender</span> <span class="nb">char</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> + <span class="n">grade</span> <span class="n">string</span><span class="p">,</span> + <span class="n">phone</span> <span class="n">string</span> +<span class="p">)</span> <span class="n">PARTITIONED</span> <span class="k">BY</span> <span class="p">(</span><span class="n">country</span> <span class="n">string</span><span class="p">,</span> <span class="n">city</span> <span class="n">string</span><span class="p">)</span> +<span class="k">ROW</span> <span class="n">FORMAT</span> <span class="n">DELIMITED</span> + <span class="n">FIELDS</span> <span class="n">TERMINATED</span> <span class="k">BY</span> <span class="s1">'|'</span> <span class="p">;</span> +</pre></div> +</div> +<p>You will see table information in Tajo:</p> +<div class="highlight-sql"><div class="highlight"><pre>default> \d student; +table name: default.student +table uri: hdfs://your_hdfs_namespace/user/hive/warehouse/student +store type: TEXT +number of rows: 0 +volume: 0 B +Options: + 'text.null'='\\N' + 'transient_lastDdlTime'='1438756422' + 'text.delimiter'='|' + +schema: +id INT4 +name TEXT +gender CHAR(1) +grade TEXT +phone TEXT + +Partitions: +type:COLUMN +columns::default.student.country (TEXT), default.student.city (TEXT) +</pre></div> +</div> +</div> +<div class="section" id="how-to-add-data-to-partition-table"> +<h3>How to add data to partition table<a class="headerlink" href="#how-to-add-data-to-partition-table" title="Permalink to this headline">¶</a></h3> +<p>In Tajo, you can add data dynamically to partition table of Hive with both <code class="docutils literal"><span class="pre">INSERT</span> <span class="pre">INTO</span> <span class="pre">...</span> <span class="pre">SELECT</span></code> and <code class="docutils literal"><span class="pre">Create</span> <span class="pre">Table</span> <span class="pre">As</span> <span class="pre">Select</span> <span class="pre">(CTAS)</span></code> statments. Tajo will automatically filter the data to HiveMetastore, create directories and move filtered data to appropriate directory on the distributed file system.</p> +</div> </div> </div> @@ -270,7 +419,7 @@ There haven’t known compatibility <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/partitioning/hash_partitioning.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/partitioning/hash_partitioning.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/partitioning/hash_partitioning.html (original) +++ tajo/site/docs/current/partitioning/hash_partitioning.html Thu Feb 4 00:29:05 2016 @@ -78,6 +78,7 @@ <li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul> @@ -100,6 +101,7 @@ <li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> @@ -120,7 +122,7 @@ <li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> </ul> </li> @@ -232,7 +234,7 @@ <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div> Modified: tajo/site/docs/current/partitioning/intro_to_partitioning.html URL: http://svn.apache.org/viewvc/tajo/site/docs/current/partitioning/intro_to_partitioning.html?rev=1728394&r1=1728393&r2=1728394&view=diff ============================================================================== --- tajo/site/docs/current/partitioning/intro_to_partitioning.html (original) +++ tajo/site/docs/current/partitioning/intro_to_partitioning.html Thu Feb 4 00:29:05 2016 @@ -78,6 +78,7 @@ <li class="toctree-l2"><a class="reference internal" href="../configuration/service_config_defaults.html">Cluster Service Configuration Defaults</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li> <li class="toctree-l2"><a class="reference internal" href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li> +<li class="toctree-l2"><a class="reference internal" href="../configuration/storage-site-json.html">The storage-site.json File</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo Shell (TSQL)</a><ul> @@ -100,6 +101,7 @@ <li class="toctree-l2"><a class="reference internal" href="../sql_language/joins.html">Joins</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/sql_expression.html">SQL Expressions</a></li> <li class="toctree-l2"><a class="reference internal" href="../sql_language/predicates.html">Predicates</a></li> +<li class="toctree-l2"><a class="reference internal" href="../sql_language/explain.html">EXPLAIN</a></li> </ul> </li> <li class="toctree-l1"><a class="reference internal" href="../time_zone.html">Time Zone</a><ul> @@ -120,7 +122,7 @@ <li class="toctree-l1"><a class="reference internal" href="../table_management.html">Table Management</a><ul> <li class="toctree-l2"><a class="reference internal" href="../table_management/table_overview.html">Overview of Tajo Tables</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/tablespaces.html">Tablespaces</a></li> -<li class="toctree-l2"><a class="reference internal" href="../table_management/file_formats.html">File Formats</a></li> +<li class="toctree-l2"><a class="reference internal" href="../table_management/data_formats.html">Data Formats</a></li> <li class="toctree-l2"><a class="reference internal" href="../table_management/compression.html">Compression</a></li> </ul> </li> @@ -243,7 +245,7 @@ Currently, Apache Tajo only provides Apa <div role="contentinfo"> <p> - © Copyright 2014, Apache Tajo Team. + © Copyright 2015, Apache Tajo Team. </p> </div>
