Author: jihoonson
Date: Tue Oct 6 06:45:04 2015
New Revision: 1706955
URL: http://svn.apache.org/viewvc?rev=1706955&view=rev
Log:
add missing files
Added:
tajo/site/docs/devel/_sources/storage_plugins/
tajo/site/docs/devel/_sources/storage_plugins.txt
tajo/site/docs/devel/_sources/storage_plugins/overview.txt
tajo/site/docs/devel/_sources/storage_plugins/postgresql.txt
tajo/site/docs/devel/_sources/table_management/json.txt
tajo/site/docs/devel/_sources/table_management/orc.txt
tajo/site/docs/devel/storage_plugins/
tajo/site/docs/devel/storage_plugins.html
tajo/site/docs/devel/storage_plugins/overview.html
tajo/site/docs/devel/storage_plugins/postgresql.html
tajo/site/docs/devel/table_management/json.html
tajo/site/docs/devel/table_management/orc.html
Added: tajo/site/docs/devel/_sources/storage_plugins.txt
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/_sources/storage_plugins.txt?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/_sources/storage_plugins.txt (added)
+++ tajo/site/docs/devel/_sources/storage_plugins.txt Tue Oct 6 06:45:04 2015
@@ -0,0 +1,11 @@
+*************************************
+Storage Plugin
+*************************************
+
+This section describes the storage plugins available in Tajo to access
datasets from different data sources.
+
+.. toctree::
+ :maxdepth: 1
+
+ storage_plugins/overview
+ storage_plugins/postgresql
\ No newline at end of file
Added: tajo/site/docs/devel/_sources/storage_plugins/overview.txt
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/_sources/storage_plugins/overview.txt?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/_sources/storage_plugins/overview.txt (added)
+++ tajo/site/docs/devel/_sources/storage_plugins/overview.txt Tue Oct 6
06:45:04 2015
@@ -0,0 +1,47 @@
+*************************************
+Storage Plugin Overview
+*************************************
+
+Overview
+========
+
+Tajo supports various storage systems, such as HDFS, Amazon S3, Openstack
Swift, HBase, and RDBMS. Tajo already embeds HDFS, S3, Openstack, HBase, RDBMS
storage plugins, and also Tajo allows users to register custom storages and
data formats to Tajo cluster instances. This section describes how you register
custom storages and data types.
+
+Register custom storage
+=======================
+
+First of all, your storage implementation should be packed as a jar file.
Then, please copy the jar file into ``tajo/extlib`` directory. Next, you should
copy ``conf/storage-site.json.template`` into ``conf/storage-site.json`` and
modify the file like the below.
+
+Configuration
+=============
+
+Tajo has a default configuration for builtin storages, such as HDFS, local
file system, and Amazon S3. it also allows users to add custom storage plugins
+
+``conf/storage-site.json`` file has the following struct:
+
+.. code-block:: json
+
+ {
+ "storages": {
+ "${scheme}": {
+ "handler": "${class name}"
+ }
+ }
+ }
+
+Each storage instance (i.e., :doc:`/table_management/tablespaces`) is
identified by an URI. The scheme of URI plays a role to identify storage type.
For example, ``hdfs://`` is used for Hdfs storage, ``jdbc://`` is used for
JDBC-based storage, and ``hbase://`` is used for HBase storage.
+
+You should substitute a scheme name without ``://`` for ``${scheme}``.
+
+See an example for HBase storage.
+
+.. code-block:: json
+
+ {
+ "storages": {
+ "hbase": {
+ "handler": "org.apache.tajo.storage.hbase.HBaseTablespace",
+ "default-format": "hbase"
+ }
+ }
+ }
\ No newline at end of file
Added: tajo/site/docs/devel/_sources/storage_plugins/postgresql.txt
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/_sources/storage_plugins/postgresql.txt?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/_sources/storage_plugins/postgresql.txt (added)
+++ tajo/site/docs/devel/_sources/storage_plugins/postgresql.txt Tue Oct 6
06:45:04 2015
@@ -0,0 +1,40 @@
+*************************************
+PostgreSQL Storage Handler
+*************************************
+
+Overview
+========
+
+PostgreSQL storage handler is available by default in Tajo. It enables users'
queries to access database objects in PostgreSQL. Tables in PostgreSQL will be
shown as tables in Tajo too. Most of the SQL queries used for PostgreSQL are
available in Tajo via this storage handles. Its main advantages is to allow
federated query processing among tables in stored HDFS and PostgreSQL.
+
+Configuration
+=============
+
+PostgreSQL storage handler is a builtin storage handler. So, you can eaisly
register PostgreSQL databases to a Tajo cluster if you just add the following
line to ``conf/storage-site.json`` file. If you want to know more information
about ``storage-site.json``, please refer to
:doc:`/table_management/tablespaces`.
+
+.. code-block:: json
+
+ {
+ "spaces": {
+ "pgsql_db1": {
+ "uri": "jdbc:postgresql://hostname:port/db1"
+
+ "configs": {
+ "mapped_database": "tajo_db1"
+ "connection_properties": {
+ "user": "tajo",
+ "password": "xxxx"
+ }
+ }
+ }
+ }
+ }
+
+``configs`` allows users to specific additional configurations.
+``mapped_database`` specifies a database name shown in Tajo. In the example,
the database ``db1`` in PostgreSQL
+will be mapped to the database ``tajo_db1`` in Tajo.
+``connection_properties`` allows users to set JDBC connection parameters.
+Please refer to https://jdbc.postgresql.org/documentation/head/connect.html in
order to know the details of
+PostgreSQL connection parameters.
+
+The storage-site.json will be effective after you restart a tajo cluster.
\ No newline at end of file
Added: tajo/site/docs/devel/_sources/table_management/json.txt
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/_sources/table_management/json.txt?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/_sources/table_management/json.txt (added)
+++ tajo/site/docs/devel/_sources/table_management/json.txt Tue Oct 6 06:45:04
2015
@@ -0,0 +1,100 @@
+****
+JSON
+****
+
+JSON(JavaScript Object Notation) is an open standard format for data
(de)serialization. Since it is simple and human-readable, it is popularly used
in many fields.
+Tajo supports JSON as its data format. In this section, you will get an
overview of how to create JSON tables and query on them.
+
+============================
+How to Create a JSON Table ?
+============================
+
+You can create a JSON table using the ``CREATE TABLE`` statement. (For more
information, please refer to :doc:`/sql_language/ddl`.)
+For example, please consider an example data as follows:
+
+.. code-block:: bash
+
+ $ hdfs dfs -cat /table1/table.json
+ { "title" : "Hand of the King", "name" : { "first_name": "Eddard",
"last_name": "Stark"}}
+ { "title" : "Assassin", "name" : { "first_name": "Arya", "last_name":
"Stark"}}
+ { "title" : "Dancing Master", "name" : { "first_name": "Syrio", "last_name":
"Forel"}}
+
+Tajo provides two ways to create a table for this data. First is a traditional
way to create tables. Here is an example.
+
+.. code-block:: sql
+
+ CREATE EXTERNAL TABLE table1 (
+ title TEXT,
+ name RECORD (
+ first_name TEXT,
+ last_name TEXT
+ )
+ ) USING JSON LOCATION '/table1/table.json';
+
+With this way, you need to specify every column which they want to use. This
will be a tedious work, and not appropriate for flexible JSON schema.
+Second is a simpler alternative to alleviate this problem. When you create an
external table of JSON format, you can simply omit the column specification as
follows:
+
+.. code-block:: sql
+
+ CREATE EXTERNAL TABLE table1 (*) USING JSON LOCATION '/table1/table.json';
+
+No matter which way you choose, you can submit any queries on this table.
+
+.. code-block:: sql
+
+ > SELECT title, name.last_name from table1 where name.first_name = 'Arya';
+ title,name/last_name
+ -------------------------------
+ Assassin,Stark
+
+.. warning::
+
+ If you create a table with the second way, every column is assumed as the
``TEXT`` type.
+ So, you need to perform type casting if you want to handle them as other
types.
+
+===================
+Physical Properties
+===================
+
+Some table storage formats provide parameters for enabling or disabling
features and adjusting physical parameters.
+The ``WITH`` clause in the CREATE TABLE statement allows users to set those
parameters.
+
+The JSON format provides the following physical properties.
+
+* ``text.delimiter``: delimiter character. ``|`` or ``\u0001`` is usually
used, and the default field delimiter is ``|``.
+* ``text.null``: ``NULL`` character. The default ``NULL`` character is an
empty string ``''``. Hive's default ``NULL`` character is ``'\\N'``.
+* ``compression.codec``: Compression codec. You can enable compression feature
and set specified compression algorithm. The compression algorithm used to
compress files. The compression codec name should be the fully qualified class
name inherited from `org.apache.hadoop.io.compress.CompressionCodec
<https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html>`_.
By default, compression is disabled.
+* ``timezone``: the time zone that the table uses for writting. When table
rows are read or written, ```timestamp``` and ```time``` column values are
adjusted by this timezone if it is set. Time zone can be an abbreviation form
like 'PST' or 'DST'. Also, it accepts an offset-based form like 'UTC+9' or a
location-based form like 'Asia/Seoul'.
+* ``text.error-tolerance.max-num``: the maximum number of permissible parsing
errors. This value should be an integer value. By default,
``text.error-tolerance.max-num`` is ``0``. According to the value, parsing
errors will be handled in different ways.
+
+ * If ``text.error-tolerance.max-num < 0``, all parsing errors are ignored.
+ * If ``text.error-tolerance.max-num == 0``, any parsing error is not
allowed. If any error occurs, the query will be failed. (default)
+ * If ``text.error-tolerance.max-num > 0``, the given number of parsing
errors in each task will be pemissible.
+
+* ``text.skip.headerlines``: Number of header lines to be skipped. Some text
files often have a header which has a kind of metadata(e.g.: column names),
thus this option can be useful.
+
+The following example is to set a custom field delimiter, ``NULL`` character,
and compression codec:
+
+.. code-block:: sql
+
+ CREATE TABLE table1 (
+ id int,
+ name text,
+ score float,
+ type text
+ ) USING JSON WITH('text.delimiter'='\u0001',
+ 'text.null'='\\N',
+
'compression.codec'='org.apache.hadoop.io.compress.SnappyCodec');
+
+.. warning::
+
+ Be careful when using ``\n`` as the field delimiter because *TEXT* format
tables use ``\n`` as the line delimiter.
+ At the moment, Tajo does not provide a way to specify the line delimiter.
+
+==========================
+Null Value Handling Issues
+==========================
+In default, ``NULL`` character in *TEXT* format is an empty string ``''``.
+In other words, an empty field is basically recognized as a ``NULL`` value in
Tajo.
+If a field domain is ``TEXT``, an empty field is recognized as a string value
``''`` instead of ``NULL`` value.
+Besides, You can also use your own ``NULL`` character by specifying a physical
property ``text.null``.
Added: tajo/site/docs/devel/_sources/table_management/orc.txt
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/_sources/table_management/orc.txt?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/_sources/table_management/orc.txt (added)
+++ tajo/site/docs/devel/_sources/table_management/orc.txt Tue Oct 6 06:45:04
2015
@@ -0,0 +1,47 @@
+***
+ORC
+***
+
+**ORC(Optimized Row Columnar)** is a columnar storage format from Hive. ORC
improves performance for reading,
+writing, and processing data.
+For more details, please refer to `ORC Files
<https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC>`_ at Hive
wiki.
+
+===========================
+How to Create an ORC Table?
+===========================
+
+If you are not familiar with ``CREATE TABLE`` statement, please refer to Data
Definition Language :doc:`/sql_language/ddl`.
+
+In order to specify a certain file format for your table, you need to use the
``USING`` clause in your ``CREATE TABLE``
+statement. Below is an example statement for creating a table using orc files.
+
+.. code-block:: sql
+
+ CREATE TABLE table1 (
+ id int,
+ name text,
+ score float,
+ type text
+ ) USING orc;
+
+===================
+Physical Properties
+===================
+
+Some table storage formats provide parameters for enabling or disabling
features and adjusting physical parameters.
+The ``WITH`` clause in the CREATE TABLE statement allows users to set those
parameters.
+
+Now, ORC file provides the following physical properties.
+
+* ``orc.max.merge.distance``: When ORC file is read, if stripes are too closer
and the distance is lower than this value, they are merged and read at once.
Default is 1MB.
+* ``orc.stripe.size``: It decides size of each stripe. Default is 64MB.
+* ``orc.compression.kind``: It means the compression algorithm used to
compress and write data. It should be one of ``none``, ``snappy``, ``zlib``.
Default is ``none``.
+* ``orc.buffer.size``: It decides size of writing buffer. Default is 256KB.
+* ``orc.rowindex.stride``: Define the default ORC index stride in number of
rows. (Stride is the number of rows an index entry represents.) Default is
10000.
+
+======================================
+Compatibility Issues with Apache Hiveâ¢
+======================================
+
+At the moment, Tajo only supports flat relational tables.
+We are currently working on adding support for nested schemas and non-scalar
types (`TAJO-710 <https://issues.apache.org/jira/browse/TAJO-710>`_).
\ No newline at end of file
Added: tajo/site/docs/devel/storage_plugins.html
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/storage_plugins.html?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/storage_plugins.html (added)
+++ tajo/site/docs/devel/storage_plugins.html Tue Oct 6 06:45:04 2015
@@ -0,0 +1,283 @@
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>Storage Plugin — Apache Tajo 0.11.0 documentation</title>
+
+
+
+
+
+
+ <link
href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700'
rel='stylesheet' type='text/css'>
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
+
+
+
+ <link rel="top" title="Apache Tajo 0.11.0 documentation"
href="index.html"/>
+ <link rel="next" title="Storage Plugin Overview"
href="storage_plugins/overview.html"/>
+ <link rel="prev" title="Hash Partitioning"
href="partitioning/hash_partitioning.html"/>
+
+
+ <script
src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-nav-search">
+ <a href="index.html" class="fa fa-home"> Apache Tajo</a>
+ <div role="search">
+ <form id ="rtd-search-form" class="wy-form" action="search.html"
method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation"
aria-label="main navigation">
+
+
+ <ul class="current">
+<li class="toctree-l1"><a class="reference internal"
href="introduction.html">Introduction</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="getting_started.html">Getting Started</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="getting_started.html#prerequisites">Prerequisites</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="getting_started.html#dowload-and-unpack-the-source-code">Dowload and
unpack the source code</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="getting_started.html#build-source-code">Build source code</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="getting_started.html#setting-up-a-local-tajo-cluster">Setting up a local
Tajo cluster</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="getting_started.html#first-query-execution">First query execution</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="configuration.html">Configuration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/preliminary.html">Preliminary</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/cluster_setup.html">Cluster Setup</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/tajo_master_configuration.html">Tajo Master
Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/worker_configuration.html">Worker Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/catalog_configuration.html">Catalog Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/ha_configuration.html">High Availability for
TajoMaster</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/service_config_defaults.html">Cluster Service Configuration
Defaults</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="tsql.html">Tajo
Shell (TSQL)</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/meta_command.html">Meta Commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/dfs_command.html">Executing HDFS commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/variables.html">Session Variables</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/admin_command.html">Administration Commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/intro.html">Introducing to TSQL</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/single_command.html">Executing a single command</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/execute_file.html">Executing Queries from Files</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="tsql/background_command.html">Executing as background process</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="sql_language.html">SQL Language</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/data_model.html">Data Model</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/ddl.html">Data Definition Language</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/alter_table.html">ALTER TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/queries.html">Queries</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/joins.html">Joins</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/sql_expression.html">SQL Expressions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="sql_language/predicates.html">Predicates</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="time_zone.html">Time Zone</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="time_zone.html#server-cluster-time-zone">Server Cluster Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="time_zone.html#table-time-zone">Table Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="time_zone.html#client-time-zone">Client Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="time_zone.html#time-zone-id">Time Zone ID</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="functions.html">Functions</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="functions.html#built-in-scalar-functions">Built-in Scalar
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="functions.html#built-in-aggregation-functions">Built-in Aggregation
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="functions.html#built-in-window-functions">Built-in Window
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="functions.html#user-defined-functions">User-defined Functions</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="table_management.html">Table Management</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="table_management/table_overview.html">Overview of Tajo Tables</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="table_management/tablespaces.html">Tablespaces</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="table_management/compression.html">Compression</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="table_partitioning.html">Table Partitioning</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="partitioning/intro_to_partitioning.html">Introduction to
Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="partitioning/column_partitioning.html">Column Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="partitioning/range_partitioning.html">Range Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="partitioning/hash_partitioning.html">Hash Partitioning</a></li>
+</ul>
+</li>
+<li class="toctree-l1 current"><a class="current reference internal"
href="">Storage Plugin</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="storage_plugins/overview.html">Storage Plugin Overview</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="storage_plugins/postgresql.html">PostgreSQL Storage Handler</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="index_overview.html">Index (Experimental Feature)</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="index/types.html">Index Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="index/how_to_use.html">How to use index?</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="index/future_work.html">Future Works</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="backup_and_restore.html">Backup and Restore</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="backup_and_restore/catalog.html">Backup and Restore Catalog</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="hive_integration.html">Hive Integration</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="hbase_integration.html">HBase Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="hbase_integration.html#create-table">CREATE TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="hbase_integration.html#drop-table">DROP TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE)
INTO</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="hbase_integration.html#usage">Usage</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="swift_integration.html">OpenStack Swift Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="swift_integration.html#swift-configuration">Swift configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="swift_integration.html#hadoop-configurations">Hadoop
configurations</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="swift_integration.html#tajo-configuration">Tajo configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="swift_integration.html#querying-on-swift">Querying on Swift</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="jdbc_driver.html">Tajo JDBC Driver</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC driver</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="jdbc_driver.html#connecting-to-the-tajo-cluster-instance">Connecting to
the Tajo cluster instance</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="jdbc_driver.html#connection-parameters">Connection Parameters</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="jdbc_driver.html#an-example-jdbc-client">An Example JDBC Client</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="tajo_client_api.html">Tajo Client API</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="faq.html">FAQ</a></li>
+</ul>
+
+
+ </div>
+
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="index.html">Apache Tajo</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="index.html">Docs</a> »</li>
+
+ <li>Storage Plugin</li>
+ <li class="wy-breadcrumbs-aside">
+
+ <a href="_sources/storage_plugins.txt" rel="nofollow"> View page
source</a>
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main">
+
+ <div class="section" id="storage-plugin">
+<h1>Storage Plugin<a class="headerlink" href="#storage-plugin"
title="Permalink to this headline">¶</a></h1>
+<p>This section describes the storage plugins available in Tajo to access
datasets from different data sources.</p>
+<div class="toctree-wrapper compound">
+<ul>
+<li class="toctree-l1"><a class="reference internal"
href="storage_plugins/overview.html">Storage Plugin Overview</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="storage_plugins/postgresql.html">PostgreSQL Storage Handler</a></li>
+</ul>
+</div>
+</div>
+
+
+ </div>
+ <footer>
+
+ <div class="rst-footer-buttons" role="navigation" aria-label="footer
navigation">
+
+ <a href="storage_plugins/overview.html" class="btn btn-neutral
float-right" title="Storage Plugin Overview"/>Next <span class="fa
fa-arrow-circle-right"></span></a>
+
+
+ <a href="partitioning/hash_partitioning.html" class="btn btn-neutral"
title="Hash Partitioning"><span class="fa fa-arrow-circle-left"></span>
Previous</a>
+
+ </div>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Apache Tajo Team.
+ </p>
+ </div>
+
+ <a href="https://github.com/snide/sphinx_rtd_theme";>Sphinx theme</a>
provided by <a href="https://readthedocs.org";>Read the Docs</a>
+</footer>
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'./',
+ VERSION:'0.11.0',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="_static/jquery.js"></script>
+ <script type="text/javascript" src="_static/underscore.js"></script>
+ <script type="text/javascript" src="_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file
Added: tajo/site/docs/devel/storage_plugins/overview.html
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/storage_plugins/overview.html?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/storage_plugins/overview.html (added)
+++ tajo/site/docs/devel/storage_plugins/overview.html Tue Oct 6 06:45:04 2015
@@ -0,0 +1,314 @@
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>Storage Plugin Overview — Apache Tajo 0.11.0
documentation</title>
+
+
+
+
+
+
+ <link
href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700'
rel='stylesheet' type='text/css'>
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
+
+
+
+ <link rel="top" title="Apache Tajo 0.11.0 documentation"
href="../index.html"/>
+ <link rel="up" title="Storage Plugin" href="../storage_plugins.html"/>
+ <link rel="next" title="PostgreSQL Storage Handler"
href="postgresql.html"/>
+ <link rel="prev" title="Storage Plugin"
href="../storage_plugins.html"/>
+
+
+ <script
src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-nav-search">
+ <a href="../index.html" class="fa fa-home"> Apache Tajo</a>
+ <div role="search">
+ <form id ="rtd-search-form" class="wy-form" action="../search.html"
method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation"
aria-label="main navigation">
+
+
+ <ul class="current">
+<li class="toctree-l1"><a class="reference internal"
href="../introduction.html">Introduction</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="../getting_started.html">Getting Started</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#prerequisites">Prerequisites</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#dowload-and-unpack-the-source-code">Dowload and
unpack the source code</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#build-source-code">Build source code</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#setting-up-a-local-tajo-cluster">Setting up a
local Tajo cluster</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#first-query-execution">First query
execution</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../configuration.html">Configuration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/preliminary.html">Preliminary</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/cluster_setup.html">Cluster Setup</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/tajo_master_configuration.html">Tajo Master
Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/worker_configuration.html">Worker Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/catalog_configuration.html">Catalog
Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/ha_configuration.html">High Availability for
TajoMaster</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/service_config_defaults.html">Cluster Service
Configuration Defaults</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo
Shell (TSQL)</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/meta_command.html">Meta Commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/dfs_command.html">Executing HDFS commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/variables.html">Session Variables</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/admin_command.html">Administration Commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/intro.html">Introducing to TSQL</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/single_command.html">Executing a single command</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/execute_file.html">Executing Queries from Files</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/background_command.html">Executing as background process</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../sql_language.html">SQL Language</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/data_model.html">Data Model</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/ddl.html">Data Definition Language</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/alter_table.html">ALTER TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/queries.html">Queries</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/joins.html">Joins</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/sql_expression.html">SQL Expressions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/predicates.html">Predicates</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../time_zone.html">Time Zone</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#server-cluster-time-zone">Server Cluster Time
Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#table-time-zone">Table Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#client-time-zone">Client Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#time-zone-id">Time Zone ID</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../functions.html">Functions</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#built-in-scalar-functions">Built-in Scalar
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#built-in-aggregation-functions">Built-in Aggregation
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#built-in-window-functions">Built-in Window
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#user-defined-functions">User-defined Functions</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../table_management.html">Table Management</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/tablespaces.html">Tablespaces</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/compression.html">Compression</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../table_partitioning.html">Table Partitioning</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/intro_to_partitioning.html">Introduction to
Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/column_partitioning.html">Column Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/range_partitioning.html">Range Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/hash_partitioning.html">Hash Partitioning</a></li>
+</ul>
+</li>
+<li class="toctree-l1 current"><a class="reference internal"
href="../storage_plugins.html">Storage Plugin</a><ul class="current">
+<li class="toctree-l2 current"><a class="current reference internal"
href="">Storage Plugin Overview</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="postgresql.html">PostgreSQL Storage Handler</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../index_overview.html">Index (Experimental Feature)</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../index/types.html">Index Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../index/how_to_use.html">How to use index?</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../index/future_work.html">Future Works</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../backup_and_restore.html">Backup and Restore</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../backup_and_restore/catalog.html">Backup and Restore Catalog</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../hive_integration.html">Hive Integration</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="../hbase_integration.html">HBase Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#create-table">CREATE TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#drop-table">DROP TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE)
INTO</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#usage">Usage</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../swift_integration.html">OpenStack Swift Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#swift-configuration">Swift
configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#hadoop-configurations">Hadoop
configurations</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#tajo-configuration">Tajo configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#querying-on-swift">Querying on Swift</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../jdbc_driver.html">Tajo JDBC Driver</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC
driver</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#connecting-to-the-tajo-cluster-instance">Connecting
to the Tajo cluster instance</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#connection-parameters">Connection Parameters</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#an-example-jdbc-client">An Example JDBC
Client</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../tajo_client_api.html">Tajo Client API</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="../faq.html">FAQ</a></li>
+</ul>
+
+
+ </div>
+
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="../index.html">Apache Tajo</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="../index.html">Docs</a> »</li>
+
+ <li><a href="../storage_plugins.html">Storage Plugin</a> »</li>
+
+ <li>Storage Plugin Overview</li>
+ <li class="wy-breadcrumbs-aside">
+
+ <a href="../_sources/storage_plugins/overview.txt" rel="nofollow">
View page source</a>
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main">
+
+ <div class="section" id="storage-plugin-overview">
+<h1>Storage Plugin Overview<a class="headerlink"
href="#storage-plugin-overview" title="Permalink to this headline">¶</a></h1>
+<div class="section" id="overview">
+<h2>Overview<a class="headerlink" href="#overview" title="Permalink to this
headline">¶</a></h2>
+<p>Tajo supports various storage systems, such as HDFS, Amazon S3, Openstack
Swift, HBase, and RDBMS. Tajo already embeds HDFS, S3, Openstack, HBase, RDBMS
storage plugins, and also Tajo allows users to register custom storages and
data formats to Tajo cluster instances. This section describes how you register
custom storages and data types.</p>
+</div>
+<div class="section" id="register-custom-storage">
+<h2>Register custom storage<a class="headerlink"
href="#register-custom-storage" title="Permalink to this headline">¶</a></h2>
+<p>First of all, your storage implementation should be packed as a jar file.
Then, please copy the jar file into <code class="docutils literal"><span
class="pre">tajo/extlib</span></code> directory. Next, you should copy <code
class="docutils literal"><span
class="pre">conf/storage-site.json.template</span></code> into <code
class="docutils literal"><span class="pre">conf/storage-site.json</span></code>
and modify the file like the below.</p>
+</div>
+<div class="section" id="configuration">
+<h2>Configuration<a class="headerlink" href="#configuration" title="Permalink
to this headline">¶</a></h2>
+<p>Tajo has a default configuration for builtin storages, such as HDFS, local
file system, and Amazon S3. it also allows users to add custom storage
plugins</p>
+<p><code class="docutils literal"><span
class="pre">conf/storage-site.json</span></code> file has the following
struct:</p>
+<div class="highlight-json"><div class="highlight"><pre><span
class="p">{</span>
+ <span class="nt">"storages"</span><span class="p">:</span> <span
class="p">{</span>
+ <span class="nt">"${scheme}"</span><span class="p">:</span>
<span class="p">{</span>
+ <span class="nt">"handler"</span><span class="p">:</span>
<span class="s2">"${class name}"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+</div>
+<p>Each storage instance (i.e., <a class="reference internal"
href="../table_management/tablespaces.html"><em>Tablespaces</em></a>) is
identified by an URI. The scheme of URI plays a role to identify storage type.
For example, <code class="docutils literal"><span
class="pre">hdfs://</span></code> is used for Hdfs storage, <code
class="docutils literal"><span class="pre">jdbc://</span></code> is used for
JDBC-based storage, and <code class="docutils literal"><span
class="pre">hbase://</span></code> is used for HBase storage.</p>
+<p>You should substitute a scheme name without <code class="docutils
literal"><span class="pre">://</span></code> for <code class="docutils
literal"><span class="pre">${scheme}</span></code>.</p>
+<p>See an example for HBase storage.</p>
+<div class="highlight-json"><div class="highlight"><pre><span
class="p">{</span>
+ <span class="nt">"storages"</span><span class="p">:</span> <span
class="p">{</span>
+ <span class="nt">"hbase"</span><span class="p">:</span> <span
class="p">{</span>
+ <span class="nt">"handler"</span><span class="p">:</span>
<span
class="s2">"org.apache.tajo.storage.hbase.HBaseTablespace"</span><span
class="p">,</span>
+ <span class="nt">"default-format"</span><span
class="p">:</span> <span class="s2">"hbase"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+</div>
+</div>
+</div>
+
+
+ </div>
+ <footer>
+
+ <div class="rst-footer-buttons" role="navigation" aria-label="footer
navigation">
+
+ <a href="postgresql.html" class="btn btn-neutral float-right"
title="PostgreSQL Storage Handler"/>Next <span class="fa
fa-arrow-circle-right"></span></a>
+
+
+ <a href="../storage_plugins.html" class="btn btn-neutral"
title="Storage Plugin"><span class="fa fa-arrow-circle-left"></span>
Previous</a>
+
+ </div>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Apache Tajo Team.
+ </p>
+ </div>
+
+ <a href="https://github.com/snide/sphinx_rtd_theme";>Sphinx theme</a>
provided by <a href="https://readthedocs.org";>Read the Docs</a>
+</footer>
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'../',
+ VERSION:'0.11.0',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="../_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file
Added: tajo/site/docs/devel/storage_plugins/postgresql.html
URL:
http://svn.apache.org/viewvc/tajo/site/docs/devel/storage_plugins/postgresql.html?rev=1706955&view=auto
==============================================================================
--- tajo/site/docs/devel/storage_plugins/postgresql.html (added)
+++ tajo/site/docs/devel/storage_plugins/postgresql.html Tue Oct 6 06:45:04
2015
@@ -0,0 +1,311 @@
+
+
+<!DOCTYPE html>
+<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
+<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
+<head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <title>PostgreSQL Storage Handler — Apache Tajo 0.11.0
documentation</title>
+
+
+
+
+
+
+ <link
href='https://fonts.googleapis.com/css?family=Lato:400,700|Roboto+Slab:400,700|Inconsolata:400,700'
rel='stylesheet' type='text/css'>
+
+
+
+
+
+
+
+
+
+ <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
+
+
+
+ <link rel="top" title="Apache Tajo 0.11.0 documentation"
href="../index.html"/>
+ <link rel="up" title="Storage Plugin" href="../storage_plugins.html"/>
+ <link rel="next" title="Index (Experimental Feature)"
href="../index_overview.html"/>
+ <link rel="prev" title="Storage Plugin Overview"
href="overview.html"/>
+
+
+ <script
src="https://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js";></script>
+
+</head>
+
+<body class="wy-body-for-nav" role="document">
+
+ <div class="wy-grid-for-nav">
+
+
+ <nav data-toggle="wy-nav-shift" class="wy-nav-side">
+ <div class="wy-side-nav-search">
+ <a href="../index.html" class="fa fa-home"> Apache Tajo</a>
+ <div role="search">
+ <form id ="rtd-search-form" class="wy-form" action="../search.html"
method="get">
+ <input type="text" name="q" placeholder="Search docs" />
+ <input type="hidden" name="check_keywords" value="yes" />
+ <input type="hidden" name="area" value="default" />
+ </form>
+</div>
+ </div>
+
+ <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation"
aria-label="main navigation">
+
+
+ <ul class="current">
+<li class="toctree-l1"><a class="reference internal"
href="../introduction.html">Introduction</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="../getting_started.html">Getting Started</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#prerequisites">Prerequisites</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#dowload-and-unpack-the-source-code">Dowload and
unpack the source code</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#build-source-code">Build source code</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#setting-up-a-local-tajo-cluster">Setting up a
local Tajo cluster</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../getting_started.html#first-query-execution">First query
execution</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../configuration.html">Configuration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/preliminary.html">Preliminary</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/cluster_setup.html">Cluster Setup</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/tajo_master_configuration.html">Tajo Master
Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/worker_configuration.html">Worker Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/catalog_configuration.html">Catalog
Configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/ha_configuration.html">High Availability for
TajoMaster</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/service_config_defaults.html">Cluster Service
Configuration Defaults</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/tajo-site-xml.html">The tajo-site.xml File</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../configuration/catalog-site-xml.html">The catalog-site.xml File</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal" href="../tsql.html">Tajo
Shell (TSQL)</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/meta_command.html">Meta Commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/dfs_command.html">Executing HDFS commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/variables.html">Session Variables</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/admin_command.html">Administration Commands</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/intro.html">Introducing to TSQL</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/single_command.html">Executing a single command</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/execute_file.html">Executing Queries from Files</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../tsql/background_command.html">Executing as background process</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../sql_language.html">SQL Language</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/data_model.html">Data Model</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/ddl.html">Data Definition Language</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/insert.html">INSERT (OVERWRITE) INTO</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/alter_table.html">ALTER TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/queries.html">Queries</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/joins.html">Joins</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/sql_expression.html">SQL Expressions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../sql_language/predicates.html">Predicates</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../time_zone.html">Time Zone</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#server-cluster-time-zone">Server Cluster Time
Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#table-time-zone">Table Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#client-time-zone">Client Time Zone</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#time-zone-id">Time Zone ID</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../time_zone.html#examples-of-time-zone">Examples of Time Zone</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../functions.html">Functions</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#built-in-scalar-functions">Built-in Scalar
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#built-in-aggregation-functions">Built-in Aggregation
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#built-in-window-functions">Built-in Window
Functions</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../functions.html#user-defined-functions">User-defined Functions</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../table_management.html">Table Management</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/table_overview.html">Overview of Tajo Tables</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/tablespaces.html">Tablespaces</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/file_formats.html">File Formats</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../table_management/compression.html">Compression</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../table_partitioning.html">Table Partitioning</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/intro_to_partitioning.html">Introduction to
Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/column_partitioning.html">Column Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/range_partitioning.html">Range Partitioning</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../partitioning/hash_partitioning.html">Hash Partitioning</a></li>
+</ul>
+</li>
+<li class="toctree-l1 current"><a class="reference internal"
href="../storage_plugins.html">Storage Plugin</a><ul class="current">
+<li class="toctree-l2"><a class="reference internal"
href="overview.html">Storage Plugin Overview</a></li>
+<li class="toctree-l2 current"><a class="current reference internal"
href="">PostgreSQL Storage Handler</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../index_overview.html">Index (Experimental Feature)</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../index/types.html">Index Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../index/how_to_use.html">How to use index?</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../index/future_work.html">Future Works</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../backup_and_restore.html">Backup and Restore</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../backup_and_restore/catalog.html">Backup and Restore Catalog</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../hive_integration.html">Hive Integration</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="../hbase_integration.html">HBase Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#create-table">CREATE TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#drop-table">DROP TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#insert-overwrite-into">INSERT (OVERWRITE)
INTO</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../hbase_integration.html#usage">Usage</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../swift_integration.html">OpenStack Swift Integration</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#swift-configuration">Swift
configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#hadoop-configurations">Hadoop
configurations</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#tajo-configuration">Tajo configuration</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../swift_integration.html#querying-on-swift">Querying on Swift</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../jdbc_driver.html">Tajo JDBC Driver</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#how-to-get-jdbc-driver">How to get JDBC
driver</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#setting-the-classpath">Setting the CLASSPATH</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#connecting-to-the-tajo-cluster-instance">Connecting
to the Tajo cluster instance</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#connection-parameters">Connection Parameters</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="../jdbc_driver.html#an-example-jdbc-client">An Example JDBC
Client</a></li>
+</ul>
+</li>
+<li class="toctree-l1"><a class="reference internal"
href="../tajo_client_api.html">Tajo Client API</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="../faq.html">FAQ</a></li>
+</ul>
+
+
+ </div>
+
+ </nav>
+
+ <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
+
+
+ <nav class="wy-nav-top" role="navigation" aria-label="top navigation">
+ <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
+ <a href="../index.html">Apache Tajo</a>
+ </nav>
+
+
+
+ <div class="wy-nav-content">
+ <div class="rst-content">
+ <div role="navigation" aria-label="breadcrumbs navigation">
+ <ul class="wy-breadcrumbs">
+ <li><a href="../index.html">Docs</a> »</li>
+
+ <li><a href="../storage_plugins.html">Storage Plugin</a> »</li>
+
+ <li>PostgreSQL Storage Handler</li>
+ <li class="wy-breadcrumbs-aside">
+
+ <a href="../_sources/storage_plugins/postgresql.txt" rel="nofollow">
View page source</a>
+
+ </li>
+ </ul>
+ <hr/>
+</div>
+ <div role="main">
+
+ <div class="section" id="postgresql-storage-handler">
+<h1>PostgreSQL Storage Handler<a class="headerlink"
href="#postgresql-storage-handler" title="Permalink to this
headline">¶</a></h1>
+<div class="section" id="overview">
+<h2>Overview<a class="headerlink" href="#overview" title="Permalink to this
headline">¶</a></h2>
+<p>PostgreSQL storage handler is available by default in Tajo. It enables
users’ queries to access database objects in PostgreSQL. Tables in
PostgreSQL will be shown as tables in Tajo too. Most of the SQL queries used
for PostgreSQL are available in Tajo via this storage handles. Its main
advantages is to allow federated query processing among tables in stored HDFS
and PostgreSQL.</p>
+</div>
+<div class="section" id="configuration">
+<h2>Configuration<a class="headerlink" href="#configuration" title="Permalink
to this headline">¶</a></h2>
+<p>PostgreSQL storage handler is a builtin storage handler. So, you can eaisly
register PostgreSQL databases to a Tajo cluster if you just add the following
line to <code class="docutils literal"><span
class="pre">conf/storage-site.json</span></code> file. If you want to know more
information about <code class="docutils literal"><span
class="pre">storage-site.json</span></code>, please refer to <a
class="reference internal"
href="../table_management/tablespaces.html"><em>Tablespaces</em></a>.</p>
+<div class="highlight-json"><div class="highlight"><pre><span
class="p">{</span>
+ <span class="nt">"spaces"</span><span class="p">:</span> <span
class="p">{</span>
+ <span class="nt">"pgsql_db1"</span><span class="p">:</span>
<span class="p">{</span>
+ <span class="nt">"uri"</span><span class="p">:</span> <span
class="s2">"jdbc:postgresql://hostname:port/db1"</span>
+
+ <span class="s2">"configs"</span><span class="p">:</span>
<span class="p">{</span>
+ <span class="nt">"mapped_database"</span><span
class="p">:</span> <span class="s2">"tajo_db1"</span>
+ <span class="s2">"connection_properties"</span><span
class="p">:</span> <span class="p">{</span>
+ <span class="nt">"user"</span><span class="p">:</span>
<span class="s2">"tajo"</span><span class="p">,</span>
+ <span class="nt">"password"</span><span class="p">:</span>
<span class="s2">"xxxx"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+</div>
+<p><code class="docutils literal"><span class="pre">configs</span></code>
allows users to specific additional configurations.
+<code class="docutils literal"><span class="pre">mapped_database</span></code>
specifies a database name shown in Tajo. In the example, the database <code
class="docutils literal"><span class="pre">db1</span></code> in PostgreSQL
+will be mapped to the database <code class="docutils literal"><span
class="pre">tajo_db1</span></code> in Tajo.
+<code class="docutils literal"><span
class="pre">connection_properties</span></code> allows users to set JDBC
connection parameters.
+Please refer to <a class="reference external"
href="https://jdbc.postgresql.org/documentation/head/connect.html";>https://jdbc.postgresql.org/documentation/head/connect.html</a>
in order to know the details of
+PostgreSQL connection parameters.</p>
+<p>The storage-site.json will be effective after you restart a tajo
cluster.</p>
+</div>
+</div>
+
+
+ </div>
+ <footer>
+
+ <div class="rst-footer-buttons" role="navigation" aria-label="footer
navigation">
+
+ <a href="../index_overview.html" class="btn btn-neutral float-right"
title="Index (Experimental Feature)"/>Next <span class="fa
fa-arrow-circle-right"></span></a>
+
+
+ <a href="overview.html" class="btn btn-neutral" title="Storage Plugin
Overview"><span class="fa fa-arrow-circle-left"></span> Previous</a>
+
+ </div>
+
+
+ <hr/>
+
+ <div role="contentinfo">
+ <p>
+ © Copyright 2014, Apache Tajo Team.
+ </p>
+ </div>
+
+ <a href="https://github.com/snide/sphinx_rtd_theme";>Sphinx theme</a>
provided by <a href="https://readthedocs.org";>Read the Docs</a>
+</footer>
+ </div>
+ </div>
+
+ </section>
+
+ </div>
+
+
+
+
+
+ <script type="text/javascript">
+ var DOCUMENTATION_OPTIONS = {
+ URL_ROOT:'../',
+ VERSION:'0.11.0',
+ COLLAPSE_INDEX:false,
+ FILE_SUFFIX:'.html',
+ HAS_SOURCE: true
+ };
+ </script>
+ <script type="text/javascript" src="../_static/jquery.js"></script>
+ <script type="text/javascript" src="../_static/underscore.js"></script>
+ <script type="text/javascript" src="../_static/doctools.js"></script>
+
+
+
+
+
+ <script type="text/javascript" src="../_static/js/theme.js"></script>
+
+
+
+
+ <script type="text/javascript">
+ jQuery(function () {
+ SphinxRtdTheme.StickyNav.enable();
+ });
+ </script>
+
+
+</body>
+</html>
\ No newline at end of file
