Repository: tajo Updated Branches: refs/heads/master 0af53d025 -> f2552bfb9
TAJO-1855: Add document for tablespace configuration to the Configuration chapter. Closes #873 Signed-off-by: Jihoon Son <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/tajo/repo Commit: http://git-wip-us.apache.org/repos/asf/tajo/commit/f2552bfb Tree: http://git-wip-us.apache.org/repos/asf/tajo/tree/f2552bfb Diff: http://git-wip-us.apache.org/repos/asf/tajo/diff/f2552bfb Branch: refs/heads/master Commit: f2552bfb9c0a3b57f66e89a9e635a0d1afbeebc8 Parents: 0af53d0 Author: Jongyoung Park <[email protected]> Authored: Wed Dec 2 21:41:13 2015 +0900 Committer: Jihoon Son <[email protected]> Committed: Wed Dec 2 21:41:13 2015 +0900 ---------------------------------------------------------------------- CHANGES | 3 + tajo-docs/src/main/sphinx/conf.py | 2 +- tajo-docs/src/main/sphinx/configuration.rst | 7 ++- .../sphinx/configuration/storage-site-json.rst | 64 ++++++++++++++++++++ .../sphinx/table_management/tablespaces.rst | 12 ++-- 5 files changed, 80 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/CHANGES ---------------------------------------------------------------------- diff --git a/CHANGES b/CHANGES index 86ed3cd..e0fd693 100644 --- a/CHANGES +++ b/CHANGES @@ -92,6 +92,9 @@ Release 0.12.0 - unreleased TASKS + TAJO-1855: Add document for tablespace configuration to the Configuration chapter. + (Contributed by Jongyoung Park. Committed by jihoon) + TAJO-1854: Add in-subquery document. (jihoon) TAJO-2003: Fix typo in worker configuration. (jinho) http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/conf.py ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/conf.py b/tajo-docs/src/main/sphinx/conf.py index f6a2d96..dbd9e14 100644 --- a/tajo-docs/src/main/sphinx/conf.py +++ b/tajo-docs/src/main/sphinx/conf.py @@ -58,7 +58,7 @@ master_doc = 'index' # General information about the project. project = u'Apache Tajo' -copyright = u'2014, Apache Tajo Team' +copyright = u'2015, Apache Tajo Team' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/configuration.rst ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/configuration.rst b/tajo-docs/src/main/sphinx/configuration.rst index 5214ea8..8b0b8a2 100644 --- a/tajo-docs/src/main/sphinx/configuration.rst +++ b/tajo-docs/src/main/sphinx/configuration.rst @@ -1,6 +1,6 @@ -************************ +************* Configuration -************************ +************* .. toctree:: :maxdepth: 1 @@ -13,4 +13,5 @@ Configuration configuration/ha_configuration configuration/service_config_defaults configuration/tajo-site-xml - configuration/catalog-site-xml \ No newline at end of file + configuration/catalog-site-xml + configuration/storage-site-json http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst b/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst new file mode 100644 index 0000000..d377551 --- /dev/null +++ b/tajo-docs/src/main/sphinx/configuration/storage-site-json.rst @@ -0,0 +1,64 @@ +************************** +The storage-site.json File +************************** + +This file is for configuring :doc:`/table_management/tablespaces`. +The syntax of ``storage-site.json`` is like this basically: + +.. code:: json + + { + "spaces": { + <space config 1>, <space config 2>, ... + }, + "storages": { + <storage config 1>, <storage config 2>, ... + } + } + +------ +Spaces +------ + +This is a section for registering tablespaces. Some space config example is here: + +.. code:: json + + "spaces": { + "jdbc_pgsql": { + "uri": "jdbc:postgresql://127.0.0.1:5740/origin", + "configs": { + "mapped_database":"tajo_pgsql_origin", + "connection_properties": { + "user":"tajouser", + "password":"123456" + } + } + }, + ... + } + +* **space name** : Your own tablespace name which indicates a specific tablespace. Alpha-numeric characters and underscore(_) are permitted. +* **uri** : An URI address of a tablespace +* **configs** : It is optional. You can specify it as JSON object to pass to each tablespace handler. + +After you specify a tablespace, you can use it in `create table statement <../sql_language/ddl.html#create-table>`_. + +-------- +Storages +-------- + +This is for registering storage format and custom storage handler class. +Tajo already supports HDFS, HBase, PostgreSQL, Amazon S3, Openstack Swift, etc, thus in usual cases using mentioned storages, you don't have to add any ``storages`` configuration. +However, if you want to use your custom storage as one of Tajo's data source, you need to add a configuration for your storage. Here is an example of HDFS storage. +See :doc:`storage_plugins/overview` for more information. + +.. code:: json + + "storages": { + "hdfs": { + "handler": "org.apache.tajo.storage.FileTablespace", + "default-format": "text" + }, + ... + } http://git-wip-us.apache.org/repos/asf/tajo/blob/f2552bfb/tajo-docs/src/main/sphinx/table_management/tablespaces.rst ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/table_management/tablespaces.rst b/tajo-docs/src/main/sphinx/table_management/tablespaces.rst index 6f43af1..ff90483 100644 --- a/tajo-docs/src/main/sphinx/table_management/tablespaces.rst +++ b/tajo-docs/src/main/sphinx/table_management/tablespaces.rst @@ -26,9 +26,11 @@ If the tablespace is not specified, the default tablespace of the table's databa Configuration ============= -By default, Tajo use in ``${tajo.rootdir}/warehouse`` in ``conf/tajo-site.xml`` as a default tablespace. It also allows users to register additional tablespaces. +By default, Tajo use in ``${tajo.rootdir}/warehouse`` in :doc:`conf/tajo-site.xml</configuration/tajo-site-xml>` as a default tablespace. It also allows users to register additional tablespaces using ``storage-site.json`` file like below. -``conf/storage-site.json`` file. +--------------------------- +conf/storage-site.json file +--------------------------- The configuration file has the following struct: @@ -36,7 +38,7 @@ The configuration file has the following struct: { "spaces": { - "${table_space_name}": { + "${tablespace_name}": { "uri": "hbase://quorum1:port,quorum2:port/" } } @@ -48,7 +50,7 @@ The following is an example for two tablespaces for hbase and hdfs: { "spaces": { - "hbase-cluster1": { + "hbase_cluster1": { "uri": "hbase://quorum1:port,quorum2:port/" }, @@ -58,6 +60,8 @@ The following is an example for two tablespaces for hbase and hdfs: } } +For more details, see :doc:`conf/storage-site.json</configuration/storage-site-json>`. + .. note::
