Repository: tajo Updated Branches: refs/heads/master 68dd96065 -> f44f92fb0
TAJO-1705: Update document for index support. Closes #887 Project: http://git-wip-us.apache.org/repos/asf/tajo/repo Commit: http://git-wip-us.apache.org/repos/asf/tajo/commit/f44f92fb Tree: http://git-wip-us.apache.org/repos/asf/tajo/tree/f44f92fb Diff: http://git-wip-us.apache.org/repos/asf/tajo/diff/f44f92fb Branch: refs/heads/master Commit: f44f92fb05cacea0bc78245220ec9ae1d24c55be Parents: 68dd960 Author: Jihoon Son <[email protected]> Authored: Wed Jan 6 16:45:48 2016 +0900 Committer: Jihoon Son <[email protected]> Committed: Wed Jan 6 16:45:48 2016 +0900 ---------------------------------------------------------------------- CHANGES | 2 ++ tajo-docs/src/main/sphinx/index/future_work.rst | 3 +- tajo-docs/src/main/sphinx/index/how_to_use.rst | 32 +++++++++++++------- tajo-docs/src/main/sphinx/index/types.rst | 4 +-- tajo-docs/src/main/sphinx/index_overview.rst | 6 ---- 5 files changed, 26 insertions(+), 21 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/tajo/blob/f44f92fb/CHANGES ---------------------------------------------------------------------- diff --git a/CHANGES b/CHANGES index 6536009..2184cde 100644 --- a/CHANGES +++ b/CHANGES @@ -138,6 +138,8 @@ Release 0.12.0 - unreleased TASKS + TAJO-1705: Update document for index support. (jihoon) + TAJO-2028: Refining Hive Integration document including typo. (Jongyoung Park via jihoon) http://git-wip-us.apache.org/repos/asf/tajo/blob/f44f92fb/tajo-docs/src/main/sphinx/index/future_work.rst ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/index/future_work.rst b/tajo-docs/src/main/sphinx/index/future_work.rst index c6ec47d..976a34f 100644 --- a/tajo-docs/src/main/sphinx/index/future_work.rst +++ b/tajo-docs/src/main/sphinx/index/future_work.rst @@ -2,7 +2,6 @@ Future Works ************************************* -* Providing more index types, such as bitmap and HBase index +* Supporting more index types, such as bitmap and HBase index * Supporting index on partitioned tables -* Supporting the backup and restore feature * Cost-based query optimization by estimating the query selectivity \ No newline at end of file http://git-wip-us.apache.org/repos/asf/tajo/blob/f44f92fb/tajo-docs/src/main/sphinx/index/how_to_use.rst ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/index/how_to_use.rst b/tajo-docs/src/main/sphinx/index/how_to_use.rst index c67af66..a56979d 100644 --- a/tajo-docs/src/main/sphinx/index/how_to_use.rst +++ b/tajo-docs/src/main/sphinx/index/how_to_use.rst @@ -1,18 +1,19 @@ -************************************* +***************** How to use index? -************************************* +***************** -------------------------------------- +--------------- 1. Create index -------------------------------------- +--------------- -The first step for utilizing index is index creation. You can create index using SQL (:doc:`/sql_language/ddl`) or Tajo API (:doc:`/tajo_client_api`). For example, you can create a BST index on the lineitem table by submitting the following SQL to Tajo. +The first step for utilizing index is to create an index. You can create an index using SQL (:doc:`/sql_language/ddl`) or Tajo API (:doc:`/tajo_client_api`). +For example, the following SQL statement will create a BST index on the lineitem table. .. code-block:: sql default> create index l_orderkey_idx on lineitem (l_orderkey); -If the index is created successfully, you can see the information about that index as follows: :: +If the index is created successfully, you can see the index information as follows: :: default> \d lineitem @@ -48,17 +49,18 @@ If the index is created successfully, you can see the information about that ind For more information about index creation, please refer to the above links. -------------------------------------- +----------------------------- 2. Enable/disable index scans -------------------------------------- +----------------------------- -When an index is successfully created, you must enable the index scan feature as follows: +Reading data using index is disabled by default. +So, exploiting the created index, you need a further step, enabling 'index scan' as following: .. code-block:: sql default> \set INDEX_ENABLED true -If you don't want to use the index scan feature anymore, you can simply disable it as follows: +If you don't want to use index scan anymore, you can simply disable it as follows: .. code-block:: sql @@ -66,4 +68,12 @@ If you don't want to use the index scan feature anymore, you can simply disable .. note:: - Once the index scan feature is enabled, Tajo currently always performs the index scan regardless of its efficiency. You should set this option when the expected number of retrieved tuples is sufficiently small. + Once index scan is enabled, Tajo will perform 'index scan' if possible. In some cases, it may cause performance + degradation. If you always want to get better performance, you should either enable or disable 'index scan' + according to selectivity. Usually, the performance gain of index will increase when the selectivity is low. + +--------------------------- +3. Index backup and restore +--------------------------- + +Tajo currently provides only the catalog backup and restore for index. Please refer to :doc:`/backup_and_restore/catalog` for more information about catalog backup and restore. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/tajo/blob/f44f92fb/tajo-docs/src/main/sphinx/index/types.rst ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/index/types.rst b/tajo-docs/src/main/sphinx/index/types.rst index 457f453..23b378c 100644 --- a/tajo-docs/src/main/sphinx/index/types.rst +++ b/tajo-docs/src/main/sphinx/index/types.rst @@ -2,6 +2,6 @@ Index Types ************************************* -Currently, Tajo supports only one type of index, ``TWO_LEVEL_BIN_TREE``, shortly ``BST``. The BST index is a kind of binary search tree which is extended to be permanently stored on disk. It consists of two levels of nodes; a leaf node indexes the keys with the positions of data in an HDFS block and a root node indexes the keys with the leaf node indices. +Currently, Tajo supports only one type of index, ``TWO_LEVEL_BIN_TREE``, shortly ``BST``. The BST index is a kind of binary search tree which is extended to be permanently stored on disk. It consists of two levels of nodes; a leaf node indexes the keys with the offsets to data stored on HDFS, and a root node indexes the keys with the offsets to the leaf nodes. -When an index scan is started, the query engine first reads the root node and finds the search key. If it finds a leaf node corresponding to the search key, it subsequently finds the search key in that leaf node. Finally, it directly reads a tuple corresponding to the search key from HDFS. \ No newline at end of file +When an index scan is started, the query engine first reads the root node and finds the search key. If it successfully finds a leaf node corresponding to the search key, it subsequently finds the search key in that leaf node. Finally, it directly reads a tuple corresponding to the search key from HDFS. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/tajo/blob/f44f92fb/tajo-docs/src/main/sphinx/index_overview.rst ---------------------------------------------------------------------- diff --git a/tajo-docs/src/main/sphinx/index_overview.rst b/tajo-docs/src/main/sphinx/index_overview.rst index a42931b..73a436e 100644 --- a/tajo-docs/src/main/sphinx/index_overview.rst +++ b/tajo-docs/src/main/sphinx/index_overview.rst @@ -4,12 +4,6 @@ Index (Experimental Feature) An index is a data structure that is used for efficient query processing. Using an index, the Tajo query engine can directly retrieve search values. -This is still an experimental feature. In order to use indexes, you must check out the source code of the ``index_support`` branch:: - - git clone -b index_support https://git-wip-us.apache.org/repos/asf/tajo.git tajo-index - -For the source code build, please refer to :doc:`getting_started`. - The following sections describe the supported index types, the query execution with an index, and the future works. .. toctree::
