-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17726/
-----------------------------------------------------------
(Updated Feb. 8, 2014, 2:17 a.m.)
Review request for Tajo.
Bugs: TAJO-584
https://issues.apache.org/jira/browse/TAJO-584
Repository: tajo
Description
-------
In Tajo, sort operator is similar to merge sort, but it works in a distributed
manner. The first sort phase sorts each fragment in local machine, the
intermediate data are shuffled in range partition, and then the the second sort
phase in each node sorts the range-partitioned data.
However, the second sort phase reads all shuffled data via one scanner. It
causes performance degrade. This patch improves the second sort phase to merge
directly all already-sorted intermediate data. It significantly reduces the
response time of sort queries.
I carried out some simple benchmark with the following query on TPC-H 100GB
data sets:
{code:sql}
select l_orderkey from lineitem order by l_orderkey;
{code}
The lineitem table occupies 75GB. The query response time are dramatically
reduced from 480 to 260 secs. This patch exploits the design of TAJO-36. So,
this patch requires TAJO-36.
Diffs
-----
CHANGES.txt 8dc5ee5f0021acacb75220d1a3ba04b22ebf925b
tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/exception/AlreadyExistsTableException.java
98c04b57e4d72f7ef02f7ec72e4df05f14ac30eb
tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSet.java
3977d76e37888defb27715c249b336bb4cc8a5f5
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/BaseAlgebraVisitor.java
6cc6fd053093480136184fc7d293c59265070f17
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlan.java
6afaaad7b067258ec0cefcca634164b8ae9afa06
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
dfa2e40e46e5658e664c7593fd7e224fe32cbfb4
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PlannerUtil.java
b59cddafadd0c254aaef97c482cacab6ca4742c1
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PreLogicalPlanVerifier.java
1843b5adc298a003a9934b180d0056c655bf44fb
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/RangePartitionAlgorithm.java
fc56b55dd4bfee16a8650de107c07f616ffc4a4b
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/UniformRangePartition.java
4f18c95225935b7104da6866c7ad98f790b965e6
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
2aa93d7f251848b267f98c6a8c1e5a193696e633
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ExternalSortExec.java
a3b37fc6445249c23ec2a99bbac312e4c2d4bbe9
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/UnaryPhysicalExec.java
a8dd877fdd3a230db86aaa13a66d24933db08735
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/utils/TupleUtil.java
42508a0767e9c5848fed705042d7d14fde8747db
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
71ab8f9bbdffb140de3b2601e8982d2c0ca3f09e
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/RangeRetrieverHandler.java
26991a00dd62256d724bfd614c71e68f005020e2
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java
1b72d1398adc097422f2bb8990561ad528ff733c
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/TestUniformRangePartition.java
acd663f18d539f0af24b789b082d323fc7db60b2
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestBSTIndexExec.java
cf240542552b492bdee9c2df69cfdfab1a238c98
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestExternalSortExec.java
d7a3eb109728a968b9bc3f2861e1e5dec6e6e265
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestLeftOuterHashJoinExec.java
4ac423b8ce8c143217523de149e840773aca0040
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestLeftOuterNLJoinExec.java
ce31851333c2929e4141b8d2b3e327a41d14f90b
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestPhysicalPlanner.java
c97ed00782a395e84ce643e764e4f9ef88fbd17b
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestRightOuterHashJoinExec.java
d582e2b10b25a0b067ce640af2f8b39b6e35bbca
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestSortExec.java
4aa1ba2339458d12ea8d297b7397f70340ccbb26
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java
9b940daac43cbb95164803d37e48973ea0a39ab7
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestSortQuery.java
5e5c70cd9233b4aba8f6bcd6321731c41b14bd55
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/util/TestTupleUtil.java
3c4d3c2d8ead6c70b863dd8a848c1396522c6e2d
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/worker/TestRangeRetrieverHandler.java
78e846de731c7fa2739dbb6dcc3bec45d4ba1ab3
tajo-core/tajo-core-backend/src/test/resources/dataset/TestSortQuery/table2.tbl
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/org/apache/tajo/jdbc/TestTajoResultSet.java
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testError.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testOuterJoinAndCaseWhen1.sql
ccb58a58f0303e01a228fc5774e2180d90a20f27
tajo-core/tajo-core-backend/src/test/resources/queries/TestSortQuery/create_table_with_asc_desc_keys.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestSortQuery/testSortWithAscDescKeys.sql
PRE-CREATION
tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/PullServerAuxService.java
1531fadaf9bbc2cb601989618936a400815d592c
tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java
1b0cadcc67c134ffe7e09c7bf7af08e970871542
tajo-storage/src/main/java/org/apache/tajo/storage/MergeScanner.java
e4439f318d080f064848543586e538b1e73b3122
tajo-storage/src/main/java/org/apache/tajo/storage/RawFile.java
73b36926144b0797615b88ad728b889a7849c8b2
tajo-storage/src/main/java/org/apache/tajo/storage/TupleComparator.java
69c1c046775d7c7193864912d54a4fa3d7024cce
tajo-storage/src/main/java/org/apache/tajo/storage/TupleRange.java
7d0f674e9af104b9f13f6f77cac18115b8224d1a
Diff: https://reviews.apache.org/r/17726/diff/
Testing (updated)
-------
mvn clean install
Thanks,
Hyunsik Choi
