[
https://issues.apache.org/jira/browse/TAJO-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768246#comment-13768246
]
Hudson commented on TAJO-184:
-----------------------------
SUCCESS: Integrated in Tajo-trunk-postcommit #434 (See
[https://builds.apache.org/job/Tajo-trunk-postcommit/434/])
TAJO-184: Refactor GlobalPlanner and global plan data structure. (hyunsik)
(hyunsik:
https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=1b1d1e8c1a6b82ccc5c3ce4daeb9e3daa309cde4)
*
tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/Column.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/EvalExprNode.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/global/TestGlobalQueryOptimizer.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlanVisitor.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestMergeJoinExec.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalOptimizer.java
*
tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogUtil.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/rewrite/ProjectionPushDownRule.java
* tajo-core/tajo-core-backend/src/test/queries/complex_union_2.sql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/IndexScanNode.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/TestPlannerUtil.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/IntersectNode.java
* tajo-core/tajo-core-backend/src/main/resources/webapps/worker/querydetail.jsp
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/master/TestExecutionBlockCursor.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlan.java
* tajo-core/tajo-core-backend/src/main/proto/TajoWorkerProtocol.proto
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/graph/DirectedGraph.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestNLJoinExec.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/ProjectionNode.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/LocalTajoTestingUtility.java
* CHANGES.txt
*
tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/index/TestSingleCSVFileBSTIndex.java
* tajo-core/tajo-core-backend/src/test/queries/complex_union_1.sql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerManagerService.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/UnaryNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/ExceptNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/BaseAlgebraVisitor.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/master/TestGlobalPlanner.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/TestGenericDirectedGraph.java
* tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/LogicalRootNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlanner.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/RelationNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/ipc/protocolrecords/QueryUnitRequest.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/worker/TestRangeRetrieverHandler.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/eval/EvalTreeUtil.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalPlanner.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/TestQueryIdFactory.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
* tajo-core/tajo-core-backend/benchmark/tpch/q4.tql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/graph/DirectedGraphVisitor.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/plan/global/TestGlobalQueryPlanner.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/QueryBlockGraph.java
*
tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/TestTupleComparator.java
*
tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/statistics/StatisticsUtil.java
* tajo-core/tajo-core-backend/src/test/resources/catalog-default.xml
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoAsyncDispatcher.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/SubQuery.java
* tajo-core/tajo-core-backend/benchmark/tpch/q2.tql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/CreateTableNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/JoinNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
* tajo-core/tajo-core-backend/benchmark/tpch/q7.tql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Query.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/LogicalNode.java
* tajo-core/tajo-core-backend/benchmark/tpch/q13.tql
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/worker/dataserver/TestHttpDataServer.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalExecutorVisitor.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/parser/TestSQLAnalyzer.java
*
tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestColumn.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanningException.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/LimitNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/SelectionNode.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/master/TestRepartitioner.java
* tajo-core/tajo-core-backend/benchmark/tpch/q8.tql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/eval/FieldEval.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestExternalSortExec.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/event/QueryStartEvent.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestPhysicalPlanner.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/UnionNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/ReceiveNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/MasterPlan.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/SortNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/ExecutionBlockCursor.java
* tajo-core/tajo-core-backend/src/test/queries/select_13.hiveql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/graph/SimpleDirectedGraph.java
*
tajo-catalog/tajo-catalog-common/src/test/java/org/apache/tajo/catalog/TestSchema.java
* tajo-core/tajo-core-backend/benchmark/tpch/q12.tql
* tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/TestTajoIds.java
* tajo-core/tajo-core-backend/benchmark/tpch/q9.tql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlan.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/Target.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/query/ResultSetMetaDataImpl.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/InsertNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/StoreTableNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/ScanNode.java
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/TaskAttemptContext.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestSortExec.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/GroupbyNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/join/JoinTree.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestSelectQuery.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/NodeType.java
*
tajo-core/tajo-core-storage/src/test/java/org/apache/tajo/storage/index/TestBSTIndex.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterTask.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestResultSetImpl.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/QueryContext.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/rewrite/FilterPushDownRule.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/global/TestMasterPlan.java
* tajo-common/src/main/java/org/apache/tajo/util/TUtil.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestTableSubQuery.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestBNLJoinExec.java
* tajo-core/tajo-core-backend/benchmark/tpch/q14.tql
* tajo-common/src/main/java/org/apache/tajo/QueryIdFactory.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/query/QueryUnitRequestImpl.java
*
tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/Schema.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashAntiJoinExec.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestGroupByQuery.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/ExplainLogicalPlanVisitor.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/DropTableNode.java
* tajo-core/tajo-core-backend/benchmark/tpch/q17.tql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PlannerUtil.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/TableSubQueryNode.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/BasicPhysicalExecutorVisitor.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/BinaryNode.java
* tajo-core/tajo-core-backend/src/test/queries/table_subquery2.sql
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/BasicLogicalPlanVisitor.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/Projector.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashJoinExec.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/ExecutionBlock.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/AlgebraVisitor.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/SendNode.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestBSTIndexExec.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TaskSchedulerImpl.java
* tajo-common/src/main/java/org/apache/tajo/ExecutionBlockId.java
*
tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/CatalogServer.java
*
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestHashSemiJoinExec.java
* tajo-core/tajo-core-backend/src/test/queries/set_4.sql
* tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/DataChannel.java
*
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PlanString.java
> Refactor GlobalPlanner and global plan data structure
> -----------------------------------------------------
>
> Key: TAJO-184
> URL: https://issues.apache.org/jira/browse/TAJO-184
> Project: Tajo
> Issue Type: Improvement
> Components: master, physical operator, planner/optimizer
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2-incubating
>
> Attachments: TAJO-184_2.patch, TAJO-184.patch
>
>
> Above all, I'm sorry for submitting a big patch. This patch modifies and
> refactors broadly global planning, logical planning, and physical planning
> parts. It was hard to separate this issue into smaller issues.
> Especially, this patch primarily rewrites GlobalPlanner and MasterPlan
> (global plan data structure) as follows:
> * Removed GlobalPlanOptimizer
> * Added DirectedGraph interface, SimpleDirectedGraph concret class, and a
> visitor class to visit a graph in post-order traverse way.
> * Improved MasterPlan by using new graph API
> ** query block graphs and an execution block graph are represented by
> SimpleDirectedGraph.
> ** Now, we can traverse above graphs easily by using graph APIs.
> ** Added DataChannel class to represent a data flow between execution
> blocks.
> * MasterPlan.toString() prints a text graph to represent relationships among
> execution blocks and a distributed plan.
> * Add more sophisticated explain feature for a distributed plan and logical
> plan. It is very useful for plan debugging.
> * Now, the limit operator is pushed down to child execution block.
> ** So, the intermediate data volume of a sort query with limit is reduced
> significantly.
> * TableSubQuery (inline view) is supported. It follows SQL standards. So,
> you can do a query as follows:
> {code}
> SELECT *
> FROM
> (
> SELECT
> l_orderkey,
> l_partkey,
> url
> FROM
> (
> SELECT
> l_orderkey,
> l_partkey,
> CASE
> WHEN
> l_partkey IS NOT NULL THEN ''
> WHEN l_orderkey = 1 THEN '1'
> ELSE
> '2'
> END AS url
> FROM
> lineitem
> ) res1
> JOIN
> (
> SELECT
> *
> FROM
> part
> ) res2
> ON l_partkey = p_partkey
> ) result
> {code}
> In addition, I've refactored as follows:
> * Column has a qualifier name.
> * Improved Schema to deal with qualified column names
> * When a TableDesc instance is retrieved, it is forced to have qualifier
> columns.
> * Fixed TAJO-162 bug.
> * Lots of trivial improvement and refactors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
