[
https://issues.apache.org/jira/browse/TAJO-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14259929#comment-14259929
]
ASF GitHub Bot commented on TAJO-269:
-------------------------------------
GitHub user hyunsik opened a pull request:
https://github.com/apache/tajo/pull/322
TAJO-269: Protocol buffer De/Serialization for LogicalNode.
It was a very long time work. This patch completely replaces json
(de)serialization of logical plan by protocol buffer. This is the first patch.
I'll do some cleanup and add more comments.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hyunsik/tajo TAJO-269
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/322.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #322
----
commit 2ec6e684d3f7df7cfda195a1355d9c1e45eedc4b
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-10T15:59:47Z
initial work for JitVecTestBase.
commit 7df2c6192a3158fc225b54b424f50f0272ceef5c
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-11T18:55:34Z
Added more utility methods to Eval, and Added basic layout of
LogicalPlanConvertor.
commit 0e910132b3d3349e8af18632f6759fbbb5a04429
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-12T16:55:44Z
Added (de)serializer for const eval.
commit 1bde69303e69582b81ecaeec7f08aa0551abe8fb
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-12T17:20:15Z
Added (de)serializer for field eval.
commit a5e3317f88f87a1182a2a1dbc9d6a805bc47b5a4
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-12T18:12:27Z
Added (de)serializer for function.
commit ecbdb340954940b01ab48a602191c5d579bc7a84
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-13T22:33:22Z
Added (de)serializer for in clause and rowconstant.
commit 95cd8ffa358da732d81c927a0dedb20742376d63
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-15T03:50:03Z
Fixed (de)serialization bug for function, and implemented IntervalDatum.
commit c58f469aa8aa31df1109374ceb9c158d9a77213c
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-15T11:25:14Z
Added (de)serialization for between.
commit 621967232d90dd0768fcfbd9e78aa64497fb0478
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-15T12:41:30Z
Added (de)serialization for case when.
commit 654b8b946b64b4572de41d59f3ad585e4b32f0b3
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-15T13:01:08Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into
JitVecTestBase
Conflicts:
tajo-core/src/main/java/org/apache/tajo/engine/eval/BinaryEval.java
tajo-core/src/main/java/org/apache/tajo/engine/eval/FunctionEval.java
commit 0b257fd924a747075c68200a676e29e352b15c99
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-15T13:03:32Z
Renamed setExpr to setChild.
commit f5e994df1f64980ae48a2d9cb5e13c68dc0fe611
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-15T13:21:43Z
Refactored eval tree proto.
commit ade69f784e2369aa2f6950ce4f95dd316f4526e3
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-15T13:51:40Z
TAJO-1008: Protocol buffer De/Serialization for EvalNode.
commit c9929b946e2aae762165c5583d475d06f039238f
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-16T20:53:35Z
Merge branch 'TAJO-1008' of github.com:hyunsik/tajo into JitVecTestBase
Conflicts:
tajo-core/src/main/java/org/apache/tajo/engine/eval/BetweenPredicateEval.java
tajo-core/src/main/java/org/apache/tajo/engine/eval/BinaryEval.java
tajo-core/src/main/java/org/apache/tajo/engine/plan/EvalTreeProtoDeserializer.java
tajo-core/src/main/java/org/apache/tajo/engine/plan/EvalTreeProtoSerializer.java
tajo-core/src/test/java/org/apache/tajo/engine/eval/ExprTestBase.java
tajo-core/src/test/java/org/apache/tajo/engine/eval/TestEvalTree.java
commit 56df752d142d0cf0d2bf1d968ba1f99fc2a89428
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-17T06:54:21Z
Refactor some methods and variables.
commit b5a1ed853e8342c89872b9a312756264ec1f4630
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-17T06:56:34Z
Refactor ScanNode and added (de)serializeion for ScanNode.
commit 059c10be3efa7c4d91d37fecaa5ba9d4ea0897cf
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-17T06:57:11Z
Merge branch 'TAJO-1008' of github.com:hyunsik/tajo into JitVecTestBase
commit 5d30cef87977e5d7582857525e37512881934a56
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-17T12:39:40Z
Add (de)serialization scan, filter, groupby, sort, and limit nodes.
commit 5a69b8512eefd5f73f02360976a7cfb649af7a3e
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-17T17:10:03Z
Add (de)serialization root, having, and exprs.
commit eb50db0b6e73dde90945c43273e59cc1fe472bab
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-17T17:10:51Z
Fixed compilation error.
commit b1eb0e9482bdd5e41a698f96738e1fe5af0b1a61
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-18T02:10:18Z
Add (de)serialization for other nodes.
commit 3fd70b59d769b1263025708a839793df1492db7a
Author: Hyunsik Choi <[email protected]>
Date: 2014-08-18T09:46:32Z
Updated NodeType, and added more (de)serialization code.
commit 800e8068f2768a2ec51d77ac0a7923e259af7c94
Author: Hyunsik Choi <[email protected]>
Date: 2014-09-16T10:26:04Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into
JitVecTestBase
Conflicts:
pom.xml
tajo-core/src/main/java/org/apache/tajo/engine/eval/EvalType.java
tajo-core/src/main/java/org/apache/tajo/engine/eval/FunctionEval.java
tajo-core/src/main/java/org/apache/tajo/engine/plan/EvalTreeProtoSerializer.java
tajo-core/src/main/java/org/apache/tajo/engine/planner/logical/ScanNode.java
tajo-core/src/main/proto/Plan.proto
tajo-core/src/test/java/org/apache/tajo/engine/eval/ExprTestBase.java
tajo-project/pom.xml
commit 8c55224f668b12287b434fb83467f67e79c77c7c
Author: Hyunsik Choi <[email protected]>
Date: 2014-12-24T06:34:17Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into
JitVecTestBase
Conflicts:
tajo-plan/src/main/java/org/apache/tajo/plan/Target.java
tajo-plan/src/main/java/org/apache/tajo/plan/expr/CaseWhenEval.java
tajo-plan/src/main/java/org/apache/tajo/plan/expr/EvalNode.java
tajo-plan/src/main/java/org/apache/tajo/plan/logical/EvalExprNode.java
tajo-plan/src/main/java/org/apache/tajo/plan/visitor/BasicLogicalPlanVisitor.java
tajo-plan/src/main/java/org/apache/tajo/plan/visitor/LogicalPlanVisitor.java
tajo-plan/src/main/proto/Plan.proto
commit 8a69a46cfc2825704f6bd689e80aac25a543d99c
Author: Hyunsik Choi <[email protected]>
Date: 2014-12-25T20:30:29Z
All unit tests are passed.
commit e55e168161628812a37b2ed82a9c71d9f34086a7
Author: Hyunsik Choi <[email protected]>
Date: 2014-12-26T08:06:47Z
Fix test failures.
commit 5ac3f67cd89922e9263d13b092a4a2e5bf1ca199
Author: Hyunsik Choi <[email protected]>
Date: 2014-12-27T09:12:53Z
Fixed manu test failures.
commit ef71d30f92ea83e9311d2e3d289733ab1cd5f880
Author: Hyunsik Choi <[email protected]>
Date: 2014-12-27T13:45:01Z
Fix test failures.
commit 8fe6b7e042bcd80d4332b2c12e2fbe3127cbdb6a
Author: Hyunsik Choi <[email protected]>
Date: 2014-12-27T17:11:21Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into
JitVecTestBase
commit dbfab9721dd7225a7f855617c9ac0c1660eec67c
Author: Hyunsik Choi <[email protected]>
Date: 2014-12-28T13:06:16Z
Injected test code.
----
> Protocol buffer De/Serialization for LogicalNode
> ------------------------------------------------
>
> Key: TAJO-269
> URL: https://issues.apache.org/jira/browse/TAJO-269
> Project: Tajo
> Issue Type: Task
> Components: query master, worker
> Reporter: Jihoon Son
> Assignee: Hyunsik Choi
> Fix For: 0.10
>
>
> In the current implementation, the logical plan is serialized into a JSON
> object and sent to each worker.
> However, the transmission of JSON object incurs the high overhead due to its
> large size.
> ProtocolBuffer is a good alternative because its overhead is quite small and
> already used in other modules of Tajo.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
