[
https://issues.apache.org/jira/browse/TAJO-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jihoon Son updated TAJO-1553:
-----------------------------
Description:
The global engine generates a logical plan, and then marks some parts of the
plan as broadcast plan which means that their input will be broadcasted to all
workers.
Currently, broadcast parts are identified according to some rigid and
hard-coded rules. This will limit the broadcast opportunities in many cases.
So, in this issue, I propose refactoring the broadcast planner to be more
general.
Broadcast parts can be identified recursively.
* A leaf node will be broadcasted if its input size does not exceed the
pre-defined threshold.
* An intermediate node will be broadcasted if it has at least one broadcast
child.
was:
The global engine generates a logical plan, and then marks some parts of the
plan as broadcast plan which means that they will be broadcasted to all
workers.
Currently, broadcast parts are identified according to some rigid and
hard-coded rules. This will limit the broadcast opportunities in many cases.
So, in this issue, I propose refactoring the broadcast planner to be more
general.
Broadcast parts can be identified recursively.
* A leaf node will be broadcasted if its input size does not exceed the
pre-defined threshold.
* An intermediate node will be broadcasted if it has at least one broadcast
child.
> Improve broadcast join planning
> -------------------------------
>
> Key: TAJO-1553
> URL: https://issues.apache.org/jira/browse/TAJO-1553
> Project: Tajo
> Issue Type: Improvement
> Components: distributed query plan, planner/optimizer
> Reporter: Jihoon Son
> Fix For: 0.11.0
>
>
> The global engine generates a logical plan, and then marks some parts of the
> plan as broadcast plan which means that their input will be broadcasted to
> all workers.
> Currently, broadcast parts are identified according to some rigid and
> hard-coded rules. This will limit the broadcast opportunities in many cases.
> So, in this issue, I propose refactoring the broadcast planner to be more
> general.
> Broadcast parts can be identified recursively.
> * A leaf node will be broadcasted if its input size does not exceed the
> pre-defined threshold.
> * An intermediate node will be broadcasted if it has at least one broadcast
> child.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
