[
https://issues.apache.org/jira/browse/TAJO-1766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710810#comment-14710810
]
ASF GitHub Bot commented on TAJO-1766:
--------------------------------------
Github user jinossy commented on the pull request:
https://github.com/apache/tajo/pull/706#issuecomment-134512935
Awesome!!
I've test your query on Tez and same cluster. but I've stop the query after
5h
```
hive> select count(*) from partsupp, supplier_small;
Warning: Shuffle Join MERGEJOIN[11][tables = [partsupp, supplier_small]] in
Stage 'Reducer 2' is a cross product
Query ID = hadoop_20150825100744_8e11a2d9-d918-473f-bad2-36e403d6eca3
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1440464539141_0001)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED
KILLED
--------------------------------------------------------------------------------
Exiting the JVM. SUCCEEDED 224 224 0 0 0
0
Interrupting... Be patient, this might take some time.0 0 0
0
Map 1 .......... SUCCEEDED 224 224 0 0 0
0
Map 4 .......... SUCCEEDED 1 1 0 0 0
0
Reducer 2 RUNNING 1 0 1 0 0
0
Reducer 3 INITED 1 0 0 1 0
0
--------------------------------------------------------------------------------
VERTICES: 02/04 [=========================>>-] 99% ELAPSED TIME:
22510.28 s
```
> Improve the performance of cross join
> -------------------------------------
>
> Key: TAJO-1766
> URL: https://issues.apache.org/jira/browse/TAJO-1766
> Project: Tajo
> Issue Type: Improvement
> Components: distributed query plan
> Reporter: Jihoon Son
> Assignee: Jihoon Son
> Fix For: 0.11.0
>
>
> Cross join is one of the very heavy operations. Furthermore, this operator is
> performed by a single worker in the current implementation. (Please see the
> implementation of HashPartitioner. If partitionKeyIds is empty,
> getPartition() always returns a single value.)
> One possible alternative is executing cross join with broadcast join. That
> is, outer table (smaller one) is always broadcasted, and join is performed by
> the machine who stores a part of inner table.
> To do so, a new session variable is required to set the broadcast threshold
> for cross join.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
