[
https://issues.apache.org/jira/browse/HIVE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HIVE-2820:
------------------------------
Attachment: HIVE-2820.D1935.3.patch
navis updated the revision "HIVE-2820 [jira] Invalid tag is used for
MapJoinProcessor".
Reviewers: JIRA
1. minimized modifications
2. added one more test case
3. rebase on trunk and passed all tests
REVISION DETAIL
https://reviews.facebook.net/D1935
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/exec/AbstractMapJoinOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java
ql/src/java/org/apache/hadoop/hive/ql/exec/JoinUtil.java
ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java
ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/LocalMapJoinProcFactory.java
ql/src/test/queries/clientpositive/join_reorder2.q
ql/src/test/results/clientpositive/join_reorder2.q.out
> Invalid tag is used for MapJoinProcessor
> ----------------------------------------
>
> Key: HIVE-2820
> URL: https://issues.apache.org/jira/browse/HIVE-2820
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.9.0
> Environment: ubuntu
> Reporter: Navis
> Assignee: Navis
> Fix For: 0.9.0
>
> Attachments: HIVE-2820.D1935.1.patch, HIVE-2820.D1935.2.patch,
> HIVE-2820.D1935.3.patch
>
>
> Testing HIVE-2810, I've found tag and alias are used in very confusing
> manner. For example, query below fails..
> {code}
> hive> set hive.auto.convert.join=true;
>
> hive> select /*+ STREAMTABLE(a) */ * from myinput1 a join myinput1 b on
> a.key=b.key join myinput1 c on a.key=c.key;
> Total MapReduce jobs = 4
> Ended Job = 1667415037, job is filtered out (removed at runtime).
> Ended Job = 1739566906, job is filtered out (removed at runtime).
> Ended Job = 1113337780, job is filtered out (removed at runtime).
> 12/02/24 10:27:14 WARN conf.HiveConf: DEPRECATED: Ignoring hive-default.xml
> found on the CLASSPATH at /home/navis/hive/conf/hive-default.xml
> Execution log at:
> /tmp/navis/navis_20120224102727_cafe0d8d-9b21-441d-bd4e-b83303b31cdc.log
> 2012-02-24 10:27:14 Starting to launch local task to process map join;
> maximum memory = 932118528
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.processOp(HashTableSinkOperator.java:312)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.startForward(MapredLocalTask.java:325)
> at
> org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:272)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:685)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Execution failed with exit status: 2
> Obtaining error information
> {code}
> Failed task has a plan which doesn't make sense.
> {noformat}
> Stage: Stage-8
> Map Reduce Local Work
> Alias -> Map Local Tables:
> b
> Fetch Operator
> limit: -1
> c
> Fetch Operator
> limit: -1
> Alias -> Map Local Operator Tree:
> b
> TableScan
> alias: b
> HashTable Sink Operator
> condition expressions:
> 0 {key} {value}
> 1 {key} {value}
> 2 {key} {value}
> handleSkewJoin: false
> keys:
> 0 [Column[key]]
> 1 [Column[key]]
> 2 [Column[key]]
> Position of Big Table: 0
> c
> TableScan
> alias: c
> Map Join Operator
> condition map:
> Inner Join 0 to 1
> Inner Join 0 to 2
> condition expressions:
> 0 {key} {value}
> 1 {key} {value}
> 2 {key} {value}
> handleSkewJoin: false
> keys:
> 0 [Column[key]]
> 1 [Column[key]]
> 2 [Column[key]]
> outputColumnNames: _col0, _col1, _col4, _col5, _col8, _col9
> Position of Big Table: 0
> Select Operator
> expressions:
> expr: _col0
> type: int
> expr: _col1
> type: int
> expr: _col4
> type: int
> expr: _col5
> type: int
> expr: _col8
> type: int
> expr: _col9
> type: int
> outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
> File Output Operator
> compressed: false
> GlobalTableId: 0
> table:
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> Stage: Stage-4
> Map Reduce
> Alias -> Map Operator Tree:
> a
> TableScan
> alias: a
> HashTable Sink Operator
> condition expressions:
> 0 {key} {value}
> 1 {key} {value}
> 2 {key} {value}
> handleSkewJoin: false
> keys:
> 0 [Column[key]]
> 1 [Column[key]]
> 2 [Column[key]]
> Position of Big Table: 0
> Local Work:
> Map Reduce Local Work
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
