[
https://issues.apache.org/jira/browse/HIVE-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677499#action_12677499
]
Zheng Shao commented on HIVE-308:
---------------------------------
The problem of the first query (map-only job) is that we have 2 file sink
operators.
See the log:
2009-02-27 11:55:42,528 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2009-02-27 11:55:42,595 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 0
2009-02-27 11:55:42,612 INFO org.apache.hadoop.mapred.MapTask: split:
hdfs://xxxx:9000/warehouse/zshao_lazy/8413_m_000000_0.gz, range: 0-28
2009-02-27 11:55:42,631 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded
the native-hadoop library
2009-02-27 11:55:42,632 INFO org.apache.hadoop.io.compress.zlib.ZlibFactory:
Successfully loaded & initialized native-zlib library
2009-02-27 11:55:42,789 INFO org.apache.hadoop.hive.ql.exec.MapOperator:
Initializing Self
2009-02-27 11:55:42,793 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Adding
alias null-subquery2:a-subquery2:zshao_lazy to work list for file
/warehouse/zshao_lazy/8413_m_000000_0.gz
2009-02-27 11:55:42,793 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Adding
alias null-subquery1:a-subquery1:zshao_lazy to work list for file
/warehouse/zshao_lazy/8413_m_000000_0.gz
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.MapOperator: Got
partitions: null
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator:
Initializing Self
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator:
Initializing children:
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing Self
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing children:
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing Self
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing children:
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.ForwardOperator:
Initializing Self
2009-02-27 11:55:42,802 INFO org.apache.hadoop.hive.ql.exec.ForwardOperator:
Initializing children:
2009-02-27 11:55:42,803 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing Self
2009-02-27 11:55:42,803 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing children:
2009-02-27 11:55:42,803 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator:
Initializing Self
2009-02-27 11:55:42,804 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator:
Writing to temp file:
/tmp/hive-zshao/_tmp.92566742.10001.insclause-0/_tmp.10549_m_000000_0
2009-02-27 11:55:42,813 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initialization Done
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.ForwardOperator:
Initialization Done
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initialization Done
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initialization Done
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator:
Initialization Done
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator:
Initializing Self
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.TableScanOperator:
Initializing children:
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing Self
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing children:
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing Self
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing children:
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.ForwardOperator:
Initializing Self
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.ForwardOperator:
Initializing children:
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing Self
2009-02-27 11:55:42,817 INFO org.apache.hadoop.hive.ql.exec.SelectOperator:
Initializing children:
2009-02-27 11:55:42,818 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator:
Initializing Self
2009-02-27 11:55:42,819 INFO org.apache.hadoop.hive.ql.exec.FileSinkOperator:
Writing to temp file:
/tmp/hive-zshao/_tmp.92566742.10001.insclause-0/_tmp.10549_m_000000_0
> UNION ALL should create different destination directories for different
> operands
> --------------------------------------------------------------------------------
>
> Key: HIVE-308
> URL: https://issues.apache.org/jira/browse/HIVE-308
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.2.0, 0.3.0
> Reporter: Zheng Shao
> Priority: Blocker
>
> The following query hangs:
> {code}
> select * from (select 1 from zshao_lazy union all select 2 from zshao_lazy) a;
> {code}
> The following query produce wrong results: (one map-reduce job
> overwrite/cannot overwrite the result of the other)
> {code}
> select * from (select 1 as id from zshao_lazy cluster by id union all select
> 2 as id from zshao_meta) a;
> {code}
> The reason of both is that the destination directory of the file sink
> operator conflicts with each other.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
