[
https://issues.apache.org/jira/browse/HIVE-5888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921063#comment-13921063
]
Jian Fang commented on HIVE-5888:
---------------------------------
Also, the two reducers often failed because no space left on device. This could
be a big problem.
java.lang.RuntimeException: org.apache.hadoop.fs.FSError: java.io.IOException:
No space left on device
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:278)
at
org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:528)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:429)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1140)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.hadoop.fs.FSError: java.io.IOException: No space left on
device
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:200)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:57)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.writeChunk(ChecksumFileSystem.java:354)
at
org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:150)
at
org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:132)
at
org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:121)
at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:112)
at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:86)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:57)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1115)
at
org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:1076)
at
org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat$1.write(HiveSequenceFileOutputFormat.java:77)
at
org.apache.hadoop.hive.ql.exec.persistence.RowContainer.spillBlock(RowContainer.java:334)
at
org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:164)
at
org.apache.hadoop.hive.ql.exec.persistence.RowContainer.add(RowContainer.java:74)
at
org.apache.hadoop.hive.ql.exec.JoinOperator.processOp(JoinOperator.java:127)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
at
org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:257)
... 7 more
Caused by: java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:345)
at
org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.write(RawLocalFileSystem.java:198)
... 26 more
> group by after join operation product no result when hive.optimize.skewjoin
> = true
> ------------------------------------------------------------------------------------
>
> Key: HIVE-5888
> URL: https://issues.apache.org/jira/browse/HIVE-5888
> Project: Hive
> Issue Type: Bug
> Affects Versions: 0.12.0
> Reporter: cyril liao
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
