[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151963#comment-14151963
]
Xuefu Zhang commented on HIVE-7627:
---
+1
> FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
> -
>
> Key: HIVE-7627
> URL: https://issues.apache.org/jira/browse/HIVE-7627
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Chengxiang Li
>Assignee: Chengxiang Li
> Labels: spark-m1
> Attachments: HIVE-7627.1-spark.patch, HIVE-7627.2-spark.patch,
> HIVE-7627.3-spark.patch, HIVE-7627.4-spark.patch, HIVE-7627.4-spark.patch,
> HIVE-7627.5-spark.patch, HIVE-7627.5-spark.patch
>
>
> Hive table statistic failed on FSStatsPublisher mode, with the following
> exception in Spark executor side:
> {noformat}
> 14/08/05 16:46:24 WARN hdfs.DFSClient: DataStreamer Exception
> java.io.FileNotFoundException: ID mismatch. Request id and saved id: 20277 ,
> 20278 for file
> /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0
> at
> org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
>at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1442)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1261)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): ID
> mismatch. Request id and saved id: 20277 , 20278 for file
> /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0
> at
> org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Serve
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151918#comment-14151918
]
Hive QA commented on HIVE-7627:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12671815/HIVE-7627.5-spark.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6509 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fs_default_name2
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/173/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/173/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-173/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12671815
> FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
> -
>
> Key: HIVE-7627
> URL: https://issues.apache.org/jira/browse/HIVE-7627
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Chengxiang Li
>Assignee: Chengxiang Li
> Labels: spark-m1
> Attachments: HIVE-7627.1-spark.patch, HIVE-7627.2-spark.patch,
> HIVE-7627.3-spark.patch, HIVE-7627.4-spark.patch, HIVE-7627.4-spark.patch,
> HIVE-7627.5-spark.patch, HIVE-7627.5-spark.patch
>
>
> Hive table statistic failed on FSStatsPublisher mode, with the following
> exception in Spark executor side:
> {noformat}
> 14/08/05 16:46:24 WARN hdfs.DFSClient: DataStreamer Exception
> java.io.FileNotFoundException: ID mismatch. Request id and saved id: 20277 ,
> 20278 for file
> /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0
> at
> org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
>at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1442)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1261)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): ID
> mismatch. Request id and saved id: 20277 , 20278 for fil
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151326#comment-14151326
]
Hive QA commented on HIVE-7627:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12671735/HIVE-7627.5-spark.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/170/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/170/console
Test logs:
http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-170/
Messages:
{noformat}
This message was trimmed, see log for full details
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_CASE KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN LPAREN KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as "STAR" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_TRUE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_FALSE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_DATE StringLiteral" using multiple
alternatives: 2, 3
As a result, alternative(s) 3 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY" using multiple alternatives: 2, 9
As a
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151259#comment-14151259
]
Hive QA commented on HIVE-7627:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12671721/HIVE-7627.4-spark.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/169/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/169/console
Test logs:
http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-169/
Messages:
{noformat}
This message was trimmed, see log for full details
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_CASE KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN LPAREN KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as "STAR" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_TRUE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_FALSE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_DATE StringLiteral" using multiple
alternatives: 2, 3
As a result, alternative(s) 3 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warni
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151115#comment-14151115
]
Xuefu Zhang commented on HIVE-7627:
---
The failure may be caused by the snapshot spark library, which may need to be
updated to include the fix for SPARK-2895. Will do that shortly.
> FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
> -
>
> Key: HIVE-7627
> URL: https://issues.apache.org/jira/browse/HIVE-7627
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Chengxiang Li
>Assignee: Chengxiang Li
> Labels: spark-m1
> Attachments: HIVE-7627.1-spark.patch, HIVE-7627.2-spark.patch,
> HIVE-7627.3-spark.patch, HIVE-7627.4-spark.patch
>
>
> Hive table statistic failed on FSStatsPublisher mode, with the following
> exception in Spark executor side:
> {noformat}
> 14/08/05 16:46:24 WARN hdfs.DFSClient: DataStreamer Exception
> java.io.FileNotFoundException: ID mismatch. Request id and saved id: 20277 ,
> 20278 for file
> /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0
> at
> org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
>at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1442)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1261)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): ID
> mismatch. Request id and saved id: 20277 , 20278 for file
> /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0
> at
> org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufR
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151065#comment-14151065
]
Hive QA commented on HIVE-7627:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12671699/HIVE-7627.4-spark.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/168/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/168/console
Test logs:
http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-168/
Messages:
{noformat}
This message was trimmed, see log for full details
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_CASE KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN LPAREN KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as "STAR" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_TRUE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_FALSE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_DATE StringLiteral" using multiple
alternatives: 2, 3
As a result, alternative(s) 3 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warni
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14151020#comment-14151020
]
Hive QA commented on HIVE-7627:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12671696/HIVE-7627.3-spark.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/167/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/167/console
Test logs:
http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-167/
Messages:
{noformat}
This message was trimmed, see log for full details
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN KW_CASE KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as "LPAREN LPAREN KW_IF" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as "KW_CLUSTER KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as "KW_PARTITION KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as "KW_DISTRIBUTE KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as "KW_SORT KW_BY LPAREN" using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as "STAR" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_STRUCT" using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_ARRAY" using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as "KW_UNIONTYPE" using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_NULL" using multiple alternatives: 1, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_TRUE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_FALSE" using multiple alternatives: 3, 8
As a result, alternative(s) 8 were disabled for that input
warning(200): IdentifiersParser.g:261:5:
Decision can match input such as "KW_DATE StringLiteral" using multiple
alternatives: 2, 3
As a result, alternative(s) 3 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY"
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY" using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "KW_BETWEEN KW_MAP LPAREN" using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:393:5:
Decision can match input such as "{KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY" using multiple alternatives: 2, 9
As a
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128338#comment-14128338
]
Hive QA commented on HIVE-7627:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12667637/HIVE-7627.2-spark.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 6343 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample_islocalmode_hook
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_fs_default_name2
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_optimize_nullscan
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/123/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/123/console
Test logs:
http://ec2-54-176-176-199.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-123/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12667637
> FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
> -
>
> Key: HIVE-7627
> URL: https://issues.apache.org/jira/browse/HIVE-7627
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Reporter: Chengxiang Li
>Assignee: Chengxiang Li
> Labels: spark-m1
> Attachments: HIVE-7627.1-spark.patch, HIVE-7627.2-spark.patch
>
>
> Hive table statistic failed on FSStatsPublisher mode, with the following
> exception in Spark executor side:
> {noformat}
> 14/08/05 16:46:24 WARN hdfs.DFSClient: DataStreamer Exception
> java.io.FileNotFoundException: ID mismatch. Request id and saved id: 20277 ,
> 20278 for file
> /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0
> at
> org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
>at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1442)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1261)
> at
> org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): ID
> mismatch. Request id and saved id: 20277
[jira] [Commented] (HIVE-7627) FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-7627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14088744#comment-14088744 ] Chengxiang Li commented on HIVE-7627: - Root cause: FSStatsPublisher collect table data info through TableScanOperator in each task, and write collect data into a file each, the file name for each task is set by prefix + conf.getInt(“mapred.task.partition”). During Hive on Spark, "mapred.task.partition" is not set, so it's all the same for all tasks, which means all tasks write to the same file. In HDFS, multi-thread write to same file with overwrite is true would lead to the previous exception. Solution: We need to set "mapred.task.partition" in Spark transformation closure, so current problem is that Spark transformation closure can't get task context information through Spark Java API, it's only available in Spark Scala API, I've created SPARK-2895 to track this. > FSStatsPublisher does fit into Spark multi-thread task mode[Spark Branch] > - > > Key: HIVE-7627 > URL: https://issues.apache.org/jira/browse/HIVE-7627 > Project: Hive > Issue Type: Bug > Components: Spark >Reporter: Chengxiang Li >Assignee: Chengxiang Li > > Hive table statistic failed on FSStatsPublisher mode, with the following > exception in Spark executor side: > 14/08/05 16:46:24 WARN hdfs.DFSClient: DataStreamer Exception > java.io.FileNotFoundException: ID mismatch. Request id and saved id: 20277 , > 20278 for file > /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0 > at > org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:525) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106) >at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1442) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1261) > at > org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:525) > Caused by: > org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): ID > mismatch. Request id and saved id: 20277 , 20278 for file > /tmp/hive-root/8833d172-1edd-4508-86db-fdd7a1b0af17/hive_2014-08-05_16-46-03_013_6279446857294757772-1/-ext-1/tmpstats-0 > at > org.apache.hadoop.hdfs.server.namenode.INodeId.checkId(INodeId.java:53) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2952) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNod
