[
https://issues.apache.org/jira/browse/HIVE-13743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281237#comment-15281237
]
Rajesh Balamohan commented on HIVE-13743:
-----------------------------------------
hive - 2.1.0-snapshot
tez - 0.8.4
hadoop - 2.8.0-snapshot (built on may-10)
{noformat}
hive> create table test(id int) stored as orc;
OK
Time taken: 0.091 seconds
hive> insert into test values(10);
Query ID = rajesh_20160512060245_a6f56633-fd78-4c8b-81a5-863f80a161e2
Total jobs = 1
Launching Job 1 out of 1
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 1 1 0 0
0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 0.08 s
----------------------------------------------------------------------------------------------
Status: DAG finished successfully in 0.08 seconds
...
Loading data to table rajesh.test
Table rajesh.test stats: [numFiles=1, numRows=1, totalSize=182, rawDataSize=4]
OK
Time taken: 0.839 seconds
...
...
..
hive> create table test_1 as select * from test;
Query ID = rajesh_20160512060301_9233d9d1-e964-4e94-935a-ac778392d031
Total jobs = 1
Launching Job 1 out of 1
----------------------------------------------------------------------------------------------
VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING
FAILED KILLED
----------------------------------------------------------------------------------------------
Map 1 .......... container SUCCEEDED 1 1 0 0
0 0
----------------------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 0.05 s
----------------------------------------------------------------------------------------------
Status: DAG finished successfully in 0.05 seconds
Moving data to directory hdfs://xyz:8020/apps/hive/warehouse/rajesh.db/test_1
Failed with exception Unable to move source
hdfs://xyz:8020/apps/hive/warehouse/rajesh.db/.hive-staging_hive_2016-05-12_06-03-01_281_5180494512279326104-1/-ext-10002
to destination hdfs://xyz:8020/apps/hive/warehouse/rajesh.db/test_1
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
hdfs://xyz:8020/apps/hive/warehouse/rajesh.db/.hive-staging_hive_2016-05-12_06-03-01_281_5180494512279326104-1/-ext-10002
to destination hdfs://xyz:8020/apps/hive/warehouse/rajesh.db/test_1
{noformat}
Notice "create table test_1 as select * from test;" fails in the move task.
> Data move codepath is broken with hive (2.1.0-SNAPSHOT)
> -------------------------------------------------------
>
> Key: HIVE-13743
> URL: https://issues.apache.org/jira/browse/HIVE-13743
> Project: Hive
> Issue Type: Bug
> Reporter: Rajesh Balamohan
>
> Data move codepath is broken with hive 2.1.0-SNAPSHOT with hadoop
> 2.8.0-snapshot.
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path
> not found: /apps/hive/warehouse/tpcds_bin_partitioned_orc_10000.db/date_dim1
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirEncryptionZoneOp.getEZForPath(FSDirEncryptionZoneOp.java:178)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getEZForPath(FSNamesystem.java:7336)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getEZForPath(NameNodeRpcServer.java:1973)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getEZForPath(ClientNamenodeProtocolServerSideTranslatorPB.java:1376)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:645)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2339)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2335)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1711)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2333)
> at org.apache.hadoop.ipc.Client.call(Client.java:1448)
> at org.apache.hadoop.ipc.Client.call(Client.java:1385)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
> at com.sun.proxy.$Proxy30.getEZForPath(Unknown
> Source)/apps/hive/warehouse/tpcds_bin_partitioned_orc_200.db/
> ...
> ...
> ...
> 2016-05-11T09:40:43,760 ERROR [main]: ql.Driver (:()) - FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Unable to
> move source
> hdfs://xyz:8020/apps/hive/warehouse/tpcds_bin_partitioned_orc_10000.db/.hive-staging_hive_2016-05-11_09-40-42_489_5056654133706433454-1/-ext-10002
> to destination
> hdfs://xyz:8020/apps/hive/warehouse/tpcds_bin_partitioned_orc_10000.db/date_dim1
> {noformat}
> https://github.com/apache/hive/blob/26b5c7b56a4f28ce3eabc0207566cce46b29b558/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2836
> hdfsEncryptionShim.isPathEncrypted(destf) in Hive could end up throwing
> FileNotFoundException as the destf is not present yet. This causes moveFile
> to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
