[
https://issues.apache.org/jira/browse/HIVE-13743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283589#comment-15283589
]
Sergio Peña commented on HIVE-13743:
------------------------------------
[~thejas] is right. The destination path that does not exist might be encrypted
(when created) by its parent directory if the parent is in an encryption zone.
May we try to check if the destination parent directory in case the destination
path does not exist?
> Data move codepath is broken with hive (2.1.0-SNAPSHOT)
> -------------------------------------------------------
>
> Key: HIVE-13743
> URL: https://issues.apache.org/jira/browse/HIVE-13743
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 2.1.0
> Reporter: Rajesh Balamohan
> Assignee: Ashutosh Chauhan
> Fix For: 2.1.0
>
> Attachments: HIVE-13743.2.patch, HIVE-13743.patch
>
>
> Data move codepath is broken with hive 2.1.0-SNAPSHOT with hadoop
> 2.8.0-snapshot.
> {noformat}
> Caused by:
> org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path
> not found: /apps/hive/warehouse/tpcds_bin_partitioned_orc_10000.db/date_dim1
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirEncryptionZoneOp.getEZForPath(FSDirEncryptionZoneOp.java:178)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getEZForPath(FSNamesystem.java:7336)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getEZForPath(NameNodeRpcServer.java:1973)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getEZForPath(ClientNamenodeProtocolServerSideTranslatorPB.java:1376)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:645)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2339)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2335)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1711)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2333)
> at org.apache.hadoop.ipc.Client.call(Client.java:1448)
> at org.apache.hadoop.ipc.Client.call(Client.java:1385)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
> at com.sun.proxy.$Proxy30.getEZForPath(Unknown
> Source)/apps/hive/warehouse/tpcds_bin_partitioned_orc_200.db/
> ...
> ...
> ...
> 2016-05-11T09:40:43,760 ERROR [main]: ql.Driver (:()) - FAILED: Execution
> Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Unable to
> move source
> hdfs://xyz:8020/apps/hive/warehouse/tpcds_bin_partitioned_orc_10000.db/.hive-staging_hive_2016-05-11_09-40-42_489_5056654133706433454-1/-ext-10002
> to destination
> hdfs://xyz:8020/apps/hive/warehouse/tpcds_bin_partitioned_orc_10000.db/date_dim1
> {noformat}
> https://github.com/apache/hive/blob/26b5c7b56a4f28ce3eabc0207566cce46b29b558/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2836
> hdfsEncryptionShim.isPathEncrypted(destf) in Hive could end up throwing
> FileNotFoundException as the destf is not present yet. This causes moveFile
> to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
