[
https://issues.apache.org/jira/browse/HIVE-28118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Palakur Eshwitha Sai updated HIVE-28118:
----------------------------------------
Summary: Hive insert into S3 with Viewfs overload scheme fails with
MoveTask error (was: Hive Insert Into Viewfs Overload without Encryption zones
fails with MoveTask Error)
> Hive insert into S3 with Viewfs overload scheme fails with MoveTask error
> -------------------------------------------------------------------------
>
> Key: HIVE-28118
> URL: https://issues.apache.org/jira/browse/HIVE-28118
> Project: Hive
> Issue Type: Bug
> Reporter: Palakur Eshwitha Sai
> Assignee: Palakur Eshwitha Sai
> Priority: Major
> Labels: pull-request-available
> Attachments: code.png
>
>
> With Viewfs overload scheme enabled in the cluster, hive insert into
> operation fails with MoveTask error when the path is not in encryption zone.
> The setup has a hive table with one of the partitions moved to S3 and mount
> point configured to it using Viewfs overload scheme. The rest of the
> partitions and other hdfs folders are mounted using linkFallback.
> S3 does not support encryption zones, below is the error stacktrace:
> {code:java}
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
> hdfs://s3cluster/warehouse/tablespace/external/hive/sales_by_state/state=CA/.hive-staging_hive_2024-01-05_05-52-09_785_7084847051463431810-8/-ext-10002
> to destination
> hdfs://s3cluster/warehouse/tablespace/external/hive/sales_by_state/state=CA/.hive-staging_hive_2024-01-05_05-52-09_785_7084847051463431810-8/-ext-10000
> (state=08S01,code=1)
> Caused by: java.io.FileNotFoundException: No such file or directory:
> s3a://testhadoop/warehouse/tablespace/external/hive/sales_by_state/state=CA/.hive-staging_hive_2024-01-05_05-52-09_785_7084847051463431810-8/-ext-10000
>
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.s3GetFileStatus(S3AFileSystem.java:2344)
> ~[hadoop-aws-3.2.2.3.2.2.4-8.jar:?]
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2226)
> ~[hadoop-aws-3.2.2.3.2.2.4-8.jar:?]
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2160)
> ~[hadoop-aws-3.2.2.3.2.2.4-8.jar:?]
> at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:888)
> ~[hadoop-common-3.2.2.3.2.2.4-6.jar:?]{code}
> Similar issue is observed for hive insert into hdfs paths without encryption
> zones when viewfs overload is configured in the cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
