[
https://issues.apache.org/jira/browse/HIVE-28118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Palakur Eshwitha Sai updated HIVE-28118:
----------------------------------------
Description:
With Viewfs overload scheme enabled in the cluster, hive insert into operation
fails with MoveTask error in below scenarios:
1. Encryption zones are enabled for hive warehouse directory and insert into S3
is done.
2. Encryption zones are not enabled for the hive warehouse directory (hadoop
security credential provider path is not null but the warehouse directory is
not encrypted using a key). Both insert into HDFS and S3 fail here.
The setup has a hive table with one of the partitions moved to S3 and mount
point configured to it using Viewfs overload scheme. The rest of the partitions
and other hdfs folders are mounted using linkFallback.
S3 does not support encryption zones.
was:
With Viewfs overload scheme enabled in the cluster, hive insert into S3
operation fails with MoveTask error when HDFS encryption zones are enabled. The
HDFS hive warehouse path is encrypted and one of the partitions is present in
S3. The logic checks if encryption is enabled using the DFSUtilClient, which is
true for HDFS paths but S3 does not support encryption zones.
The setup has a hive table with one of the partitions moved to S3 and mount
point configured to it using Viewfs overload scheme. The rest of the partitions
and other hdfs folders are mounted using linkFallback.
> Hive insert into S3 with Viewfs overload scheme fails with MoveTask error
> -------------------------------------------------------------------------
>
> Key: HIVE-28118
> URL: https://issues.apache.org/jira/browse/HIVE-28118
> Project: Hive
> Issue Type: Bug
> Reporter: Palakur Eshwitha Sai
> Assignee: Palakur Eshwitha Sai
> Priority: Major
> Labels: pull-request-available
> Attachments: code.png
>
>
> With Viewfs overload scheme enabled in the cluster, hive insert into
> operation fails with MoveTask error in below scenarios:
> 1. Encryption zones are enabled for hive warehouse directory and insert into
> S3 is done.
> 2. Encryption zones are not enabled for the hive warehouse directory (hadoop
> security credential provider path is not null but the warehouse directory is
> not encrypted using a key). Both insert into HDFS and S3 fail here.
> The setup has a hive table with one of the partitions moved to S3 and mount
> point configured to it using Viewfs overload scheme. The rest of the
> partitions and other hdfs folders are mounted using linkFallback.
> S3 does not support encryption zones.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
