[
https://issues.apache.org/jira/browse/SPARK-46523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhixingheyi-tian updated SPARK-46523:
-------------------------------------
Description:
Single Hive/Datasource table partitioning to multiple storage system- (e.g, S3
and HDFS)
For Hive table:
{code:java}
CREATE TABLE htable a string, b string) PARTITIONED BY ( p string ) location
"hdfs://{cluster}}/user/hadoop/htable/";
alter table htable add partition(p='p1') location
's3a://{bucketname}/usr/hive/warehouse/htable/p=p1';{code}
When inserting into htable, or insert overwrite htable. New data of “p=p1”
will insert table location storage. This does not meet the requirements.
Is there any best practise? Or is there a plan to support this feature?
Thanks
> Single Hive/Datasource table partitioning to multiple storage system- (e.g,
> S3 and HDFS)
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-46523
> URL: https://issues.apache.org/jira/browse/SPARK-46523
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.3.2
> Reporter: zhixingheyi-tian
> Priority: Major
>
> Single Hive/Datasource table partitioning to multiple storage system- (e.g,
> S3 and HDFS)
> For Hive table:
>
> {code:java}
> CREATE TABLE htable a string, b string) PARTITIONED BY ( p string )
> location "hdfs://{cluster}}/user/hadoop/htable/";
> alter table htable add partition(p='p1') location
> 's3a://{bucketname}/usr/hive/warehouse/htable/p=p1';{code}
>
> When inserting into htable, or insert overwrite htable. New data of “p=p1”
> will insert table location storage. This does not meet the requirements.
> Is there any best practise? Or is there a plan to support this feature?
> Thanks
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
