zhixingheyi-tian created SPARK-46523:
----------------------------------------
Summary: Single Hive/Datasource table partitioning to multiple
storage system- (e.g, S3 and HDFS)
Key: SPARK-46523
URL: https://issues.apache.org/jira/browse/SPARK-46523
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.3.2
Environment: Single Hive/Datasource table partitioning to multiple
storage system- (e.g, S3 and HDFS)
For Hive table:
{code:java}
CREATE TABLE htable a string, b string) PARTITIONED BY ( p string ) location
"hdfs://{cluster}}/user/hadoop/htable/";
alter table htable add partition(p='p1') location
's3a://{bucketname}/usr/hive/warehouse/htable/p=p1';{code}
When inserting into htable, or insert overwrite htable. New data of “p=p1”
will insert table location storage. This does not meet the requirements.
Is there any best practise? Or is there a plan to support this feature?
Thanks
Reporter: zhixingheyi-tian
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
