[
https://issues.apache.org/jira/browse/HUDI-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-6453.
----------------------------
Fix Version/s: 1.0.0
Resolution: Fixed
Fixed via master branch: 2024d17704e6856174fc6b56ba51cd2d1b93b49d
> Cascade schema changes to partition level
> -----------------------------------------
>
> Key: HUDI-6453
> URL: https://issues.apache.org/jira/browse/HUDI-6453
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Shawn Chang
> Assignee: Shawn Chang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Attachments: schema_reproduce.scala
>
>
> https://github.com/apache/hudi/blob/a07549fd13bae6a74d9b82956e10520c55de3c63/hudi-aws/src/main/java/org/apache/hudi/aws/sync/AWSGlueCatalogSyncClient.java#L333
> Currently Glue sync tool in Hudi won't cascade partition-level changes to
> glue .
> This would cause schema mismatch issue when user altered their table schema
> in a newer partition and then query older partitions with engine that are
> aware of partition-level schema like Athena.
> Glue doesn't provide a convenient API for cascading so we have to update Glue
> partitions directly
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
