[
https://issues.apache.org/jira/browse/HUDI-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386369#comment-17386369
]
ASF GitHub Bot commented on HUDI-1771:
--------------------------------------
vinothchandar commented on pull request #3285:
URL: https://github.com/apache/hudi/pull/3285#issuecomment-885764276
@codope please help me close out the schema evolution story here.
my point was: when mixing old files where _hoodie_operation is NOT present
with new files where it is present, the queries can break?
@nsivabalan's response
> yes. Check row 4 here
https://hudi.apache.org/docs/writing_data.html#schema-evolution
> Here is what could be happening. Your new batch of write (with new
operation field), might be routed to all existing file groups and hence it
might have worked. Can you try sending to only partial file groups.
> If you are doing some sanity test, you can do this.
> batch1: old schema: route to partition_0
> batch2: new schema w/ operation field: route to partition_1
> unless operation field is appended to the end of the existing schema, this
may not work.
> Let us know what do you see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Propagate CDC format for hoodie
> -------------------------------
>
> Key: HUDI-1771
> URL: https://issues.apache.org/jira/browse/HUDI-1771
> Project: Apache Hudi
> Issue Type: New Feature
> Components: Flink Integration
> Reporter: Danny Chen
> Assignee: Zheng yunhong
> Priority: Major
> Labels: pull-request-available, sev:normal
> Fix For: 0.9.0
>
>
> Like what we discussed in the dev mailing list:
> https://lists.apache.org/thread.html/r31b2d1404e4e043a5f875b78105ba6f9a801e78f265ad91242ad5eb2%40%3Cdev.hudi.apache.org%3E
> Keep the change flags make new use cases possible: using HUDI as the unified
> storage format for DWD and DWS layer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
