[
https://issues.apache.org/jira/browse/HUDI-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar updated HUDI-7538:
---------------------------------
Description:
For sake of more consistency, we need to consolidate the the changelog mode
(currently supported for Flink MoR) and RFC-51 based CDC feature which is a
debezium style change log (currently supported for CoW for Spark/Flink)
|Format Name|CDC Source Required|Resource Cost(writer)|Resource
Cost(reader)|Friendly to Streaming|
|CDC|*No*|low/high|low/high (based on logging modes we choose)|No (the debezium
style output is not what Flink needs for e.g)|
|Changelog|Yes|low|low|Yes|
This proposal is to converge onto "CDC" as the path going forward, with the
following changes to incorporate.
was:
For sake of more consistency, we need to consolidate the the changelog mode
(currently supported for Flink MoR) and RFC-51 based CDC feature which is a
debezium style change log (currently supported for CoW for Spark/Flink)
|Format Name|CDC Source Required|Resource Cost(writer)|Resource
Cost(reader)|Friendly to Streaming|
|CDC|*No*|low/high|low/high (based on logging modes we choose)|No (the debezium
style output is not what Flink needs for e.g)|
|Changelog|Yes|low|low|Yes|
This proposal is to converge onto "CDC" as the path going forward, with the
following changes to incorporate.
> Consolidate the CDC Formats (changelog format, RFC-51)
> ------------------------------------------------------
>
> Key: HUDI-7538
> URL: https://issues.apache.org/jira/browse/HUDI-7538
> Project: Apache Hudi
> Issue Type: Improvement
> Components: storage-management
> Reporter: Vinoth Chandar
> Assignee: Vinoth Chandar
> Priority: Major
> Labels: hudi-1.0.0-beta2
> Fix For: 1.0.0
>
>
> For sake of more consistency, we need to consolidate the the changelog mode
> (currently supported for Flink MoR) and RFC-51 based CDC feature which is a
> debezium style change log (currently supported for CoW for Spark/Flink)
>
> |Format Name|CDC Source Required|Resource Cost(writer)|Resource
> Cost(reader)|Friendly to Streaming|
> |CDC|*No*|low/high|low/high (based on logging modes we choose)|No (the
> debezium style output is not what Flink needs for e.g)|
> |Changelog|Yes|low|low|Yes|
> This proposal is to converge onto "CDC" as the path going forward, with the
> following changes to incorporate.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
