[
https://issues.apache.org/jira/browse/FLINK-25330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17461040#comment-17461040
]
Jing Ge commented on FLINK-25330:
---------------------------------
Hi Bruce,
thanks for your contribution. It is correct to delete all versions in your
case, but, tbh, I have actually also concern similar to what WenLong Lyu said.
There are many different business scenarios, many of them might expect only
deleting the last version and keep the rest of them. If we modified the core
logic to just delete all versions of the column, unexpected data lost will
happen, which will raise critical incident in production. Comparing to that,
current solution is the safe and feasible one.
I think the issue is about what is the right default deletion logic and how to
make it flexible enough e.g. configurable, for different use cases.
Are you using SQL for your CDC job? Could you provide more information about it?
> Flink SQL doesn't retract all versions of Hbase data
> ----------------------------------------------------
>
> Key: FLINK-25330
> URL: https://issues.apache.org/jira/browse/FLINK-25330
> Project: Flink
> Issue Type: Bug
> Components: Connectors / HBase
> Affects Versions: 1.14.0
> Reporter: Bruce Wong
> Assignee: Jing Ge
> Priority: Critical
> Labels: pull-request-available
> Attachments: image-2021-12-15-20-05-18-236.png
>
>
> h2. Background
> When we use CDC to synchronize mysql data to HBase, we find that HBase
> deletes only the last version of the specified rowkey when deleting mysql
> data. The data of the old version still exists. You end up using the wrong
> data. And I think its a bug of HBase connector.
> The following figure shows Hbase data changes before and after mysql data is
> deleted.
> !image-2021-12-15-20-05-18-236.png|width=910,height=669!
>
> h2.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
