[
https://issues.apache.org/jira/browse/FLINK-35859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17900229#comment-17900229
]
Hongshun Wang commented on FLINK-35859:
---------------------------------------
> Users cannot immediately perceive task issues in production applications.
I agree with you. CDC should also provide metics about AssignerStatus to let
others know.
> [flink-cdc] Fix: The assigner is not ready to offer finished split
> information, this should not be called
> ---------------------------------------------------------------------------------------------------------
>
> Key: FLINK-35859
> URL: https://issues.apache.org/jira/browse/FLINK-35859
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.1.1
> Reporter: Hongshun Wang
> Assignee: Hongshun Wang
> Priority: Minor
> Fix For: cdc-3.2.0
>
>
> When use CDC with newly added table, an error occurs:
> {code:java}
> The assigner is not ready to offer finished split information, this should
> not be called. {code}
> It's because:
> 1. when stop then restart the job , the status is
> NEWLY_ADDED_ASSIGNING_SNAPSHOT_FINISHED.
>
> 2. Then Enumerator will send each reader with
> BinlogSplitUpdateRequestEvent to update binlog. (see
> org.apache.flink.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator#syncWithReaders).
> 3. The Reader will suspend binlog reader then send
> BinlogSplitMetaRequestEvent to Enumerator.
> 4. The Enumerator found that some tables are not sent, an error will occur
> {code:java}
> private void sendBinlogMeta(int subTask, BinlogSplitMetaRequestEvent
> requestEvent) {
> // initialize once
> if (binlogSplitMeta == null) {
> final List<FinishedSnapshotSplitInfo> finishedSnapshotSplitInfos =
> splitAssigner.getFinishedSplitInfos();
> if (finishedSnapshotSplitInfos.isEmpty()) {
> LOG.error(
> "The assigner offers empty finished split information,
> this should not happen");
> throw new FlinkRuntimeException(
> "The assigner offers empty finished split information,
> this should not happen");
> }
> binlogSplitMeta =
> Lists.partition(
> finishedSnapshotSplitInfos,
> sourceConfig.getSplitMetaGroupSize());
> }
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
