Hongshun Wang created FLINK-35859:
-------------------------------------
Summary: [flink-cdc] Fix: The assigner is not ready to offer
finished split information, this should not be called
Key: FLINK-35859
URL: https://issues.apache.org/jira/browse/FLINK-35859
Project: Flink
Issue Type: Bug
Components: Flink CDC
Affects Versions: cdc-3.1.1
Reporter: Hongshun Wang
Fix For: cdc-3.2.0
When use CDC with newly added table, an error occurs:
{code:java}
The assigner is not ready to offer finished split information, this should not
be called. {code}
It's because:
1. when stop then restart the job , the status is
NEWLY_ADDED_ASSIGNING_SNAPSHOT_FINISHED.
2. Then Enumerator will send each reader with
BinlogSplitUpdateRequestEvent to update binlog. (see
org.apache.flink.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator#syncWithReaders).
3. The Reader will suspend binlog reader then send BinlogSplitMetaRequestEvent
to Enumerator.
4. The Enumerator found that some tables are not sent, an error will occur
{code:java}
private void sendBinlogMeta(int subTask, BinlogSplitMetaRequestEvent
requestEvent) {
// initialize once
if (binlogSplitMeta == null) {
final List<FinishedSnapshotSplitInfo> finishedSnapshotSplitInfos =
splitAssigner.getFinishedSplitInfos();
if (finishedSnapshotSplitInfos.isEmpty()) {
LOG.error(
"The assigner offers empty finished split information, this
should not happen");
throw new FlinkRuntimeException(
"The assigner offers empty finished split information, this
should not happen");
}
binlogSplitMeta =
Lists.partition(
finishedSnapshotSplitInfos,
sourceConfig.getSplitMetaGroupSize());
}
}{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
