[
https://issues.apache.org/jira/browse/FLINK-35375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
田元志 updated FLINK-35375:
------------------------
Description:
This is an error reported by Flink CDC data synchronization because a lot of
data was missed during the process of synchronizing from polardb to selectdb.
We restarted the task from the binlog position and reported the same error
later. We saw from the Alibaba community that someone encountered this problem
and provided a debezium parameter setting that can solve it.
{code:java}
connect.keep.live.interval.ms=3000{code}
.We set this parameter but did not change any results, and as soon as the task
was started, an error was immediately reported, as shown below:
java.lang.RuntimeException: One or more fetchers have encountered exception at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:225)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:169)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:130)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:156)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at java.lang.Thread.run(Thread.java:750)
~[?:1.8.0_341]
{color:#FF0000}Caused by:{color} {color:#ff0000}java.lang.RuntimeException:
SplitFetcher thread 0 received unexpected exception while polling the records
at{color}
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:150)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_341] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_341] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_341] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_341] ... 1 more
{color:#ff0000}Caused by: org.apache.kafka.connect.errors.ConnectException: An
exception occurred in the change event producer. This connector will be
stopped. at
io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50){color}
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.ververica.cdc.connectors.mysql.debezium.task.context.MySqlErrorHandler.setProducerThrowable(MySqlErrorHandler.java:85)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1553)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1064)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
{color:#ff0000}Caused by: io.debezium.DebeziumException: Failed to deserialize
data of EventHeaderV4\{timestamp=1715762712000, eventType=WRITE_ROWS,
serverId=27545817, headerLength=19, dataLength=8069, nextPosition=394608452,
flags=0} at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1488)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] {color}at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1553)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1064)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
{color:#ff0000}Caused by:
com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException:
Failed to deserialize data of EventHeaderV4\{timestamp=1715762712000,
eventType=WRITE_ROWS, serverId=27545817, headerLength=19, dataLength=8069,
nextPosition=394608452, flags=0}{color} at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:341)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:244)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:259)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
{color:#ff0000}Caused by:
com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException:
No TableMapEventData has been found for table id:73858. Usually that means
that you have started reading binary log 'within the logical event group' (e.g.
from WRITE_ROWS and not proceeding TABLE_MAP at
com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:109){color}
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserializeRows(WriteRowsEventDataDeserializer.java:64)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:56)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:32)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:244)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:259)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
was:
This is an error reported by Flink CDC data synchronization because a lot of
data was missed during the process of synchronizing from polardb to selectdb.
We restarted the task from the binlog position and reported the same error
later. We saw from the Alibaba community that someone encountered this problem
and provided a debezium parameter setting that can solve it.
{code:java}
connect.keep.live.interval.ms=3000{code}
.We set this parameter but did not change any results, and as soon as the task
was started, an error was immediately reported, as shown below:
java.lang.RuntimeException: One or more fetchers have encountered exception at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:225)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:169)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:130)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:156)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at java.lang.Thread.run(Thread.java:750)
~[?:1.8.0_341] Caused by: {color:#FF0000}java.lang.RuntimeException:
SplitFetcher thread 0 received unexpected exception while polling the records
at{color}
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:150)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[?:1.8.0_341] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_341] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
~[?:1.8.0_341] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
~[?:1.8.0_341] ... 1 more {color:#FF0000}Caused by:
org.apache.kafka.connect.errors.ConnectException: An exception occurred in the
change event producer. This connector will be stopped. at
io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50){color}
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.ververica.cdc.connectors.mysql.debezium.task.context.MySqlErrorHandler.setProducerThrowable(MySqlErrorHandler.java:85)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1553)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1064)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more {color:#FF0000}Caused by:
io.debezium.DebeziumException: Failed to deserialize data of
EventHeaderV4\{timestamp=1715762712000, eventType=WRITE_ROWS,
serverId=27545817, headerLength=19, dataLength=8069, nextPosition=394608452,
flags=0} at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1488)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] {color}at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1553)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1064)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more {color:#FF0000}Caused by:
com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException:
Failed to deserialize data of EventHeaderV4\{timestamp=1715762712000,
eventType=WRITE_ROWS, serverId=27545817, headerLength=19, dataLength=8069,
nextPosition=394608452, flags=0}{color} at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:341)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:244)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:259)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more {color:#FF0000}Caused by:
com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException:
No TableMapEventData has been found for table id:73858. Usually that means
that you have started reading binary log 'within the logical event group' (e.g.
from WRITE_ROWS and not proceeding TABLE_MAP at
com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:109){color}
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserializeRows(WriteRowsEventDataDeserializer.java:64)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:56)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:32)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:244)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:259)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
> About a question report "Failed to deserialize data of EventHeaderV4" of
> flink cdc2.4.2
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-35375
> URL: https://issues.apache.org/jira/browse/FLINK-35375
> Project: Flink
> Issue Type: Bug
> Reporter: 田元志
> Priority: Major
>
> This is an error reported by Flink CDC data synchronization because a lot of
> data was missed during the process of synchronizing from polardb to selectdb.
> We restarted the task from the binlog position and reported the same error
> later. We saw from the Alibaba community that someone encountered this
> problem and provided a debezium parameter setting that can solve it.
> {code:java}
> connect.keep.live.interval.ms=3000{code}
> .We set this parameter but did not change any results, and as soon as the
> task was started, an error was immediately reported, as shown below:
>
> java.lang.RuntimeException: One or more fetchers have encountered exception
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:225)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:169)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:130)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:156)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:550)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at java.lang.Thread.run(Thread.java:750)
> ~[?:1.8.0_341]
> {color:#FF0000}Caused by:{color} {color:#ff0000}java.lang.RuntimeException:
> SplitFetcher thread 0 received unexpected exception while polling the records
> at{color}
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:150)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:105)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[?:1.8.0_341] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[?:1.8.0_341] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> ~[?:1.8.0_341] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ~[?:1.8.0_341] ... 1 more
> {color:#ff0000}Caused by: org.apache.kafka.connect.errors.ConnectException:
> An exception occurred in the change event producer. This connector will be
> stopped. at
> io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50){color}
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.ververica.cdc.connectors.mysql.debezium.task.context.MySqlErrorHandler.setProducerThrowable(MySqlErrorHandler.java:85)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1553)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1064)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
> {color:#ff0000}Caused by: io.debezium.DebeziumException: Failed to
> deserialize data of EventHeaderV4\{timestamp=1715762712000,
> eventType=WRITE_ROWS, serverId=27545817, headerLength=19, dataLength=8069,
> nextPosition=394608452, flags=0} at
> io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1488)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] {color}at
> io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onEventDeserializationFailure(MySqlStreamingChangeEventSource.java:1553)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1064)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
> {color:#ff0000}Caused by:
> com.github.shyiko.mysql.binlog.event.deserialization.EventDataDeserializationException:
> Failed to deserialize data of EventHeaderV4\{timestamp=1715762712000,
> eventType=WRITE_ROWS, serverId=27545817, headerLength=19, dataLength=8069,
> nextPosition=394608452, flags=0}{color} at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:341)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:244)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:259)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
> {color:#ff0000}Caused by:
> com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException:
> No TableMapEventData has been found for table id:73858. Usually that means
> that you have started reading binary log 'within the logical event group'
> (e.g. from WRITE_ROWS and not proceeding TABLE_MAP at
> com.github.shyiko.mysql.binlog.event.deserialization.AbstractRowsEventDataDeserializer.deserializeRow(AbstractRowsEventDataDeserializer.java:109){color}
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserializeRows(WriteRowsEventDataDeserializer.java:64)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:56)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.event.deserialization.WriteRowsEventDataDeserializer.deserialize(WriteRowsEventDataDeserializer.java:32)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.deserializeEventData(EventDeserializer.java:335)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.event.deserialization.EventDeserializer.nextEvent(EventDeserializer.java:244)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> io.debezium.connector.mysql.MySqlStreamingChangeEventSource$1.nextEvent(MySqlStreamingChangeEventSource.java:259)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1051)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] at
> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932)
> ~[hlkj-bigdata-1.0-SNAPSHOT.jar:?] ... 1 more
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
