[ 
https://issues.apache.org/jira/browse/FLINK-41604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated FLINK-41604:
-----------------------------------
    Labels: pull-request-available  (was: )

> [db2] Db2 CDC source cannot restore from a checkpoint or savepoint taken 
> after it read a change: ClassCastException in Db2OffsetContext$Loader
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-41604
>                 URL: https://issues.apache.org/jira/browse/FLINK-41604
>             Project: Flink
>          Issue Type: Bug
>          Components: Flink CDC
>    Affects Versions: cdc-3.5.0, cdc-3.7.0
>            Reporter: Purushottam Sinha
>            Priority: Major
>              Labels: pull-request-available
>
> h3. Problem
> A Db2 CDC job ({{db2-cdc}}, incremental source) cannot restore from a 
> checkpoint or savepoint taken after it has read at least one change event. 
> Every restore attempt fails, so the job neither resumes after 
> stop-with-savepoint nor recovers from a checkpoint after a failure; it keeps 
> restarting:
> {noformat}
> java.lang.ClassCastException: class java.lang.String cannot be cast to class 
> java.lang.Long
>     at 
> io.debezium.connector.db2.Db2OffsetContext$Loader.load(Db2OffsetContext.java:152)
>     at 
> org.apache.flink.cdc.connectors.db2.source.fetch.Db2SourceFetchTaskContext.loadStartingOffsetState(Db2SourceFetchTaskContext.java:170)
>     at 
> org.apache.flink.cdc.connectors.db2.source.fetch.Db2SourceFetchTaskContext.configure(Db2SourceFetchTaskContext.java:113)
>     at 
> org.apache.flink.cdc.connectors.base.source.reader.external.IncrementalSourceStreamFetcher.submitTask(IncrementalSourceStreamFetcher.java:88)
> {noformat}
> A checkpoint or savepoint taken before any change is read (right after the 
> snapshot) restores fine.
> h3. Cause
> The Db2 incremental source stores the stream position as strings, including 
> {{event_serial_no}} ({{db2/source/offset/LsnOffset.java}}):
> {code:java}
> offsetMap.put(EVENT_SERIAL_NO_KEY, String.valueOf(eventSerialNo));
> {code}
> On restore, {{Db2SourceFetchTaskContext#loadStartingOffsetState}} passes that 
> map to Debezium's (1.9.8.Final) {{Db2OffsetContext.Loader}}, which expects a 
> {{Long}}:
> {code:java}
> Long eventSerialNo = ((Long) offset.get(EVENT_SERIAL_NO_KEY));   // 
> Db2OffsetContext.java:152
> {code}
> Positions saved before any change was read carry no {{event_serial_no}}, so 
> the cast gets {{null}}; that is why the snapshot and the first start work.
> FLINK-36086 hit the same mismatch for SQL Server and added a patched 
> {{io.debezium.connector.sqlserver.SqlServerOffsetContext}} whose loader 
> parses the string. {{flink-connector-db2-cdc}} patches {{Db2Connection}} and 
> {{Db2StreamingChangeEventSource}} but not {{Db2OffsetContext}}. The Db2 
> {{LsnOffset}} has stored {{event_serial_no}} as a string since the 
> incremental source was added (FLINK-34842, first released in cdc-3.2.0); 
> master is unchanged.
> {{Db2SourceITCase}}'s stream-phase failover tests do not catch it: they 
> trigger the failover about 200 ms after making changes, with 1 s checkpoints, 
> so the restored checkpoint predates the change events.
> h3. Reproduction
> Apache Flink's official {{flink:1.20.3-java17}} image with 
> {{flink-sql-connector-db2-cdc-3.5.0.jar}} and {{com.ibm.db2:jcc:11.5.9.0}} in 
> {{lib/}}; {{icr.io/db2_community/db2:11.5.9.0}} with ASN Capture set up by 
> the {{db2_server}} scripts of this repository's Db2 tests (table registered 
> with {{ASNCDC.ADDTABLE}}); checkpoints every 2 s.
> {code:sql}
> CREATE TABLE src (ID INT NOT NULL, ITEM STRING, QTY INT, PRIMARY KEY (ID) NOT 
> ENFORCED) WITH (
>   'connector' = 'db2-cdc', 'hostname' = '...', 'port' = '50000', 'username' = 
> 'db2inst1', 'password' = '...',
>   'database-name' = 'cdc', 'table-name' = 'DB2INST1.ORDERS');
> CREATE TABLE out_rows (ID INT, ITEM STRING, QTY INT) WITH ('connector' = 
> 'print');
> INSERT INTO out_rows SELECT * FROM src;
> {code}
> # Start the job; the 3 snapshot rows are printed.
> # Insert, update and delete rows in Db2; the changes are printed.
> # Stop the job with a savepoint, change more rows, start the job again from 
> the savepoint.
> # The job restarts in a loop with the exception above and prints nothing.
> If the job is stopped right after the snapshot instead (no change read), it 
> resumes and prints the later changes.
> Reproduced with cdc-3.5.0 on Flink 1.20.3 and with cdc-3.7.0 ({{-2.2}} build) 
> on Flink 2.2.0.
> h3. Proposed fix
> Add a patched {{io.debezium.connector.db2.Db2OffsetContext}} to 
> {{flink-connector-db2-cdc}} whose loader accepts a string or a {{Long}} for 
> {{event_serial_no}}, as FLINK-36086 did for SQL Server; parsing on read also 
> restores positions saved by current releases. Add a test that restores from a 
> checkpoint or savepoint taken after a change event was read.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to