Simon Tao created FLINK-22089:
-
Summary: cdc checkpoint invalid
Key: FLINK-22089
URL: https://issues.apache.org/jira/browse/FLINK-22089
Project: Flink
Issue Type: Bug
Components: Connectors / Common, Runtime / Checkpointing
Affects Versions: 1.12.0
Environment: public static void main(String[] args) {
SourceFunction sourceFunction = MySQLSource.builder()
.hostname("xxx")
.port(3306)
.databaseList("xxx") // monitor all tables under inventory database
.username("xxx")
.password("xxx")
.deserializer(new StringDebeziumDeserializationSchema()) // converts
SourceRecord to String
.build();
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(5000);
try
{ env.setStateBackend(new RocksDBStateBackend("file:///E:/tmp/t2")); }
catch (IOException e)
{ e.printStackTrace(); }
env.addSource(sourceFunction).print().setParallelism(1); // use parallelism 1
for sink to keep message ordering
try \{ env.execute(); } catch (Exception e) \{ e.printStackTrace(); }
}
maven=
maven-assembly-plugin
false
jar-with-dependencies
make-assembly
package
assembly
org.apache.maven.plugins
maven-compiler-plugin
8
8
org.apache.flink
flink-runtime-web_${scala.version}
${flink.version}
org.apache.flink
flink-connector-jdbc_${scala.version}
${flink.version}
org.apache.flink
flink-clients_${scala.version}
${flink.version}
org.apache.flink
flink-test-utils_${scala.version}
${flink.version}
org.apache.flink
flink-statebackend-rocksdb_${scala.version}
${flink.version}
com.alibaba.ververica
flink-connector-mysql-cdc
1.3.0
ru.ivi.opensource
flink-clickhouse-sink
1.3.0
Reporter: Simon Tao
Fix For: shaded-13.0
i turn on checkpoint but it seems invalid , during the whole process ,the
checkpoint file can write to localfile and the flink can read the cdc recrods
normally, but when i restart flink in idea ,it always reload consumed records .
i paste on my code and maven configuration below
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
