[
https://issues.apache.org/jira/browse/FLINK-22089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jark Wu reopened FLINK-22089:
-----------------------------
> cdc checkpoint invalid
> ----------------------
>
> Key: FLINK-22089
> URL: https://issues.apache.org/jira/browse/FLINK-22089
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common, Runtime / Checkpointing
> Affects Versions: 1.12.0
> Environment: public static void main(String[] args) {
> SourceFunction<String> sourceFunction = MySQLSource.<String>builder()
> .hostname("xxx")
> .port(3306)
> .databaseList("xxx") // monitor all tables under inventory database
> .username("xxx")
> .password("xxx")
> .deserializer(new StringDebeziumDeserializationSchema()) // converts
> SourceRecord to String
> .build();
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> env.enableCheckpointing(5000);
> try
> { env.setStateBackend(new RocksDBStateBackend("file:///E:/tmp/t2")); }
> catch (IOException e)
> { e.printStackTrace(); }
> env.addSource(sourceFunction).print().setParallelism(1); // use parallelism
> 1 for sink to keep message ordering
>
> try \{ env.execute(); } catch (Exception e) \{ e.printStackTrace(); }
> }
>
> ====================maven=========================
>
> <build>
> <plugins>
> <plugin>
> <artifactId>maven-assembly-plugin</artifactId>
> <configuration>
> <appendAssemblyId>false</appendAssemblyId>
> <descriptorRefs>
> <descriptorRef>jar-with-dependencies</descriptorRef>
> </descriptorRefs>
> </configuration>
> <executions>
> <execution>
> <id>make-assembly</id>
> <phase>package</phase>
> <goals>
> <goal>assembly</goal>
> </goals>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-compiler-plugin</artifactId>
> <configuration>
> <source>8</source>
> <target>8</target>
> </configuration>
> </plugin>
> </plugins>
> </build>
> <dependencies>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-runtime-web_${scala.version}</artifactId>
> <version>${flink.version}</version>
> </dependency>
> <!--
> https://mvnrepository.com/artifact/org.apache.flink/flink-connector-jdbc -->
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-connector-jdbc_${scala.version}</artifactId>
> <version>${flink.version}</version>
> </dependency>
> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients -->
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-clients_${scala.version}</artifactId>
> <version>${flink.version}</version>
> </dependency>
> <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-test-utils -->
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-test-utils_${scala.version}</artifactId>
> <version>${flink.version}</version>
> <!--<scope>test</scope>-->
> </dependency>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-statebackend-rocksdb_${scala.version}</artifactId>
> <version>${flink.version}</version>
> </dependency>
> <!--
> https://mvnrepository.com/artifact/com.alibaba.ververica/flink-connector-mysql-cdc
> -->
> <dependency>
> <groupId>com.alibaba.ververica</groupId>
> <artifactId>flink-connector-mysql-cdc</artifactId>
> <version>1.3.0</version>
> </dependency>
> <!--
> https://mvnrepository.com/artifact/ru.ivi.opensource/flink-clickhouse-sink -->
> <dependency>
> <groupId>ru.ivi.opensource</groupId>
> <artifactId>flink-clickhouse-sink</artifactId>
> <version>1.3.0</version>
> </dependency>
> </dependencies>
>
> Reporter: Simon Tao
> Priority: Major
> Fix For: shaded-13.0
>
>
> i turn on checkpoint but it seems invalid , during the whole process ,the
> checkpoint file can write to localfile and the flink can read the cdc recrods
> normally, but when i restart flink in idea ,it always reload consumed records
> . i paste on my code and maven configuration below
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
