pnowojski commented on issue #7405: [FLINK-11249] FlinkKafkaProducer011 can not be migrated to FlinkKafkaProducer URL: https://github.com/apache/flink/pull/7405#issuecomment-458530867 Thanks again for the progress in this issue. I think there might be couple of issues: 1. You call snapshot with the same ID twice ``` testHarness.snapshot(1L, 3L); OperatorSubtaskState snapshot = testHarness.snapshot(1L, 3L); ``` 2. all 3 of your tests seem to run against the same Kafka brokers, ZooKeeper instances. You shouldn't be recovering to the same savepoint twice, without reseting external state in Kafka brokers and ZooKeeper. Otherwise second (and subsequent) recoveries will be working against Kafka/ZooKeeper state as it was left by the first recovery (pending transactions committed/aborted etc). 3. Kafka has a transaction timeout for pending transactions, after which the transactions are being aborted. If the time between writing down the Kafka/ZooKeeper state and restoring from it is greater then the said timeout, probably transactions will be aborted. I'm afraid that this might prove impossible to test migration with pending transactions from previous Flink versions :/ The only thing that we could test is to write down the Kafka/ZooKeeper state with pending transactions with current code version, restart Kafka (possibly with newer version) and try to recover from savepoint...
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
