pnowojski commented on issue #7405: [FLINK-11249] FlinkKafkaProducer011 can not be migrated to FlinkKafkaProducer URL: https://github.com/apache/flink/pull/7405#issuecomment-458609356 Sorry, I forgot to respond to that part: > I have started to work on the last test FlinkKafkaProducer011 --> FlinkKafkaProducer 👍 But I'm quickly stuck with an exception when initializing the testHarness ( created with FlinkKafkaProducer ) with the savepoint of FlinkKakfaProducer011. Is this action feasible with the testHarness API or there is a turn around to this issue ? I don't know. I'm not aware of any test that attempts to do that, so maybe this is a missing feature and maybe you are right that it has to be fixed somehow for the test. I have also thought more about the issue with timeouts on pending transactions. This means that the only way to test for 1.7 -> 1.8 migration with pending transactions, would be some awful code that downloads the Flink 1.7 sources/binaries and creates a savepoint during the test execution. I think that would be extremely difficult to do so we can ignore this issue for now... That would leave us with the following tests that theoretically we could implement: 1. migration of `FlinkKafkaProducer` from pre-made 1.7 savepoint to master without pending transactions 2. migration of `FlinkKafkaProducer011` from pre-made 1.7 savepoint to master without pending transactions 3. migration from `FlinkKafkaProducer011` master savepoint to `FlinkKafkaProducer` master without pending transactions 4. (optional) migration from `FlinkKafkaProducer011` pre-made 1.7 savepoint to `FlinkKafkaProducer` master without pending transactions 5. (optional) migration `FlinkKafkaProducer011` -> `FlinkKafkaProducer` from savepoint created on demand (during unit test) from master to master versions, with pending transactions 6. (optional) upgrading Kafka brokers when using `FlinkKafkaProducer` from savepoint created on demand (during unit test) from master to master versions, with pending transactions 1, 2 and 3 are IMO must have. There is a chance that Flink 1.8 will support "stop with savepoint" feature, which would mean that there would be no pending transactions in savepoints. With that guarantee, 1, 2 and 3 would essentially cover all of the required upgrade paths. Story of upgrading from 1.7 `FlinkKafkaProducer011` to 1.8 `FlinkKafkaProducer` would be covered first by step 1, then by step 3. 3, 4 and 5. have the issue that you reported that probably test harness needs to be adjusted 6. will have yet another issue of handling two different. 4, 5 and 6. would be nice to have, but as long as "stop with savepoint" is there, they are not strictly required.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
