Mason Chen created FLINK-31305: ---------------------------------- Summary: KafkaWriter doesn't wait for errors for in-flight records before completing flush Key: FLINK-31305 URL: https://issues.apache.org/jira/browse/FLINK-31305 Project: Flink Issue Type: Improvement Components: Connectors / Kafka Affects Versions: 1.16.1, 1.17.0 Reporter: Mason Chen Fix For: 1.17.0
The KafkaWriter flushing needs to wait for all in-flight records to send successfully. This can be achieved by tracking requests and returning a response from the registered callback from the producer#send() logic. There is potential for data loss since the checkpoint does not accurately reflect that all records have been sent successfully, to preserve at least once semantics. -- This message was sent by Atlassian Jira (v8.20.10#820010)