Danny Cranmer created FLINK-19899: ------------------------------------- Summary: [Kinesis][EFO] Optimise error handling to use a separate exception delivery mechanism Key: FLINK-19899 URL: https://issues.apache.org/jira/browse/FLINK-19899 Project: Flink Issue Type: Improvement Components: Connectors / Kinesis Reporter: Danny Cranmer Fix For: 1.12.0
*Background* There is a queue used to pass events between the network client and consumer application. When an error is thrown in the network thread, the queue is cleared to make space for the error event. This means that records will be thrown away to make space for errors (the records would be subsequently reloaded from the shard). *Scope* Add a new mechanism to pass exceptions between threads, meaning data does not need to be discarded. When an error is thrown, the error event will be processed by the consumer once all of the records have been processed. -- This message was sent by Atlassian Jira (v8.3.4#803005)