Jiabao Sun created FLINK-41565:
----------------------------------
Summary: Prevent MongoDB sink cleanup failures from terminating
the TaskManager in Flink 2.2.x
Key: FLINK-41565
URL: https://issues.apache.org/jira/browse/FLINK-41565
Project: Flink
Issue Type: Bug
Components: Connectors / MongoDB
Affects Versions: 2.2.1, 2.2.0
Reporter: Jiabao Sun
Assignee: Jiabao Sun
Since FLINK-38180, Flink treats exceptions thrown during the cleanup of an
already failed task as fatal, potentially terminating the entire TaskManager.
Currently, MongoWriter.close() attempts to flush any remaining buffered records
to MongoDB and propagates exceptions if the write fails. When a task has
already failed due to a MongoDB write error (e.g., duplicate key, document
validation failure, or server unavailability), the subsequent flush during
close() may encounter the same error again.
With Flink 2.2, this cleanup exception can cause the TaskManager to terminate
instead of allowing the task failure to be handled through the normal recovery
mechanism.
This introduces a behavioral regression compared to Flink 1.19/1.20. Although
no released version is currently affected, the issue needs to be addressed
before the MongoDB connector's first Flink 2.x release.
*Expected Behavior*
{{MongoWriter.close()}} should handle failures during its final flush without
propagating exceptions that could terminate the TaskManager.
Such failures should be logged at ERROR level to ensure they remain visible for
diagnostics.
The behavior should also account for the configured delivery guarantee:
* {{{}AT_LEAST_ONCE{}}}: Records are flushed during checkpointing and normal
end-of-input processing. Records remaining after a failed or cancelled task can
be replayed from the last successful checkpoint.
* {{{}NONE{}}}: The final flush during {{close()}} may be the only attempt to
write the remaining buffered records, particularly when stopping with a
savepoint without draining. Suppressing an exception in this case may result in
record loss, which must be clearly documented and logged.
*Proposed Changes*
# Update {{MongoWriter.close()}} to catch and log MongoDB write exceptions
without rethrowing them during cleanup.
# Document the implications for different delivery guarantees.
# Add an integration test in {{MongoWriterITCase}} that triggers a MongoDB
write failure and verifies that {{close()}} returns normally without
propagating the exception.
# Add a release note describing the change in cleanup failure handling.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)