Hi Robson,

Thank you for the detailed bug report.

I believe the behavior that you're describing is caused by this flaw:
https://issues.apache.org/jira/browse/KAFKA-15090 which is still under
discussion. Since the above flaw was introduced in 3.0, source
connectors need to return from poll() before the graceful shutdown
timeout to avoid the error.
You may be able to mitigate the error if the connector allows you to
reduce its poll timeout/interval to something less than the graceful
timeout, but that will depend on the specific connector
implementation, so check the documentation for your connector. I know
some implementations have received patches to compensate for this
behavior in the framework, so also consider upgrading or checking
release notes for your connectors.

As for the effects of this error: whenever a non-graceful stop occurs,
the runtime will immediately close the producer so that the task will
not be able to write any further records. However, it will still leave
resources for that task (threads, in-memory records, database
connections, etc) allocated, until the task does finally return from
poll(). While this is not desirable behavior, it seems to be treated
as just a nuisance error by most operators.

I hope this gives some context for the error message you're seeing.

Thanks,
Greg

Reply via email to