driessamyn-sparta commented on PR #29676: URL: https://github.com/apache/flink/pull/29676#issuecomment-6603305948
> For me the only valid thing to do is 2. in one way or another. > > The easiest thing would be to fix this in `flink-connector-kafka` somehow, I don't know how. Maybe it is as simple, as just cleaning async exception after it has been thrown to the task thread at least once? It's probably not a full solution, but maybe a good enough. Probably best to bring this up with `flink-connector-kafka` maintainers. > > I could also see this requiring us to change the closing procedures of the operators. For example instead of throwing from `#close`, return status: > > ``` > OK > FAILED > FAILED_BUT_CLEANED_UP > ``` > > ? > > On the other hand, if failure happens inside `#close`, after `#notifyCheckpointComplete` or `#notifyCheckpointAborted` calls, it's hard to argue that throwing async exceptions from `#close` in the EOS connector makes any sense. Maybe we should swallow all exceptions except of clean up exceptions. Failing subtask might not make sense at this stage for any other reasons. Transactions have already been aborted or committed either way. @MartijnVisser I can see you are a flink-kafka-connector contributor too. What is your view on this? I am happy to create a PR on that project instead or as well, but looking for guidance on what direction we think is best. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
