rdblue commented on issue #4666: URL: https://github.com/apache/iceberg/issues/4666#issuecomment-1113714627
@prodeezy, I think I know what's happening. The `RuntimeException` catch block actually suppresses the exception, which prevents _Spark_ from running cleanup. But OOM is not a `RuntimeException` so it propagates and triggers Spark to remove the data files it wrote. I think the solution is to catch and suppress any `Throwable` during cleanup, although that is pretty ugly! We should also move the notification into the block so that if a listener fails we don't have this situation either. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
