Neuw84 commented on issue #17241: URL: https://github.com/apache/iceberg/issues/17241#issuecomment-5014895843
As I know you where interested @RussellSpitzer, just take a look, took me a while as I did many experiments to track this down. #17296 Fixes it. #17281 Driver optimization, maybe it is not worthy as will add some memory overhead on the driver ( but in long jobs will make sense). #17287 Optimizations on on-heap memory while writing DV, didn´t solve the issue ( as we had off-heap leak), but I think that is worthy to have ( same throughput in tests). Right now the job has been running for more than 6 hours without any failure. <img width="770" height="191" alt="Image" src="https://github.com/user-attachments/assets/3e99ac08-0411-4736-bf77-3ccb263b9f1e"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
