xloya opened a new pull request #3977: URL: https://github.com/apache/iceberg/pull/3977
When we write to the partition table, if an executor needs to write a very large number of partitions, and the map's size of writers is not limited, it will eventually lead to OOM. We have had this problem in production environment. This pr changes the map of storing writers to caffine cache, providing two conditions of chche size and expire time for writers to expire useless writers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
