Heltman commented on issue #7843: URL: https://github.com/apache/iceberg/issues/7843#issuecomment-1598028108
@rdblue We have table with 400 partitions, every partition with 10k files, every files with 64~256MB. Total size ~500TB. Table has ~100 columns with complex types. As I see, one FileScanTask use 20KB memory, our trino coordinator have 40cores, default use 80 threads load to this queue. Which means at least we used 15GB (80 x 10k x 20KB) memory in this queue. When we has 4 queries, coordinator will oom. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
