ayush-san commented on issue #2900: URL: https://github.com/apache/iceberg/issues/2900#issuecomment-894014352
> are tables partitioned by event time or ingestion time? or some other partition spec that can make writers append to multiple files concurrently? These are unpartitioned tables > do you run multiple writer tasks/threads in the same core? E.g., assuming your DB has 10 tables, does each slot/core run 10 writers or only 1 writer? If each taskmanager has too many writer tasks, memory consumption by Parquet is going to be a problem. Currently, all the tables are sharing the same taskmanager and each has its own writer  > If each taskmanager has too many writer tasks, memory consumption by Parquet is going to be a problem. @stevenzwu Can you please give more detail on this issue? Why is this causing the problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
