zhangjun0x01 opened a new issue #1660: URL: https://github.com/apache/iceberg/issues/1660
The current default parallelism of RewriteDataFilesAction is the number of CombinedScanTask tasks, but if there is a lot of data in the partition, the task size will be very large, which will occupy a lot of system resources and affect the running tasks. I think we should set The max parallelism of RewriteDataFilesAction to avoid taking up too much system resources ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
