RussellSpitzer opened a new pull request #1362: URL: https://github.com/apache/iceberg/pull/1362
Because we use LocalIterator in ExpireSnapshotAction, every partition runs it's own spark job, almost all of which are completely empty. This leads to a lot of overhead which we don't need in the Test Suite. Setting shuffle parallelism to 1 (from 200) greatly reduces the test runtime. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
