leaves12138 opened a new pull request, #7778: URL: https://github.com/apache/paimon/pull/7778
### What changed - Parallelize manifest reads during manifest full compaction using the existing `ManifestReadThreadPool` and `scan.manifest.parallelism`. - Keep compaction output ordering stable by reading in parallel batches and writing results sequentially. - Add a regression test that blocks manifest reads until two readers enter concurrently. ### Why Commit full compaction previously read each manifest file serially in the second read pass. On object storage or Jindo-backed file systems, one slow manifest read could hold the global committer for a long time. This change lets the existing manifest read parallelism speed up that IO-bound phase. ### Validation - `mvn -pl paimon-core -Pfast-build -Dtest=ManifestFileMetaTest test` - `mvn -pl paimon-core -DskipTests compile` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
