JingsongLi opened a new pull request, #9967: URL: https://github.com/apache/paimon/pull/9967
### Purpose Snapshot expiration previously built a full data-file index for every referenced tag. On large tables, concurrent tag reads retained memory proportional to the number of tags multiplied by all table files and could cause OOM. This change: - plans deletion candidates in bounded expiration batches; - prunes tag manifests by candidate bucket ranges; - pushes partition and bucket filtering into Avro decoding before nested file metadata; - filters file names on a reusable projected entry before materializing ExpireFileEntry; - keeps only matching tag entries while preserving changelog-decoupled cleanup behavior. The filename filter is streaming rather than storage-index based; manifests still need to be scanned within matching partitions and buckets. ### Tests - Targeted manifest streaming/filter-pushdown and tag expiration regression tests - ExpireSnapshotsTest and FileDeletionTest - mvn -pl paimon-core -DskipTests spotless:check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
