zhoulii opened a new issue, #9320: URL: https://github.com/apache/paimon/issues/9320
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation Partition discovery only requires a small subset of manifest fields, but `readPartitionEntries()` currently reads complete manifest entries. This causes unnecessary decoding and object allocation for fields such as key/value statistics and embedded indexes. Although partition entries are created eagerly, the scan still builds an intermediate list for each manifest before merging the partition statistics. ### Solution Introduce a projected, streaming manifest scan for partition discovery: - Project only the fields required for partition aggregation and structural filtering. - Push partition and bucket filters into the manifest reader. - Merge projected entries directly into the partition map without per-manifest intermediate lists. - Fall back to complete manifest entries when active filters require additional fields. - Preserve the existing cache path for cache-eligible manifests. This change does not modify the manifest format or partition scan semantics. ### Anything else? _No response_ ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
