JingsongLi opened a new pull request, #9351: URL: https://github.com/apache/paimon/pull/9351
### Purpose Catalog-managed format table partitions can carry exact row counts, but FormatDataSplit cannot safely expose a partition count because one partition may produce multiple splits. This change attaches the pruned, scan-level row count to FormatTableScan.Plan and feeds it into Spark PaimonStatistics. Split generation and row-count aggregation reuse one catalog partition listing per plan. Unknown partition counts keep the scan row count unknown, and runtime filtering creates a fresh plan. The scan statistics allow Spark to make better cost-based decisions. A TPC-DS-style store_sales/date_dim test verifies that an analyzed dimension partition changes the physical plan from SortMergeJoin to BroadcastHashJoin and avoids fact-side shuffle. ### Tests - CatalogManagedPartitionScanTest (19 tests) - CatalogManagedPartitionAnalyzeTest (21 tests) - FormatTableTest on Spark 3.5 (21 tests) - Spotless checks for paimon-core, paimon-spark-common, and paimon-spark-ut - git diff --check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
