rekbun opened a new pull request, #59970: URL: https://github.com/apache/spark/pull/59970
### What changes were proposed in this pull request? Add a note to the "Upgrading from Spark SQL 4.1 to 4.2" section of the SQL migration guide for SPARK-55092: since Spark 4.2, V2 scans no longer group input splits that share a reported partition key into one task unless an operator needs that grouping. ### Why are the changes needed? SPARK-55092 changed this behavior in Spark 4.2 without a migration note. Connectors that report many small splits per key can see many more scan tasks after upgrading. Raised in the review of #59522. This should also be merged to `branch-4.2` and `branch-4.x`; the patch applies cleanly to `branch-4.2`. ### Does this PR introduce _any_ user-facing change? No. Documentation only. ### How was this patch tested? Documentation only. The new bullet is part of the existing list, with no blank line after it. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5.5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
