ulysses-you commented on PR #58522: URL: https://github.com/apache/spark/pull/58522#issuecomment-5564248263
Fixed in 973bde7e6ce. This was a regression of this change. `DataSourceV2ScanExecBase` now reports two separate views: `reportedKeyedPartitioning`, the source's own report at full key width, and `outputPartitioning`, with the pruned key positions projected away. `filteredPartitions` takes the former, and `replanWithRuntimeFilters` now takes `Option[KeyedPartitioning]` so the projected view is not passable there at all. Pinned by two new tests: a pruned leading key losing join rows, and a pruned leading `String` key throwing `ClassCastException` when read at the projected `Integer` type. Pointing the call site back at `outputPartitioning` fails exactly those two and nothing else. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
