sunchao commented on PR #57437: URL: https://github.com/apache/spark/pull/57437#issuecomment-5062800899
Thanks @viirya, @dongjoon-hyun, and @peter-toth for the careful reviews. I have pushed `5c57963` with the following updates: - Support default-collation string projection and arbitrary non-null literal date formats, with an end-to-end `category` regression. - Preserve the original non-adaptive direct-broadcast reuse behavior; apply exact broadcast-mode and null-aware checks only to projected reuse. - Throw from `executeCollect()` for unavailable projected domains and assert that fail-open semantics are used only for dynamic partition pruning. - Add an explicit reordered-column regression; the value is bound against the broadcast output by expression ID, not assumed ordinal. - Use Spark's `ExtractEquiJoinKeys`, so ancestor broadcast joins with residual conditions can still reuse their actual ordered hash keys. The four settings remain intentionally internal and default-off; this PR does not introduce a public configuration or API. I replied separately to the metadata-design thread because lazy re-derivation versus a transient second parameter list is the remaining design decision. Validation for the updated commit: **279 tests passed** across the Catalyst suite, all six adaptive/non-adaptive V1/V2 dynamic-pruning variants, and the iterative V2 connector suite. All four Catalyst/SQL Scala style checks also passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
