zahed1994 commented on PR #59095: URL: https://github.com/apache/spark/pull/59095#issuecomment-5871255273
Thanks for the sharp catch, @uros-b - that is a subtle and critical edge case. I have updated the PR accordingly: 1. **Value side (`ctagV`)**: Reverted unconditionally to `fakeClassTag`. Because `HashPartitioner` is fundamentally a key-only guard, inferring value types was unnecessary and risked `ArrayStoreException` when collecting heterogeneous values. 2. **Key side (`ctagK`)**: Gated the specialization strictly to array types (`isArray`). For any non-array or heterogeneous keys (e.g. `Number` with `Integer` and `Double`), it falls back to `fakeClassTag`, ensuring `keys().collect()` continues allocating `Object[]` as before without `ArrayStoreException`. 3. **Tests**: Added a regression test in `JavaAPISuite` covering heterogeneous keys and values to ensure neither throws `ArrayStoreException`, while array keys under `HashPartitioner` continue to be blocked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
