lokeshj1703 opened a new pull request, #19622: URL: https://github.com/apache/hudi/pull/19622
### Describe the issue this Pull Request addresses Closes #19621 ### Summary and Changelog Clustering and bulk-insert sort keys flow through `SortUtils.getComparableSortColumns` -> `FlatLists.ofComparableArray`, which casts every sort value to `Comparable`. On the Spark record path a BINARY sort column arrives as a raw `byte[]` (not `Comparable`), so `ofComparableArray` threw `ClassCastException: [B cannot be cast to java.lang.Comparable`, failing every clustering / bulk-insert on that column. The Avro path was unaffected because `HoodieAvroUtils.getNestedFieldVal` returns a `java.nio.ByteBuffer` for the same column. This wraps `byte[]` into `ByteBuffer.wrap(...)` inside the shared `HoodieUTF8StringFactory.wrapArrayOfObjects` hook used by both `RDDCustomColumnsSortPartitioner` and `RDDBucketIndexPartitioner`. `ByteBuffer` is `Comparable` and `Serializable` (so it survives the `sortBy` shuffle) and preserves the exact byte-lexicographic ordering the Avro path already produced, so the change is behavior-preserving and needs no change to `FlatLists`. Adds `TestBulkInsertInternalPartitioner#testSortColumnsWithBinaryValueAreComparable`, which reproduces the CCE without the fix and asserts the ordering with it. ### Impact Clustering and `bulk_insert` can sort by BINARY columns on the Spark record path. No behavior change for existing (string / primitive) sort columns. ### Risk Level low Behavior-preserving: the byte-lexicographic ordering is identical to the pre-existing Avro path, and it is covered by a new unit test that fails without the change. The wrapped `ByteBuffer` sort key is shuffled by the sort partitioner exactly as the Avro binary path already shuffles it (the same `ByteBuffer` in the same `FlatLists.ComparableList`), so there is no new serialization behavior. ### Documentation Update none ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
