carloea2 opened a new issue, #8181: URL: https://github.com/apache/texera/issues/8181
### Root cause Java string hashing iterates UTF-16 code units, while Python tuple hashing iterates Unicode code points. These representations differ for supplementary characters, so equal string keys can choose different hash-shuffle workers in Python and Scala. ### Reproduction Use two receivers and the string key represented by `U+1F600`. Before: Python hash 128543 selects worker 1, Scala hash 1772930 selects worker 0 After: both hashes are 1772930 and both select worker 0 ASCII, BMP Unicode, and supplementary Unicode are covered. ### Expected behavior Python string tuple hashes use Java UTF-16 code-unit semantics. ### Version Current `main` at 70c21145887920528d7d5540e3fb790b43e8b759. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
