pkubaj opened a new pull request, #327: URL: https://github.com/apache/datasketches-rust/pull/327
The crate refuses to build for big-endian targets (#28, #31) because the Java library requires a little-endian native byte order. The Rust code does not need that for serialization: SketchBytes and SketchSlice read and write with an explicit byte order, and the crate has no native-endian loads or stores. The one place where the host byte order leaked in was hashing. Both MurmurHash3X64128 and XxHash64 implement only Hasher::write, so the default write_u16/u32/u64/u128/usize methods (and the signed variants, which forward to them) hash the native-endian bytes of the integer. On a big-endian host every integer item therefore hashes to a different value than on a little-endian host or in the Java, C++ and Go implementations. With only the compile_error removed, 17 integration tests fail on FreeBSD powerpc64 (big-endian): the bloom, CPC, HLL and XOR serialization compatibility tests against the Java, C++ and Go reference images, several theta set-operation tests and three HLL tests. Override the fixed-width Hasher methods in both hashers to hash the little-endian representation. This changes nothing on little-endian targets. With it, `cargo x test` passes on powerpc64 with the same results as on powerpc64le, including all serialization tests against the TCK snapshots, and stable clippy with -D warnings is clean. Drop the compile_error and the README note, and add unit tests that pin the contract: hashing an integer equals hashing its little-endian bytes. Co-Authored-By: Claude Opus 5.5 <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
