GitHub user tisonkun added a comment to the discussion: Count-Min sketch is not compatible across languages, or across C++ standard libraries
Thanks for the detailed analysis and feedback. I’ve opened a draft Rust PR to make the proposed direction concrete: https://github.com/apache/datasketches-rust/pull/299 The draft currently: - writes new sketches as serial version 2; - derives `rowSeed[i] = XxHash64(i as an 8-byte little-endian value, seed = baseSeed)`; - uses XxHash64 for item hashing as suggested in the discussion; - maps item hashes to buckets with `(hash >> 1) % numBuckets` on an unsigned 64-bit value; - retains the legacy Rust derivation and mapping for version 1 images, and serializes them back as version 1 so existing counters are not mislabeled as using the new scheme; - rejects merges between sketches with different serial versions. The tests include independently computed XxHash64 known-answer vectors for default and custom seeds, plus legacy Rust v1 coverage for estimates, updates after deserialization, merges, and byte round-trips. Existing C++ v1 snapshots still pass deserialization and byte round-trip tests; we continue to document that cross-language v1 queries, updates, and merges are not reliable. I’d especially like confirmation on two details before moving this beyond a draft: 1. Is the row-seed formula above exactly the intended construction, with no additional `+ baseSeed` after hashing? 2. Should version 2 switch item hashing to XxHash64 at the same time, as suggested in the discussion, so that changing the item hash later does not require another serial version? GitHub link: https://github.com/apache/datasketches-tck/discussions/18#discussioncomment-19117953 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
