proost commented on PR #163:
URL: https://github.com/apache/datasketches-go/pull/163#issuecomment-5150776059
@tisonkun
Each implementation derives the per-row Murmur3 hash seeds from the
user-provided sketch seed using its language-specific random number generator.
Java:
```java
final Random rand = new Random(seed);
for (int i = 0; i < numHashes; i++) {
hashSeeds_[i] = rand.nextLong();
}
```
C++:
```cpp
std::default_random_engine rng(_seed);
std::uniform_int_distribution<uint64_t> extra_hash_seeds(
0, std::numeric_limits<uint64_t>::max());
for (uint64_t i = 0; i < num_hashes; ++i) {
hash_seeds.push_back(extra_hash_seeds(rng) + _seed);
}
```
Go:
```go
rng := rand.New(rand.NewSource(seed))
hashSeeds := make([]int64, numHashes)
for i := range int(numHashes) {
hashSeeds[i] = int64(rng.Int()) + seed
}
```
The serialized form contains the counter array and a hash of the
user-provided seed, but it does not contain the generated per-row `hashSeeds`.
During deserialization, each implementation reconstructs `hashSeeds` from
the provided seed using its own random number generator. Because these random
number generators do not produce the same sequence across Java, C++, and Go,
the reconstructed Murmur3 seeds can differ even when the same input seed is
provided.
For example:
1. A C++ sketch updates item `x`, placing its counts into buckets selected
using the C++-generated `hashSeeds`.
2. Go successfully deserializes the C++ counter array.
3. Go regenerates different `hashSeeds`.
4. Calling `GetEstimate(x)` in Go may inspect different buckets from those
updated by C++.
Therefore, the binary format can be deserialized across languages, but we
cannot currently guarantee that queries, subsequent updates, or merges behave
correctly across language boundaries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
