This is an automated email from the ASF dual-hosted git repository.
leerho pushed a commit to branch docs/count-min-format-v2
in repository https://gitbox.apache.org/repos/asf/datasketches-tck.git
The following commit(s) were added to refs/heads/docs/count-min-format-v2 by
this push:
new 31edd94 docs: Count-Min v2, add the cross-language test files section
31edd94 is described below
commit 31edd941aa9b50cbe7e26e1d5554b9e2c0082e29
Author: Lee Rhodes <[email protected]>
AuthorDate: Tue Nov 10 16:56:36 2026 -0800
docs: Count-Min v2, add the cross-language test files section
Section 8 defines the version 2 CL files every implementation generates
and checks: names (count_min_v2_<case>_<language>.sk), contents for each
case, the checks a reader performs (including byte-identical images
across languages), and how to treat the existing C++ version 1 files.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
docs/count-min-format.md | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/docs/count-min-format.md b/docs/count-min-format.md
index 3e7f68a..2d7a48d 100644
--- a/docs/count-min-format.md
+++ b/docs/count-min-format.md
@@ -210,3 +210,44 @@ After `update(1, 10)`, `update(9, 810)` and
`update("apple", 5)` (168 bytes). Ro
```
Reading this image back with seed 9001 gives estimates of 10 for 1, 810 for 9
and 5 for `"apple"`.
+
+## 8. Cross-language test files
+
+Each implementation generates the files below into its
`serialization_test_data/<language>_generated_files` directory, as described in
[Serialization snapshots](serialization-snapshots.md), and checks the files
generated by the other languages.
+
+### 8.1 Names and contents
+
+File names have the form `count_min_v2_<case>_<language>.sk`, where
`<language>` is the implementation's snapshot suffix (for example `cpp`, `go`,
`java` or `rust`).
+
+Every file uses `numHashes` = 3 and `numBuckets` = 1000.
+
+| Case | Seed | `N` |
+|---|---:|---:|
+| `n0` | 9001 | 0 (empty) |
+| `n1` | 9001 | 1 |
+| `n10` | 9001 | 10 |
+| `n100` | 9001 | 100 |
+| `n1000` | 9001 | 1000 |
+| `n10000` | 9001 | 10000 |
+| `seed42_n1000` | 42 | 1000 |
+
+For each `i` from 0 to `N - 1`, the sketch is updated twice:
+- with the 64-bit integer `i` and weight `i + 1`;
+- with the string of `i` in decimal (`"0"`, `"1"`, ..., with no sign, padding
or separators) and weight 1.
+
+The total weight is therefore `N × (N + 1) / 2 + N`.
+
+Version 2 is deterministic, so a file with a given case must be
**byte-identical** in every language. For example, `count_min_v2_n1000_cpp.sk`
and `count_min_v2_n1000_java.sk` contain the same bytes.
+
+### 8.2 Checks
+
+A reader checks each file from another language as follows:
+- It deserializes with the file's seed and reports serial version 2,
`numHashes` 3 and `numBuckets` 1000.
+- It is empty if and only if `N` = 0.
+- The total weight is `N × (N + 1) / 2 + N`.
+- For each `i` from 0 to `N - 1`, the estimate for integer `i` is at least `i
+ 1`, and the estimate for the string of `i` is at least 1. All weights are
positive, so a Count-Min sketch never underestimates. Each estimate is also at
most its upper bound.
+- If the reader's own language has generated the same case, the two files are
equal byte for byte. This is the strongest check, because it covers the row
seeds, item encoding and bucket mapping at once.
+
+### 8.3 Version 1 files
+
+The existing files `count_min_empty_cpp.sk` and `count_min_non_empty_cpp.sk`
are version 1 images. Readers may check that they deserialize as version 1
sketches, but shouldn't compare their estimates across languages, because
version 1 hashing differs between languages. Version 2 files always use the
`count_min_v2_` prefix, so these names are never reused for version 2 images.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]