This is an automated email from the ASF dual-hosted git repository.

leerho pushed a commit to branch docs/count-min-format-v2
in repository https://gitbox.apache.org/repos/asf/datasketches-tck.git


The following commit(s) were added to refs/heads/docs/count-min-format-v2 by 
this push:
     new 31edd94  docs: Count-Min v2, add the cross-language test files section
31edd94 is described below

commit 31edd941aa9b50cbe7e26e1d5554b9e2c0082e29
Author: Lee Rhodes <[email protected]>
AuthorDate: Tue Nov 10 16:56:36 2026 -0800

    docs: Count-Min v2, add the cross-language test files section
    
    Section 8 defines the version 2 CL files every implementation generates
    and checks: names (count_min_v2_<case>_<language>.sk), contents for each
    case, the checks a reader performs (including byte-identical images
    across languages), and how to treat the existing C++ version 1 files.
    
    Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
 docs/count-min-format.md | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/docs/count-min-format.md b/docs/count-min-format.md
index 3e7f68a..2d7a48d 100644
--- a/docs/count-min-format.md
+++ b/docs/count-min-format.md
@@ -210,3 +210,44 @@ After `update(1, 10)`, `update(9, 810)` and 
`update("apple", 5)` (168 bytes). Ro
 ```
 
 Reading this image back with seed 9001 gives estimates of 10 for 1, 810 for 9 
and 5 for `"apple"`.
+
+## 8. Cross-language test files
+
+Each implementation generates the files below into its 
`serialization_test_data/<language>_generated_files` directory, as described in 
[Serialization snapshots](serialization-snapshots.md), and checks the files 
generated by the other languages.
+
+### 8.1 Names and contents
+
+File names have the form `count_min_v2_<case>_<language>.sk`, where 
`<language>` is the implementation's snapshot suffix (for example `cpp`, `go`, 
`java` or `rust`).
+
+Every file uses `numHashes` = 3 and `numBuckets` = 1000.
+
+| Case | Seed | `N` |
+|---|---:|---:|
+| `n0` | 9001 | 0 (empty) |
+| `n1` | 9001 | 1 |
+| `n10` | 9001 | 10 |
+| `n100` | 9001 | 100 |
+| `n1000` | 9001 | 1000 |
+| `n10000` | 9001 | 10000 |
+| `seed42_n1000` | 42 | 1000 |
+
+For each `i` from 0 to `N - 1`, the sketch is updated twice:
+- with the 64-bit integer `i` and weight `i + 1`;
+- with the string of `i` in decimal (`"0"`, `"1"`, ..., with no sign, padding 
or separators) and weight 1.
+
+The total weight is therefore `N × (N + 1) / 2 + N`.
+
+Version 2 is deterministic, so a file with a given case must be 
**byte-identical** in every language. For example, `count_min_v2_n1000_cpp.sk` 
and `count_min_v2_n1000_java.sk` contain the same bytes.
+
+### 8.2 Checks
+
+A reader checks each file from another language as follows:
+- It deserializes with the file's seed and reports serial version 2, 
`numHashes` 3 and `numBuckets` 1000.
+- It is empty if and only if `N` = 0.
+- The total weight is `N × (N + 1) / 2 + N`.
+- For each `i` from 0 to `N - 1`, the estimate for integer `i` is at least `i 
+ 1`, and the estimate for the string of `i` is at least 1. All weights are 
positive, so a Count-Min sketch never underestimates. Each estimate is also at 
most its upper bound.
+- If the reader's own language has generated the same case, the two files are 
equal byte for byte. This is the strongest check, because it covers the row 
seeds, item encoding and bucket mapping at once.
+
+### 8.3 Version 1 files
+
+The existing files `count_min_empty_cpp.sk` and `count_min_non_empty_cpp.sk` 
are version 1 images. Readers may check that they deserialize as version 1 
sketches, but shouldn't compare their estimates across languages, because 
version 1 hashing differs between languages. Version 2 files always use the 
`count_min_v2_` prefix, so these names are never reused for version 2 images.


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to