tisonkun commented on PR #329:
URL:
https://github.com/apache/datasketches-rust/pull/329#issuecomment-6661308636
Retain the cached segment length: deriving `fingerprints.len() / 3` on every
lookup adds work to the query loop. The final change derives it once in
`from_parts(seed, fingerprints, num_items)`, removing the redundant import
argument.
Before: `cb079f874dd346bb0ecd1042dde0c9cb54f1f29b`. Final:
`6faf16148b10fab8363d5b6bb7a78f11acb68e70`. Both use optimized Cargo bench
profiles. The per-query-derived variant was measured but is not included in the
PR.
For `xor::query_shards::8_present/128` (32 filters, 256 lookups per filter,
20,000 rounds):
| Variant | Profile | Before ms | After ms | Time
change |
| ---------------------- | ---------------- | --------: | -------: |
----------: |
| Compute on each query | Ordinary release | 263.7 | 278.3 | +5.6%
|
| Compute on each query | ThinLTO | 263.3 | 277.3 | +5.3%
|
| Final: cache on import | Ordinary release | 267.7 | 270.7 | +1.1%
|
| Final: cache on import | ThinLTO | 276.0 | 280.3 | +1.6%
|
The final multi-filter query loops match the baseline instructions apart
from addresses. Measurements still include host noise: ordinary-release
`8_absent/16384` has a 591 ms sample, and `8_absent/50000` has large samples on
both revisions. These rows are inconclusive; no samples were dropped. This
comparison measures this library change, not end-to-end application performance.
<details>
<summary>All 45 final workloads: arithmetic means and all three samples, in
milliseconds</summary>
Apple M4 Max, macOS 26.6.2, aarch64, rustc 1.100.0-nightly (`e457a7b0d`,
nightly-2026-08-28). Ordinary bench profile and the same profile with
`CARGO_PROFILE_BENCH_LTO=thin`. No concurrent builds or tests during
measurement. Fixed inputs and registered round counts are identical across
revisions. Each row ran in before/after/after/before/before/after order, with
setup and cleanup included. Each cell is `mean [samples in measurement order]`;
positive time change means slower.
Build each revision with:
```sh
cargo +nightly-2026-08-28 bench -p benchmarks --bench benchmarks --no-run
CARGO_PROFILE_BENCH_LTO=thin cargo +nightly-2026-08-28 bench -p benchmarks
--bench benchmarks --no-run
```
Invoke each saved binary for each registered XOR scenario with `--exact
--baseline-only --repeat 1 --raw`, interleaving revisions as above.
| Workload | Release before | Release final
| Time change | ThinLTO before | ThinLTO final | Time
change |
| -------------------------------- | --------------------: |
--------------------: | ----------: | --------------------: |
--------------------: | ----------: |
| `build::8_distinct/1000` | 307.3 [323, 303, 296] | 305.7 [307,
308, 302] | -0.5% | 299.0 [299, 297, 301] | 300.3 [301, 301, 299] | +0.4%
|
| `build::8_deduplicate/1000` | 346.3 [349, 345, 345] | 354.3 [358,
352, 353] | +2.3% | 350.3 [346, 351, 354] | 352.0 [348, 353, 355] | +0.5%
|
| `build::16_distinct/1000` | 308.7 [313, 305, 308] | 306.0 [307,
305, 306] | -0.9% | 302.3 [303, 302, 302] | 301.3 [303, 299, 302] | -0.3%
|
| `build::16_deduplicate/1000` | 356.0 [359, 353, 356] | 358.3 [359,
355, 361] | +0.7% | 356.7 [362, 357, 351] | 358.0 [360, 361, 353] | +0.4%
|
| `build::32_distinct/1000` | 308.7 [310, 310, 306] | 315.3 [316,
318, 312] | +2.2% | 302.0 [302, 304, 300] | 313.3 [314, 313, 313] | +3.8%
|
| `build::32_deduplicate/1000` | 356.0 [356, 352, 360] | 353.0 [355,
350, 354] | -0.8% | 356.0 [355, 362, 351] | 356.3 [358, 353, 358] | +0.1%
|
| `build::8_distinct/16384` | 358.7 [359, 360, 357] | 358.0 [356,
359, 359] | -0.2% | 358.7 [357, 356, 363] | 360.7 [355, 358, 369] | +0.6%
|
| `build::8_deduplicate/16384` | 249.0 [243, 251, 253] | 247.7 [250,
244, 249] | -0.5% | 247.7 [255, 243, 245] | 257.0 [273, 245, 253] | +3.8%
|
| `build::16_distinct/16384` | 355.7 [358, 355, 354] | 361.7 [368,
360, 357] | +1.7% | 357.0 [362, 355, 354] | 355.3 [356, 353, 357] | -0.5%
|
| `build::16_deduplicate/16384` | 242.7 [242, 241, 245] | 245.0 [245,
242, 248] | +1.0% | 244.0 [244, 245, 243] | 244.3 [244, 245, 244] | +0.1%
|
| `build::32_distinct/16384` | 358.3 [360, 358, 357] | 362.3 [361,
370, 356] | +1.1% | 368.0 [370, 367, 367] | 371.3 [372, 376, 366] | +0.9%
|
| `build::32_deduplicate/16384` | 244.7 [244, 244, 246] | 243.7 [242,
244, 245] | -0.4% | 251.3 [254, 252, 248] | 250.3 [250, 252, 249] | -0.4%
|
| `build::8_distinct/1000000` | 153.3 [154, 153, 153] | 152.0 [152,
152, 152] | -0.9% | 153.7 [156, 151, 154] | 154.7 [153, 151, 160] | +0.7%
|
| `build::8_deduplicate/1000000` | 149.3 [148, 150, 150] | 147.0 [147,
147, 147] | -1.6% | 153.7 [150, 156, 155] | 151.0 [154, 148, 151] | -1.7%
|
| `build::16_distinct/1000000` | 150.0 [150, 149, 151] | 154.0 [151,
151, 160] | +2.7% | 155.7 [152, 157, 158] | 167.3 [173, 163, 166] | +7.5%
|
| `build::16_deduplicate/1000000` | 148.7 [149, 149, 148] | 149.7 [150,
150, 149] | +0.7% | 156.0 [154, 154, 160] | 154.7 [153, 152, 159] | -0.9%
|
| `build::32_distinct/1000000` | 155.0 [151, 156, 158] | 159.7 [157,
158, 164] | +3.0% | 159.0 [163, 158, 156] | 158.0 [162, 157, 155] | -0.6%
|
| `build::32_deduplicate/1000000` | 152.7 [153, 153, 152] | 150.7 [150,
153, 149] | -1.3% | 152.3 [151, 150, 156] | 153.3 [150, 150, 160] | +0.7%
|
| `build_batches::8/128` | 264.7 [264, 263, 267] | 260.3 [262,
257, 262] | -1.6% | 262.3 [255, 269, 263] | 266.0 [262, 264, 272] | +1.4%
|
| `build_batches::8/16384` | 138.7 [139, 139, 138] | 139.7 [139,
140, 140] | +0.7% | 141.0 [142, 141, 140] | 143.0 [145, 143, 141] | +1.4%
|
| `build_batches::8/50000` | 164.3 [164, 165, 164] | 164.7 [165,
165, 164] | +0.2% | 169.7 [166, 175, 168] | 174.7 [167, 188, 169] | +2.9%
|
| `query::8_present/16384` | 253.0 [255, 252, 252] | 255.0 [256,
255, 254] | +0.8% | 262.3 [260, 259, 268] | 260.3 [259, 258, 264] | -0.8%
|
| `query::8_present/50000` | 236.7 [228, 254, 228] | 231.3 [230,
232, 232] | -2.3% | 236.0 [234, 237, 237] | 238.0 [244, 231, 239] | +0.8%
|
| `query::8_absent/16384` | 270.3 [257, 247, 307] | 363.3 [252,
247, 591] | +34.4% | 254.7 [256, 253, 255] | 254.7 [255, 256, 253] | +0.0%
|
| `query::8_absent/50000` | 306.0 [461, 231, 226] | 393.7 [470,
485, 226] | +28.6% | 230.0 [236, 229, 225] | 230.3 [234, 230, 227] | +0.1%
|
| `query::16_present/16384` | 260.0 [257, 260, 263] | 261.7 [259,
263, 263] | +0.6% | 274.3 [261, 259, 303] | 262.0 [260, 259, 267] | -4.5%
|
| `query::16_present/50000` | 267.3 [301, 250, 251] | 251.3 [255,
248, 251] | -6.0% | 251.3 [248, 252, 254] | 251.3 [259, 248, 247] | +0.0%
|
| `query::16_absent/16384` | 251.3 [254, 250, 250] | 250.3 [251,
250, 250] | -0.4% | 256.0 [252, 256, 260] | 258.3 [262, 253, 260] | +0.9%
|
| `query::16_absent/50000` | 240.7 [240, 242, 240] | 244.0 [241,
241, 250] | +1.4% | 246.7 [243, 249, 248] | 247.0 [249, 247, 245] | +0.1%
|
| `query::32_present/16384` | 251.0 [252, 252, 249] | 250.0 [254,
250, 246] | -0.4% | 250.0 [253, 248, 249] | 250.7 [252, 253, 247] | +0.3%
|
| `query::32_present/50000` | 260.7 [260, 262, 260] | 265.0 [270,
268, 257] | +1.7% | 262.3 [260, 262, 265] | 261.3 [261, 262, 261] | -0.4%
|
| `query::32_absent/16384` | 249.3 [247, 249, 252] | 254.0 [257,
253, 252] | +1.9% | 249.7 [250, 249, 250] | 247.0 [246, 246, 249] | -1.1%
|
| `query::32_absent/50000` | 271.0 [265, 261, 287] | 262.3 [262,
260, 265] | -3.2% | 262.7 [263, 262, 263] | 267.0 [269, 265, 267] | +1.6%
|
| `query_shards::8_present/128` | 267.7 [267, 267, 269] | 270.7 [266,
272, 274] | +1.1% | 276.0 [274, 276, 278] | 280.3 [280, 280, 281] | +1.6%
|
| `query_shards::8_absent/128` | 268.0 [269, 269, 266] | 268.0 [266,
266, 272] | +0.0% | 284.0 [276, 301, 275] | 276.7 [281, 279, 270] | -2.6%
|
| `query_shards::8_present/16384` | 362.3 [383, 320, 384] | 350.3 [387,
344, 320] | -3.3% | 358.0 [400, 341, 333] | 338.3 [346, 332, 337] | -5.5%
|
| `query_shards::8_absent/16384` | 361.0 [384, 322, 377] | 340.3 [378,
321, 322] | -5.7% | 373.0 [398, 394, 327] | 346.3 [328, 381, 330] | -7.1%
|
| `query_shards::16_present/128` | 267.0 [271, 265, 265] | 266.0 [266,
265, 267] | -0.4% | 278.0 [282, 275, 277] | 276.3 [274, 275, 280] | -0.6%
|
| `query_shards::16_absent/128` | 266.3 [266, 266, 267] | 267.3 [267,
269, 266] | +0.4% | 274.3 [279, 272, 272] | 273.3 [276, 273, 271] | -0.4%
|
| `query_shards::16_present/16384` | 330.3 [331, 329, 331] | 331.0 [333,
330, 330] | +0.2% | 340.0 [337, 339, 344] | 338.7 [335, 340, 341] | -0.4%
|
| `query_shards::16_absent/16384` | 335.7 [335, 336, 336] | 334.0 [331,
334, 337] | -0.5% | 356.0 [362, 341, 365] | 342.3 [343, 345, 339] | -3.8%
|
| `query_shards::32_present/128` | 272.7 [271, 275, 272] | 265.7 [264,
262, 271] | -2.6% | 282.0 [284, 281, 281] | 299.0 [285, 283, 329] | +6.0%
|
| `query_shards::32_absent/128` | 265.7 [271, 262, 264] | 269.0 [261,
271, 275] | +1.3% | 276.7 [280, 271, 279] | 278.0 [278, 281, 275] | +0.5%
|
| `query_shards::32_present/16384` | 337.0 [338, 333, 340] | 347.7 [351,
345, 347] | +3.2% | 349.7 [357, 347, 345] | 346.7 [343, 350, 347] | -0.9%
|
| `query_shards::32_absent/16384` | 338.0 [336, 340, 338] | 334.7 [337,
332, 335] | -1.0% | 352.0 [347, 354, 355] | 360.0 [361, 363, 356] | +2.3%
|
</details>
Validation: `cargo x check`, `cargo x test` (813 tests including doctests),
and `cargo x lint`. Existing Java and application-format serialization fixtures
remain unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
