xndai opened a new pull request, #1126:
URL: https://github.com/apache/iceberg-cpp/pull/1126

   Closes #1125. Part of #1107.
   
   Adds an internal reader for the [Mumbling bitmap 
format](https://github.com/apache/iceberg/blob/main/format/mumbling-spec.md) (a 
draft spec, apache/iceberg#16518), which v4 manifest deletion vectors use 
(`manifest_info.dv`, `tracking.deleted_positions` and 
`tracking.replaced_positions`). Because the spec is a draft, the header is 
`util/mumbling_bitmap_internal.h` and isn't installed.
   
   `MumblingBitmapView::Make()` wraps the serialized bytes without copying 
them. In scan planning those bytes belong to the root manifest entry, which 
outlives the planning of its leaf manifest. `Make()` checks the header, the 
PFOR-encoded descriptor array ([Appendix 
A](https://github.com/apache/iceberg/blob/main/format/mumbling-spec.md#appendix-a-pfor-encoding-for-unsigned-bytes)),
 the descriptor types, and the container sizes against the buffer, so 
`Contains()` never reads out of bounds. Like the Java reader, it doesn't 
validate container contents. Sparse-container lookups use SSE2 on x86-64 and a 
portable SWAR search elsewhere; no compiler flags are added.
   
   Tests cover the spec examples, malformed input (including the vectors from 
Java's `TestMumblingBitmap`), and the Java-generated fixtures from 
apache/iceberg-go#1750. `mumbling_bitmap_benchmark` (built with 
`-DICEBERG_BUILD_BENCHMARKS=ON`) measures decoding and lookups.
   
   Benchmark against the Java reader on the same input bytes (Release; gcc-14 
with Google Benchmark, JDK 17 with JMH; Xeon Platinum 8175M):
   
   | Operation | C++ | Java |
   |---|---:|---:|
   | Decode 8,192 PFOR-encoded descriptors | 0.35–3.1 µs | 1.3–12.4 µs |
   | `Make`, 100,000 random positions | 13.3 µs | 29.2 µs |
   | `Make`, all 2,097,152 positions | 9.4 µs | 14.2 µs |
   | `Contains`, 4,096 probes, 100,000 random positions | 23.7 µs | 83.5 µs |
   | `Contains`, 4,096 probes, all positions | 14.6 µs | 24.1 µs |
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to