cetra3 opened a new pull request, #11512:
URL: https://github.com/apache/arrow-rs/pull/11512
# Which issue does this PR close?
- N/A
# Rationale for this change
The existing unshredded object benchmarks
(`variant_get_unshredded_object_path_262k_rows`,
`shred_variant_*_object_8k_rows`) repeat one metadata and value buffer for
every row through `BinaryView`. Data written one record at a time has its own
metadata dictionary in each row, with different field names and different
bytes. Writers that build each row independently, such as
`VariantArrayBuilder`, produce this shape. A benchmark with shared metadata
cannot show how a change behaves on that shape. For example, it favors caches
keyed on the metadata pointer.
# What changes are included in this PR?
- A generator for objects in which every row has its own metadata
dictionary. Each row holds a field that every row has, a field that about 10%
of rows have, and about 16 other fields drawn from a pool of 300 names. The
fields are inserted in name order, so each dictionary is sorted. The values
alternate between integers and short strings.
- `variant_get_per_row_metadata_{present,sparse,absent}_64k_rows`: reads one
field as `Int32` that every row, about 10% of rows, or no row has.
- `shred_variant_per_row_metadata_8k_rows`: shreds 5 fields from the same
shape.
# Are these changes tested?
The benchmarks check the number of non-null results for each case before
timing.
# Are there any significant user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
