jayzhan211 opened a new issue, #10906: URL: https://github.com/apache/arrow-rs/issues/10906
### Is your feature request related to a problem or challenge? variant_kernels.rs benchmarks json_to_variant and variant_get, but nothing covers the other direction, so changes to variant_to_json perf go unnoticed. #10810 rewrote it on top of VariantArray so it could handle shredded input and dictionary/run-end-encoded metadata. Good change, but the row loop now goes through try_value(i), which re-matches the data type and re-downcasts both the metadata and value columns on every row — the old loop hoisted those out. On my machine (M4 Pro) that's ~1.4x slower on batches of scalars, where there's barely any JSON to write and the dispatch dominates, and ~1.1x on objects. ### Describe the solution you'd like Two things, and the first is worth doing on its own: Add variant_to_json benchmarks to variant_kernels.rs — scalars, small objects, wide objects, plus a shredded array and one with dictionary metadata. Use them to decide how far to take the fast path. Most of the regression comes back by resolving the column dispatch once and keeping try_value only as the fallback for shredded and encoded-metadata arrays — that gets to ~1.05x. The rest is a per-row enum match that would need monomorphizing over the concrete array types, which may or may not be worth the code. ### Describe alternatives you've considered _No response_ ### Additional context Whatever lands needs to keep try_value's edge cases: a null value in an unshredded row is Variant::Null rather than an error, and a null metadata row that isn't masked by the struct's null buffer must stay an error instead of feeding garbage to Variant::new. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
