scovich opened a new pull request, #8481: URL: https://github.com/apache/arrow-rs/pull/8481
# Which issue does this PR close? - Closes https://github.com/apache/arrow-rs/issues/8336 - Closes https://github.com/apache/arrow-rs/issues/8334 # Rationale for this change The `VariantArray::value` method was really inefficient but had quietly crept into important code paths like `variant_get`. Since the complex and inefficient code was in support of variant unshredding, we should just add and use a proper `unshred_variant` function (which uses row builders like the other variant manipulating functions). # What changes are included in this PR? * Define the new `unshred_variant` function, which does what it says. It supports all the types `typed_value_to_variant` supported, plus Time64 and Struct as a bonus. The former because it was ~10LoC and the latter because it demonstrates the superiority of this new approach vs. e.g. https://github.com/apache/arrow-rs/pull/8446 * Wire up `variant_get` unshredding path to call it, which immediately benefits from all that function's existing test coverage. * Define a new `ShreddingState::extract_fields_from` method that allows borrowing the extracted value and typed_value columns instead of cloning them. * Define a new `VariantValueArrayBuilder::as_builder_ext` method (with corresponding `VariantValueArrayBuilderExt`) which makes it easier to work with value builders. * Update the variant_integration test to `unshred_variant` instead of looping over rows calling `value(i)`. # Are these changes tested? Yes, a bunch of variant integration tests now pass that used to fail. # Are there any user-facing changes? Several new pub methods. I don't think any breaking changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
