bkirwi opened a new issue, #6063:
URL: https://github.com/apache/arrow-rs/issues/6063
**Is your feature request related to a problem or challenge? Please describe
what you are trying to do.**
I'm trying to convert back and forth between `Rows` data and raw bytes.
(Think an external sort, for example: converting to rows and then shifting the
data to off-heap storage.)
**Describe the solution you'd like**
Arrow's APIs for this are already pretty good, but there are two things that
would make my life easier.
*** A `data` accessor on `Row` ***
Right now it's impossible to write this function:
```rust
fn bytes<'a>(row: &Row<'a>) -> &'a [u8] { ... }
```
While `Row` has an `AsRef` implementation, that will only give you access to
the bytes with the lifetime of the `Row`, not the underlying `Rows` buffer.
Something like this would allow it:
```rust
impl<'a> Row<'a> {
/// The row's bytes, with the lifetime of the underlying data.
pub fn data(&self) -> &'a [u8] {
self.data
}
}
```
*** Rows to `BinaryArray`
I would love to have something like:
```rust
impl Rows {
fn into_array(self) -> BinaryArray { ... }
}
This should be fairly straightforward to implement - it can at least reuse
the allocation for the binary data - and lets the caller take advantage of all
the functionality on array. (For example, if I want to copy the data, I can do
a single memcpy instead of working row-by-row.)
**Describe alternatives you've considered**
Both of these things have workarounds - they just take extra compute or
allocations. (For example, I can emulate `rows.into_array()` with `rows.iter()`
and an `BinaryBuilder`, though that costs an extra `Vec` allocation and a bunch
of compute to do all the small copies.
A more invasive change would be to change `Rows` to be _actually_ backed by
an array, probably also adding a `RowsBuilder` that was backed by an
`ArrayBuilder`. That has a few other advantages, but it's a much more invasive
change!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
