jorgecarleitao commented on a change in pull request #9496:
URL: https://github.com/apache/arrow/pull/9496#discussion_r582568136
##########
File path: rust/arrow/src/buffer/immutable.rs
##########
@@ -36,7 +36,7 @@ use super::MutableBuffer;
#[derive(Clone, PartialEq, Debug)]
pub struct Buffer {
/// the internal byte buffer.
- data: Arc<Bytes>,
+ data: Arc<Bytes<u8>>,
Review comment:
I think that any work that unblocks you is worth pursuing :)
A path could be:
1. Make `Buffer` support both bit slices and byte slices:
* Add a new `offset_bits` to `Buffer`.
* Expose a method `Buffer::slice_bitmap` that increments `offset_bits` (and
panics if called with `offset != 0`).
* Make `Buffer::slice` panic if called with `offset_bits != 0`.
These will allow us to slice `Buffer` when it is either holder of bits and a
holder of bytes (we just need to use either API independently). This is
required because the Boolean array's values are in bits (and thus part of
ArrayData::buffers).
2. Implement a method
```pub fn slice(&self, offset: usize, length: usize) -> Self```
on every array, that calls `slice` or `slice_bits` depending on what the
Buffer represents (and also fix the incorrect slicing of arrays with offsets).
3. make `Array::slice` call the concrete implementations
4. Make all calls of `get_bit` and the like to use `Buffer::offset_bits` to
compute slot nullability and/or values.
Note that I have not tested this and thus this may not work. I also do not
know the consequences to IPC and FFI. :/
I am more confident that the proposal works because it passes IPC tests (IPC
read and write, little and big endian ;)).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
