Rich-T-kid commented on PR #9778: URL: https://github.com/apache/arrow-rs/pull/9778#issuecomment-5025446597
> > it seems we ran into the same issue in #10125, mis-aligned buffers were causing issues in arrow-ipc. #10273 resolved these issues. Im not sure if replacing MutableBuffer for Vec can work here unless we had a way to check that vectors underlying allocation was aligned. > > @Rich-T-kid is there an issue for whatever problem you ran into? no, i tried removing the buffer copy that occured for every `flightdata` chunk that came in, but this caused issues when the tonic buffer returned a mis-aligned buffer. this was resolved it #10273. the benchmarks for using a vector directly does look good, if arrow exposed an way to align vectors it would be the best of both worlds due to how many optimziations are built into the vector type. I thought about a possible new type pattern where we wrapped a vector in a byte slice of 64 chunks. ex ```rust #[repr(align(64))] struct Chunk([u8; 64]); ``` this has the benefit of being aligned when its first created as well as being aligned for re-allocations when the vector grows. If this sounds interesting I could continue working on it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
