CurtHagenlocher commented on issue #40517: URL: https://github.com/apache/arrow/issues/40517#issuecomment-1997510164
Yeah, it was late and I only had time for a quick look. This avoids the exception but ends up producing the wrong results. The entire ArrowBuffer is being serialized instead of just the part needed for the slice, and while the length is recorded correctly there's (of course) no offset being serialized so the data being read back starts at the beginning of the buffer for each slice -- meaning it's the wrong data. The root of this problem is that ArrowStreamWriter.ArrowRecordBatchFlatBufferBuilder is serializing entire buffers instead of just the parts of the buffer associated with the slice. I suspect this will not be a trivial fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
