alamb commented on issue #9716: URL: https://github.com/apache/arrow-rs/issues/9716#issuecomment-4261755578
> Strikingly, C++ explicitely exercises this permissive case , including on [validation](https://github.com/apache/arrow/blob/4eca50770f7f2c5938a676f0719fbfc8aae4803c/cpp/src/arrow/array/validate.cc#L916). Now the question is whether the spec should be updated (and then other arrow libraries such as polars/arrow2 will have to change), or whether we stick with the spec and output strictly compliant files. This looks like it came from - https://github.com/apache/arrow/pull/35345 which has more context. This is the mysterious ARROW-554 which doesn't seem to be very helpful honestly - https://github.com/apache/arrow/issues/16191 So i agree this is not explicitly allowed in the spec However, I think following the cpp implementation makes sense in general. Does the cpp implementation write such files? Or does it write files with an offset buffer of size 1 as you are proposing? Perhaps we should start a discussion on the arrow mailing list as the next step -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
