jorgecarleitao commented on pull request #11280: URL: https://github.com/apache/arrow/pull/11280#issuecomment-931492822
Thanks for the heads-up! In arrow2 we are exporting the null buffer of the `Union` as a null pointer - I did not notice that this was not in spec :/ We are importing under the same assumption: * the 0th buffer is the validity, * 1st is types, * 2nd is offsets (if sparse) Shifting all the buffers by one in the union array from pyarrow 5 to pyarrow 6 is an issue but fortunately will not result in UB on older importers that consume a dense union: * ignore the 0th buffer (it should read `i8`) * interpret the 1st buffer as `i8` (it should read `i32`) since `size_of<i8>() < size_of<i32>()`, the importer will not read out of bounds when accessing the types (and will just read wrong types). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
