kszucs commented on issue #9637: URL: https://github.com/apache/arrow-rs/issues/9637#issuecomment-4169257955
This is due to the allowed null leaf array elements which is explicitly disallowed in the C++ parquet writer. Apparently the rust implementation allows it and the benchmarks generate list arrays with values for null list elements. I assume this only affects rare cases, but adjusting the cdc chunker to support both layouts. > Similar to the layout of variable-size binary, a null value may correspond to a non-empty segment in the child array. When this is true, the content of the corresponding segment can be arbitrary. See https://arrow.apache.org/docs/format/Columnar.html#list-layout -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
