westonpace commented on issue #35450: URL: https://github.com/apache/arrow/issues/35450#issuecomment-1644714162
> Ok, so the record batch is invalid. @pitrou This seems to disagree with your statement on the ML thread: https://lists.apache.org/thread/6jtyf5xhfdocb2rlx1jfjwx0rj4hn6o1 > The fact that a struct field may be backed by a > physically larger C++ ArrayData is irrelevant, as long as it's logically > interpreted as having "the same length". However, in the ML, we were talking about struct arrays, and not strictly record batches, and it's probably ok for those two things to act differently. If we want to align on `RecordBatch::Validate` then I think we need to update `RecordBatch::FromStructArray` to check for this case and repair it (instead of using the struct array's length directly). If we instead want record batches to behave like struct arrays were described in the ML discussion then `RecordBatch::Validate` is wrong and this should be valid. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
