Le 26/10/2023 à 20:02, Benjamin Kietzman a écrit :
Is this buffer lengths buffer only present if the array type is Utf8View?
IIUC, the proposal would add the buffer lengths buffer for all types if the
schema's
flags include ARROW_FLAG_BUFFER_LENGTHS. I do find it appealing to avoid
the special case and that `n_buffers` would continue to be consistent with
IPC.
This begs the question of what happens if a consumer receives an unknown
flag value. We haven't specified that unknown flag values should be
ignored, so a consumer could judiciously choose to error out instead of
potentially misinterpreting the data.
All in all, personally I'd rather we make a special case for Utf8View
instead of adding a flag that can lead to worse interoperability.
Regards
Antoine.
