Hi Robert,
I agree this should be handled on the producer side of the C Data
Interface (namely, detect embeded nul bytes in a UTF-8 string and error
out).
Regards
Antoine.
Le 10/07/2026 à 12:18, Robert Kruszewski a écrit :
I don't think my example runs into any of the issues you outline. This is a
valid utf-8, nul byte is a valid utf-8 byte. Therefore this schema can
roundtrip via IPC but cannot roundtrip via C FFI.
I think I found the answer in the meantime. I believe the C FFI has to guard
around this case but ideally it wouldn't outright panic. Some kind of error
indicating that this is a valid arrow name but not valid C name is preferable
so the caller can handle it.
Best,
Robert
On Wed, 8 Jul 2026, at 14:56, Weston Pace wrote:
The C FFI interface specifies the schema names as regular char *
The C FFI interface actually goes a little further in the spec[1].
Optional. A null-terminated, UTF8-encoded string of the field or array name.
This is mainly used to reconstruct child fields of nested types.
So for your question "should be duality between the IPC and C FFI apis" I think
the answer is yes, there should be, and there is.
What the fuzzer have found is that column names of shape "abc\0def\0" don't
correctly roundtrip through C FFI
It's not clear to me what you expect would happen. The entire string can't round trip, as it isn't
a valid UTF8 encoded string. Are you expecting an error and not getting an error from some
libraries? If a C FFI library were importing that string then I think it would interpret it as the
string "abc". There is no way for an importer to validate it beyond that as array
lengths don't cross the FFI boundary. It's impossible for the importer to even know the
"def\0" bytes exist.
[1] https://arrow.apache.org/docs/format/CDataInterface.html#c.ArrowSchema.name
On Wed, Jul 8, 2026 at 5:53 AM Felipe Oliveira Carvalho <[email protected]>
wrote:
IMO, crashing is the only viable option. If an application can’t trust a C
FFI exporter, it should have access to validation functions, but the
default importing flow should validate for security reasons (as it parses)
and crash as a safety measure against malicious exporters.
On Wed, 8 Jul 2026 at 09:35 Robert Kruszewski <[email protected]> wrote:
Hi all,
I have been running fuzz tests on arrow C FFI and noticed an interesting
edge case that I couldn't find an answer in the docs/code for. The format
spec has field names as flatbuffer string
https://github.com/apache/arrow/blob/39125bac3960fb0a0625d924b5b3fe174974f530/format/Schema.fbs#L514
which is as defined in https://flatbuffers.dev/schema/#scalars is
Strings (indicated by `string`) are zero-terminated strings, prefixed by
their length. Strings may only hold UTF-8 or 7-bit ASCII. For other text
encodings or general binary data use vectors (`[byte]` or `[ubyte]`)
instead.
The C FFI interface specifies the schema names as regular char * in
https://github.com/apache/arrow/blob/39125bac3960fb0a0625d924b5b3fe174974f530/cpp/src/arrow/c/abi.h#L53.
The question I have is whether there should be duality between the IPC and
C FFI apis, i.e. should they all handle the same schemas and array. What
the fuzzer have found is that column names of shape "abc\0def\0" don't
correctly roundtrip through C FFI. The arrow-rs has an interesting case
that it detects this case and panics in
https://github.com/apache/arrow-rs/blob/52920f946cf8aa038d04fe432e77889f8253b7ce/arrow-schema/src/ffi.rs#L174.
I am curious to know where do you think validation should live or even if
there should be any and we should just let the users decide how they want
to deal with it?
Best,
Robert
