Wes McKinney created ARROW-76:
---------------------------------
Summary: Revise format document to include null count, defer
non-nullable arrays to the domain of metadata
Key: ARROW-76
URL: https://issues.apache.org/jira/browse/ARROW-76
Project: Apache Arrow
Issue Type: New Feature
Components: Format
Reporter: Wes McKinney
After many mailing list discussions and a first draft of metadata serialization
(https://github.com/apache/arrow/blob/master/format/Message.fbs), I propose we
strike the notion of required fields from the physical memory layout and
instead make this a property of the metadata (see the {{nullable}} member of
{{Field}} in the serialized schema metadata).
Note that Arrow implementations will be free to create specialized containers
for data having {{nullable = false}} in their metadata. It will be the
implementations' responsibility to validate that {{null_count == 0}} at IPC
boundaries when interacting with "untrusted" data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
