[
https://issues.apache.org/jira/browse/ARROW-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney resolved ARROW-262.
--------------------------------
Resolution: Fixed
Issue resolved by pull request 121
[https://github.com/apache/arrow/pull/121]
> [Format] Add a new format document for metadata and logical types for
> messaging and IPC / on-wire/file representations
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-262
> URL: https://issues.apache.org/jira/browse/ARROW-262
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Format
> Reporter: Wes McKinney
> Assignee: Wes McKinney
>
> The existing document
> https://github.com/apache/arrow/blob/master/format/Layout.md
> Only describes the physical layout of fixed-size, variable-size, and other
> nested types (struct, union)
> Meanwhile, we have begun drafting Flatbuffers IDL for Arrow metadata:
> https://github.com/apache/arrow/blob/master/format/Message.fbs
> I will add a document that will, to begin with:
> * Explain the mapping between logical types in the metadata. For example,
> definitions of important data types: integers, floating point, boolean,
> string (UTF-8) and binary
> * Where relevant, describing how each logical type's physical memory is
> converted to metadata for messaging purposes (e.g. the {{RecordBatch}}
> concept in the IDL)
> We have already begun prototype implementations in the C++ codebase
> (https://github.com/apache/arrow/tree/master/cpp/src/arrow/ipc) so this will
> serve as implementation-agnostic documentation.
> Subsequently, I will make a follow up patch for discussion to hopefully
> address metadata shortfall between the canonical Arrow metadata and the
> similar metadata used by the bespoke Feather format
> (https://github.com/wesm/feather/blob/master/cpp/src/feather/metadata.fbs)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
