[
https://issues.apache.org/jira/browse/ARROW-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026868#comment-17026868
]
Antoine Pitrou commented on ARROW-1059:
---------------------------------------
Would it be sufficient to have something like:
{code}
union MessageHeader {
Schema, DictionaryBatch, RecordBatch, Tensor, SparseTensor, RawData
}
table RawData {
buffers: [Buffer];
}
{code}
> [C++] Define API for embedding user-defined metadata / Flatbuffer message
> types in Arrow IPC machinery
> ------------------------------------------------------------------------------------------------------
>
> Key: ARROW-1059
> URL: https://issues.apache.org/jira/browse/ARROW-1059
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Wes McKinney
> Priority: Major
>
> Currently, the {{MessageHeader}} Flatbuffer union must be modified to
> serialize new kinds of metadata:
> https://github.com/apache/arrow/blob/master/format/Message.fbs#L85
> It would be interesting if user metadata could be embedded within a
> particular application that wishes to use the Arrow C++ libraries' zero-copy
> IPC machinery for serialization of other kinds of data structures.
> As one approach, the message metadata could be an application-dependent
> unique identifier for the user defined type, which would internally dispatch
> to an implementation of an abstract deserializer interface. So in addition to
> describing the serialized representation of the user type, we also will have
> to create the abstract API for the user to implement so that the code in
> {{arrow/ipc}} can be configured to dispatch appropriately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
