[
https://issues.apache.org/jira/browse/ARROW-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611680#comment-17611680
]
Steve M. Kim edited comment on ARROW-16430 at 9/30/22 4:25 PM:
---------------------------------------------------------------
After further thought and re-reading the flatbuffer definitions and c++
implementation, I rescind my statement about custom metadata on a RecordBatch.
The {{RecordBatch}} table doesn't have a {{custom_metadata}} field.
However, I still think that there is a confusion between the metadata of a
{{pyarrow.Schema}} versus the metadata of a {{pyarrow.Message}} that contains a
{{{}Schema{}}}. It ought to be possible to specify custom metadata when
serializing a {{Schema}} as a IPC message, in exactly the same way that it
ought to be possible to specify custom metadata when serializing a
{{RecordBatch}} or {{DictionaryBatch}} as a IPC message.
was (Author: chairmank):
After further thought and re-reading the flatbuffer definitions and c++
implementation, I rescind my statement about custom metadata on a RecordBatch.
The {{RecordBatch}} table doesn't have a {{custom_metadata}} field.
However, I still think that there is a confusion between the metadata of a
{{pyarrow.Schema}} versus the metadata of a {{pyarrow.Message}} that contains
a{{ Schema.}} It ought to be possible to specify custom metadata when
serializing a {{Schema}} as a IPC message, in exactly the same way that it
ought to be possible to specify custom metadata when serializing a
{{RecordBatch}} or {{DictionaryBatch}} as a IPC message.
> [Python] Read/Write record batch custom metadata API in pyarrow
> ---------------------------------------------------------------
>
> Key: ARROW-16430
> URL: https://issues.apache.org/jira/browse/ARROW-16430
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Affects Versions: 7.0.0
> Reporter: Yue Ni
> Assignee: Yue Ni
> Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> In https://issues.apache.org/jira/browse/ARROW-16131, Arrow C++ APIs were
> added so that users can read/write record batch custom metadata for IPC file.
> But pyarrow still lacks corresponding APIs for doing this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
