Seth Hollyman created ARROW-13191:
-------------------------------------
Summary: [Go] Support external schema in ipc readers
Key: ARROW-13191
URL: https://issues.apache.org/jira/browse/ARROW-13191
Project: Apache Arrow
Issue Type: Improvement
Components: Go
Reporter: Seth Hollyman
(Apologies if I'm imprecise here, I'm still coming up to speed on the arrow
details.)
The IPC message format describes how data and metadata messages are
encapsulated, but it is not a requirement that each message must include the
schema.
In Go, github.com/apache/arrow/go/arrow/ipc contains NewReader() for setting up
reading of IPC messages, and accepts the option WithSchema to pass the schema
into said reader. However, the implementation merely uses that information to
compare that the schema it reads from the IPC stream matches the passed in
reader. This request is to allow WithSchema to behave as expected, and use the
option-provided Schema for performing reads.
The one gotcha here appears to be the dictionary type map, which is currently
retained independently of the schema but is part of the internal readSchema()
setup. Completeness may warrant another option for communicating those
externally as well? Or perhaps option-passed Schema should be documented to
not support dictionary types?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
