zeroshade commented on a change in pull request #12158:
URL: https://github.com/apache/arrow/pull/12158#discussion_r829417887
##########
File path: go/arrow/ipc/file_reader.go
##########
@@ -157,22 +162,9 @@ func (f *FileReader) readSchema() error {
return err
}
- id, dict, err := readDictionary(msg.meta, f.fields, f.r)
- msg.Release()
- if err != nil {
- return xerrors.Errorf("arrow/ipc: could not read
dictionary %d from file: %w", i, err)
+ if _, err = readDictionary(&f.memo, msg.meta,
bytes.NewReader(msg.body.Bytes()), f.mem); err != nil {
Review comment:
Good point, i've added the check here so that we error on a replacement
since that is not supported for IPC files.
As far as expected type: that is validated inside the `readDictionary`
function. We get the expected types from the schema, and read dictionaries
using the types as pulled from the memo table populated by the schema. If the
type doesn't match, `ctx.loadArray` inside of `readDictionary` will fail with
the wrong type.
We don't currently do the stats on the number of dictionaries like the C++
lib does, but it might get added. So usage of the dictionary kind will likely
get utilized more than just having the IPC File fail on replacements in a later
change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
