zeroshade commented on PR #14989:
URL: https://github.com/apache/arrow/pull/14989#issuecomment-1385883135
@minyoung So I spent a while digging into this and I think the issue is that
your data type in the test says the field is not nullable, but you are adding a
null..
Currently your test defines the builder like so:
```go
bldr := array.NewStructBuilder(memory.DefaultAllocator, arrow.StructOf(
arrow.Field{
Name: "l",
Type: arrow.ListOf(arrow.StructOf(
arrow.Field{Name: "a", Type:
arrow.BinaryTypes.String},
)),
},
))
```
If I add `Nullable: true` so it looks like this:
```go
bldr := array.NewStructBuilder(memory.DefaultAllocator, arrow.StructOf(
arrow.Field{
Name: "l",
Type: arrow.ListOf(arrow.StructOf(
arrow.Field{Name: "a", Type:
arrow.BinaryTypes.String},
)),
Nullable: true,
},
))
```
Then the test no longer fails *even without the +1 hack*. The issue is that
the code makes some assumptions based on the data types since it generates the
corresponding Parquet schema which, in the case of your original version, says
it's a required field (ie: no nulls allowed). But you're adding a null to
write. So things get wonky because of the mismatch of putting a null into a
field marked required and non-null. Can you please try testing and confirm
that? I think this is the source of your problem, and not an actual bug in the
code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
