jayshrivastava opened a new issue, #35119:
URL: https://github.com/apache/arrow/issues/35119
### Describe the usage question you have. Please include as many useful
details as possible.
👋🏽 I have an schema field defined by `schema.NewPrimitiveNode("mycol",
parquet.Repetitions.Repeated, parquet.Types.Int32, -1, -1)`. What's the correct
way to encode an array which is empty?
I'm also not 100% sure about my understanding of def levels and rep levels.
Here are some examples of data I'm encoding. For def levels, I put 1 when a
value is defined and 0 when a value is null. For rep levels, I put a 0 for the
first datum in an array and 1 for the remaining ones.
Values: [1, 2, 3], [4, 5]
Def Level: 1, 1, 1, 1, 1
Rep Level: 0, 1, 1, 0, 1
Values: [NULL, 5, 6]
Def Level: 0, 1, 1
Rep Level: 0, 1, 1
Values: [NULL], [1, 2, 3]
Def Level: 0, 1, 1, 1
Rep Level: 0, 0, 1, 1
Values: [NULL], [NULL]
Def Level: 0, 0
Rep Level: 0, 0
Values: [1], [], [2]
Def Level: 1, ?, 1
Rep Level: 0, ?, 0
I'm not sure how to represent the empty array since 0, 0 for (def, rep)
represents an array of size 1 with a NULL value. If I don't put anything for
the def level or rep level of the empty array, then a reader cannot tell that
an empty array exists.
### Component(s)
Go, Parquet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]