Stefan Savic created SPARK-60711:
------------------------------------
Summary: is_valid_variant returns true when an object or array has
a zero child count but is missing its required final offset
Key: SPARK-60711
URL: https://issues.apache.org/jira/browse/SPARK-60711
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Stefan Savic
Using valid empty metadata [1, 0, 0], with bytes shown in decimal:
- Object value [2, 0]: currently returns true; expected false.
- Array value [3, 0]: currently returns true; expected false.
- Complete empty object [2, 0, 0]: correctly returns true.
- Complete empty array [3, 0, 0]: correctly returns true.
An object or array with n children requires n + 1 offsets. Even an empty
container requires one final offset, equal to zero.
VariantUtil.validateImpl validates children in a loop without checking the
final offset. When the child count is zero, the loop is skipped and the
incomplete encoding is accepted.
Validation should reject missing final offsets.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
