JingsongLi commented on code in PR #10226:
URL: https://github.com/apache/paimon/pull/10226#discussion_r4253768152
##########
paimon-common/src/main/java/org/apache/paimon/data/variant/PaimonShreddingUtils.java:
##########
@@ -598,16 +610,19 @@ public static InternalRow assembleVariantStruct(
ByteBuffer topLevelMetadata = binaryBuffer(inputRow,
schema.topLevelMetadataIdx);
int numFields = fields.length;
GenericRow resultRow = new GenericRow(numFields);
+ if (numFields == 0) {
+ return resultRow;
+ }
+
+ VariantMetadata metadata = fields[0].reader.metadata();
+ metadata.adopt(topLevelMetadata);
Review Comment:
[P2] Initialize each distinct cache used by the supplied field extractors
The retained four-argument buildFieldsToExtract overload creates a fresh
VariantMetadata per call, but this now adopts metadata only for fields[0].
Previously valid code that builds two extractors with separate ordinary calls
(for example $.a as INT and $.b as STRING) and passes them together to
assembleVariantStruct leaves the second cache empty. A real
castShredded/assembleVariantStruct probe with {"a":1,"b":"hello"} now throws
IllegalStateException (`metadata must be adopted before read`), instead of
returning both fields; an unshredded fallback can dereference a null buffer.
The baseline tests used this construction and this PR changes them to explicit
shared caches, hiding the compatibility break. The standard Parquet
getFieldsToExtract path shares a cache correctly, so this is specifically a
regression in the existing public helper combination. Please adopt once per
distinct cache in the supplied fields, or otherwise preserve the four-argument
helper contract while keeping s
hared/recursive readers on the single-adoption path. Add a multi-field test
using the retained overload.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
