JingsongLi commented on code in PR #9532:
URL: https://github.com/apache/paimon/pull/9532#discussion_r3909588215
##########
paimon-common/src/main/java/org/apache/paimon/data/variant/InferVariantShreddingSchema.java:
##########
@@ -574,7 +574,11 @@ private DataType finalizeAdaptiveSchema(
if (current != null && !(current instanceof VariantType)) {
combined = current;
} else if (previousSelected != null) {
- combined = previousSelected;
+ // This node has no evidence in the current file. A previously
selected schema is
+ // not
+ // evidence - its fields carry no counts - so it cannot be run
through admission and
+ // retention again; carry it forward unchanged.
+ return previousSelected;
Review Comment:
[P2] Count preserved descendants against the shared width budget
`maxFields` is shared across every Variant path in this inference, but this
early return only consumes the single unit decremented at method entry. A
preserved `RowType`/`ArrayType` can contain descendants that would normally
consume additional units, so later Variant columns can now exceed
`variant.shredding.maxSchemaWidth`.
I reproduced this with `maxSchemaWidth = 8` and two Variant columns: the
first evolves from `p: scalar` to `p: {x: scalar}` and then omits `p`, while
the second adds a new `r` field in the third round. This branch carries `ROW<x
BIGINT>` forward without accounting for `x`, then also selects `q` and `r` in
the second column, using 9 budget units. A focused assertion that `r` remains
untyped fails because it becomes `BIGINT`.
Please preserve the selected schema while also walking/debiting its nested
width (and handling insufficient remaining budget consistently) before
returning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
