uros-b commented on code in PR #56864:
URL: https://github.com/apache/spark/pull/56864#discussion_r3740623108
##########
common/variant/src/main/java/org/apache/spark/types/variant/VariantBuilder.java:
##########
@@ -171,6 +171,16 @@ public static Variant arrayAppendAtPath(Variant v,
PathSegment[] segments, Varia
return builder.result();
}
+ // Return a new variant with null-valued object fields removed, recursing
into nested objects
+ // and arrays. When `includeArrays` is true, null array elements are removed
too; when false,
+ // arrays keep their nulls but objects inside them are still cleaned. Empty
containers and a
+ // top-level variant null are left unchanged. The result is always rebuilt
with fresh metadata.
Review Comment:
"Empty containers ... are left unchanged" reads as if the guarantee only
covers containers that were already empty. The more useful guarantee is that a
container emptied by stripping is preserved as {} / [] rather than collapsed to
null; worth saying explicitly, since that's exactly what the tests at
VariantExpressionSuite pin down.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
