uros-b commented on code in PR #56864:
URL: https://github.com/apache/spark/pull/56864#discussion_r3735141777
##########
python/pyspark/sql/tests/test_functions.py:
##########
@@ -3641,6 +3641,8 @@ def check(resultDf, expected):
df.select(F.to_json(F.try_variant_array_append(arr, df.arrpath,
F.lit(9)))),
["[1,2,9]", "[[3,9],4]"],
)
+ check(df.select(F.to_json(F.variant_strip_nulls(v))), ['{"a":1}',
'{"b":2}'])
+ check(df.select(F.to_json(F.variant_strip_nulls(v, False))),
['{"a":1}', '{"b":2}'])
Review Comment:
v here is parse_json over {"a":1} and {"b":2} — no nulls anywhere in the
fixture. Both assertions expect the input back unchanged, so nothing is
stripped, and the second line's expected value is identical to the first,
meaning include_arrays=False adds no coverage over the default. This confirms
wiring but not behavior.
The neighbouring variant_delete checks in this same test do exercise real
behavior. A fixture with a null object field and a null array element would
make both lines meaningful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
