HyukjinKwon commented on code in PR #47790:
URL: https://github.com/apache/spark/pull/47790#discussion_r1721546283
##########
python/pyspark/sql/connect/plan.py:
##########
@@ -281,9 +281,13 @@ def __init__(
assert schema is None or isinstance(schema, str)
if options is not None:
+ new_options = {}
for k, v in options.items():
- assert isinstance(k, str)
- assert isinstance(v, str)
+ if v is not None:
+ assert isinstance(k, str)
+ assert isinstance(v, str)
+ new_options[k] = v
+ options = new_options
Review Comment:
> I wonder if there are any options where we intentionally want to use
None/null?
I am not sure about newly added options but I actually took a look few years
ago. Virtually no option that should legitimately support `None` / `null`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
