juliuszsompolski commented on code in PR #47790:
URL: https://github.com/apache/spark/pull/47790#discussion_r1721523525
##########
python/pyspark/sql/connect/plan.py:
##########
@@ -281,9 +281,13 @@ def __init__(
assert schema is None or isinstance(schema, str)
if options is not None:
+ new_options = {}
for k, v in options.items():
- assert isinstance(k, str)
- assert isinstance(v, str)
+ if v is not None:
+ assert isinstance(k, str)
+ assert isinstance(v, str)
+ new_options[k] = v
+ options = new_options
Review Comment:
hm ok, so not allowing None seems like a separate issue with Spark Connect.
I wonder if there are any options where we intentionally want to use
None/null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
