HyukjinKwon commented on code in PR #47790:
URL: https://github.com/apache/spark/pull/47790#discussion_r1721519179
##########
python/pyspark/sql/connect/plan.py:
##########
@@ -281,9 +281,13 @@ def __init__(
assert schema is None or isinstance(schema, str)
if options is not None:
+ new_options = {}
for k, v in options.items():
- assert isinstance(k, str)
- assert isinstance(v, str)
+ if v is not None:
+ assert isinstance(k, str)
+ assert isinstance(v, str)
+ new_options[k] = v
+ options = new_options
Review Comment:
Hm, actually Spark Connect doesn't allow `None`. At least it will test Spark
Classic path for now so I will just merge this in as it tests at least.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
