squito commented on a change in pull request #23337: [SPARK-26019][PYSPARK]
Allow insecure py4j gateways
URL: https://github.com/apache/spark/pull/23337#discussion_r242664610
##########
File path: python/pyspark/context.py
##########
@@ -112,6 +112,18 @@ def __init__(self, master=None, appName=None,
sparkHome=None, pyFiles=None,
ValueError:...
"""
self._callsite = first_spark_call() or CallSite(None, None, None)
+ if gateway is not None and gateway.gateway_parameters.auth_token is
None:
+ if conf and conf.get("spark.python.allowInsecurePy4j", "false") ==
"true":
Review comment:
I thought about this more, and I realized that if we're requiring a code
change for the end user, it sort of defeats the entire purpose of this change
(eg. users need this because they can't upgrade zeppelin). One thing I could
do is delay this check until we get the full spark conf back from java:
https://github.com/apache/spark/blob/5ed89ceaf367590f79401abbf9ff7fc66507fe4e/python/pyspark/context.py#L169-L171
then the check could be based on the real conf, which includes anything set
in a file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
