Thomas Graves created SPARK-26201:
-------------------------------------
Summary: python broadcast.value on driver fails with disk
encryption enabled
Key: SPARK-26201
URL: https://issues.apache.org/jira/browse/SPARK-26201
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 2.3.2
Reporter: Thomas Graves
I was trying python with rpc and disk encryption enabled and when I tried a
python broadcast variable and just read the value back on the driver side the
job failed with:
Traceback (most recent call last): File "broadcast.py", line 37, in <module>
words_new.value File "/pyspark.zip/pyspark/broadcast.py", line 137, in value
File "pyspark.zip/pyspark/broadcast.py", line 122, in load_from_path File
"pyspark.zip/pyspark/broadcast.py", line 128, in load EOFError: Ran out of input
To reproduce use configs: --conf spark.network.crypto.enabled=true --conf
spark.io.encryption.enabled=true
Code:
words_new = sc.broadcast(["scala", "java", "hadoop", "spark", "akka"])
words_new.value
print(words_new.value)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
