[jira] [Commented] (SPARK-26201) python broadcast.value on driver fails with disk encryption enabled
[ https://issues.apache.org/jira/browse/SPARK-26201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702118#comment-16702118 ] Apache Spark commented on SPARK-26201: -- User 'redsanket' has created a pull request for this issue: https://github.com/apache/spark/pull/23166 > python broadcast.value on driver fails with disk encryption enabled > --- > > Key: SPARK-26201 > URL: https://issues.apache.org/jira/browse/SPARK-26201 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.3.2 >Reporter: Thomas Graves >Priority: Major > > I was trying python with rpc and disk encryption enabled and when I tried a > python broadcast variable and just read the value back on the driver side the > job failed with: > > Traceback (most recent call last): File "broadcast.py", line 37, in > words_new.value File "/pyspark.zip/pyspark/broadcast.py", line 137, in value > File "pyspark.zip/pyspark/broadcast.py", line 122, in load_from_path File > "pyspark.zip/pyspark/broadcast.py", line 128, in load EOFError: Ran out of > input > To reproduce use configs: --conf spark.network.crypto.enabled=true --conf > spark.io.encryption.enabled=true > > Code: > words_new = sc.broadcast(["scala", "java", "hadoop", "spark", "akka"]) > words_new.value > print(words_new.value) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26201) python broadcast.value on driver fails with disk encryption enabled
[ https://issues.apache.org/jira/browse/SPARK-26201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702046#comment-16702046 ] Sanket Reddy commented on SPARK-26201: -- Thanks [~tgraves] will put up the patch shortly > python broadcast.value on driver fails with disk encryption enabled > --- > > Key: SPARK-26201 > URL: https://issues.apache.org/jira/browse/SPARK-26201 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.3.2 >Reporter: Thomas Graves >Priority: Major > > I was trying python with rpc and disk encryption enabled and when I tried a > python broadcast variable and just read the value back on the driver side the > job failed with: > > Traceback (most recent call last): File "broadcast.py", line 37, in > words_new.value File "/pyspark.zip/pyspark/broadcast.py", line 137, in value > File "pyspark.zip/pyspark/broadcast.py", line 122, in load_from_path File > "pyspark.zip/pyspark/broadcast.py", line 128, in load EOFError: Ran out of > input > To reproduce use configs: --conf spark.network.crypto.enabled=true --conf > spark.io.encryption.enabled=true > > Code: > words_new = sc.broadcast(["scala", "java", "hadoop", "spark", "akka"]) > words_new.value > print(words_new.value) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26201) python broadcast.value on driver fails with disk encryption enabled
[ https://issues.apache.org/jira/browse/SPARK-26201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702028#comment-16702028 ] Thomas Graves commented on SPARK-26201: --- the issue here seems to be that it isn't decrypting the file before trying to read it, we will have a patch up for this shortly. > python broadcast.value on driver fails with disk encryption enabled > --- > > Key: SPARK-26201 > URL: https://issues.apache.org/jira/browse/SPARK-26201 > Project: Spark > Issue Type: Bug > Components: PySpark >Affects Versions: 2.3.2 >Reporter: Thomas Graves >Priority: Major > > I was trying python with rpc and disk encryption enabled and when I tried a > python broadcast variable and just read the value back on the driver side the > job failed with: > > Traceback (most recent call last): File "broadcast.py", line 37, in > words_new.value File "/pyspark.zip/pyspark/broadcast.py", line 137, in value > File "pyspark.zip/pyspark/broadcast.py", line 122, in load_from_path File > "pyspark.zip/pyspark/broadcast.py", line 128, in load EOFError: Ran out of > input > To reproduce use configs: --conf spark.network.crypto.enabled=true --conf > spark.io.encryption.enabled=true > > Code: > words_new = sc.broadcast(["scala", "java", "hadoop", "spark", "akka"]) > words_new.value > print(words_new.value) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
