[
https://issues.apache.org/jira/browse/SPARK-4603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen resolved SPARK-4603.
-------------------------------
Resolution: Cannot Reproduce
I'm going to resolve this as "Cannot Reproduce" for now. Please re-open if you
have more information.
> EOF when broadcasting a dict with an empty string value.
> --------------------------------------------------------
>
> Key: SPARK-4603
> URL: https://issues.apache.org/jira/browse/SPARK-4603
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 1.1.0
> Environment: OSX 10.10
> Reporter: Alex Angelini
>
> Steps to reproduce:
> 1. Broadcast {'a': ''}
> 2. Try to read the value of the broadcast
> {code}
> Welcome to
> ____ __
> / __/__ ___ _____/ /__
> _\ \/ _ \/ _ `/ __/ '_/
> /__ / .__/\_,_/_/ /_/\_\ version 1.3.0-SNAPSHOT
> /_/
> Using Python version 2.7.8 (default, Oct 19 2014 16:02:00)
> SparkContext available as sc.
> In [1]: sc
> Out[1]: <pyspark.context.SparkContext at 0x1077e4710>
> In [2]: b = sc.broadcast({'a': ''})
> In [3]: b.value
> ---------------------------------------------------------------------------
> EOFError Traceback (most recent call last)
> <ipython-input-3-59996048613f> in <module>()
> ----> 1 b.value
> /Users/alexangelini/src/starscream/spark/current/python/pyspark/broadcast.pyc
> in value(self)
> 75 if not hasattr(self, "_value") and self.path is not None:
> 76 ser = LargeObjectSerializer()
> ---> 77 self._value = ser.load_stream(open(self.path)).next()
> 78 return self._value
> 79
> /Users/alexangelini/src/starscream/spark/current/python/pyspark/serializers.pyc
> in load_stream(self, stream)
> 615 yield value
> 616 elif type == 'P':
> --> 617 yield cPickle.load(stream)
> 618 else:
> 619 raise ValueError("unknown type: %s" % type)
> EOFError:
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
