Github user silentsnooc commented on the issue:
https://github.com/apache/spark/pull/8707
Hi! I am not sure if this is related but is I look for this issue
everything points me here basically. I'm getting
```
f(self, obj) # Call unbound method with explicit self
File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "/usr/lib/python2.7/pickle.py", line 681, in _batch_setitems
save(v)
File "/usr/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/opt/spark/python/lib/pyspark.zip/pyspark/cloudpickle.py", line
315, in save_builtin_function
return self.save_function(obj)
File "/opt/spark/python/lib/pyspark.zip/pyspark/cloudpickle.py", line
191, in save_function
if islambda(obj) or obj.__code__.co_filename == '<stdin>' or themodule
is None:
AttributeError: 'builtin_function_or_method' object has no attribute
'__code__'
```
When trying to create a data frame from an RDD:
```python
rdd = self.sc.textFile(self.input_file_path).map(lambda line:
self.process_line(line))
schema = StructType([StructField(u'Variable', StringType(),
nullable=False),
StructField(u'Time', TimestampType(),
nullable=False),
StructField(u'Value', FloatType(),
nullable=False)])
return sql_context.createDataFrame(rdd, schema)
```
I am on PySpark 1.6.0 - any ideas what I'm doing wrong here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
