Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/455#issuecomment-46745394
@rjurney this works for me (building Spark from current master):
https://gist.github.com/MLnick/5864741781b9340cb211
if you run ```mvn package``` and then add that to ```SPARK_CLASSPATH``` and
use it in IPython console.
However it seems to come through as only strings (not a dict). I verified
that if I take only the string field and explicitly convert to string (ie
```Map[String, String]```) then it works. I suspect then that Avro doesn't have
the type information at all, so Pyrolite cannot pickle it. I guess you might
have to do something more in depth in the ```AvroConverter``` to read the type
info from the Avro schema and do a cast...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
