Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/3213#issuecomment-63139261
Thanks for working on this. I have two high level comments:
- I think it would be better to have a single implementation in Scala with
a wrapper in python. This way we don't have to serialize / ship the objects to
python which seems like it might be expensive, especially if the next thing you
are going to do is something like `saveAsTextFile`
- It would also be better to use jackson to do the generation of the JSON
string as there are a lot of tricky edge cases around escaping that we need to
handle if we do it ourselves. For example, I think this version will fail if a
column name contains a quote character.
/cc @yhuai
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
