Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/30#issuecomment-37108992
I left a few minor comments in the diff, but overall this looks good to me.
It might be worth adding build/run instructions in either the PySpark
Programming Guide or YARN guide.
It also occurred to me that the Makefile-based build for the PySpark fat
zip might be a problem for Windows users; Scala/Java Spark works fine under
Cygwin, but PySpark only works in cmd.exe / powershell (the main difficulty is
that in some cases the Java and Python halves of the PySpark driver expect
different types of paths, so we'd have to replicate parts of the cygpath logic
in Java and Python). I suppose we could use the Python
[`zipfile`](http://docs.python.org/2/library/zipfile) library and implement the
build script in Python. Or, as @ahirreddy suggested, maybe we could package
the Python libraries into a JAR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
