Github user mateiz commented on the issue:
https://github.com/apache/spark/pull/8318
Something like this would be great IMO. A few questions though:
* How will it work if users want to run a different version of PySpark from
a different version of Spark (maybe something they installed locally)? How can
they easily swap that out? We don't want this making it harder to use Spark
against a real cluster because the version you got from pip is wrong.
* What are the mechanics of publishing to PyPI? Can we make an account
that's shared by all the committers somehow? Can we sign releases? Note that
there is a [release policy](http://www.apache.org/dev/release.html) at the ASF
that we need to make sure this follows. In particular, does anyone have
examples of other ASF projects that publish to PyPI?
* What features will and won't work out of the box in the current
implementation -- e.g. can you use it to access existing Hadoop clusters or S3,
or is it just for local mode?
* How do we automatically test this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
