GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/4011
[SPARK-4348] [SPARK-4821] Backport PySpark random.py -> rand.py fix to
branch-1.1
This backports #3216 and #3669 to `branch-1.1` in order to fix the PySpark
unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark pyspark-rand-fix-1.1-backport
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4011.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4011
----
commit 7ae5a1cf53a68a05f7b36e06d4f90f48485c0db4
Author: Davies Liu <[email protected]>
Date: 2014-11-13T18:24:54Z
[SPARK-4348] [PySpark] [MLlib] rename random.py to rand.py
This PR rename random.py to rand.py to avoid the side affects of conflict
with random module, but still keep the same interface as before.
```
>>> from pyspark.mllib.random import RandomRDDs
```
```
$ pydoc pyspark.mllib.random
Help on module random in pyspark.mllib:
NAME
random - Python package for random data generation.
FILE
/Users/davies/work/spark/python/pyspark/mllib/rand.py
CLASSES
__builtin__.object
pyspark.mllib.random.RandomRDDs
class RandomRDDs(__builtin__.object)
| Generator methods for creating RDDs comprised of i.i.d samples from
| some distribution.
|
| Static methods defined here:
|
| normalRDD(sc, size, numPartitions=None, seed=None)
```
cc mengxr
reference link: http://xion.org.pl/2012/05/06/hacking-python-imports/
Author: Davies Liu <[email protected]>
Closes #3216 from davies/random and squashes the following commits:
7ac4e8b [Davies Liu] rename random.py to rand.py
(cherry picked from commit ce0333f9a008348692bb9a200449d2d992e7825e)
Signed-off-by: Josh Rosen <[email protected]>
Conflicts:
python/pyspark/mllib/feature.py
python/run-tests
commit ace4cb614588e7f8423d854ace6bf2b1c61fd1dc
Author: Joseph K. Bradley <[email protected]>
Date: 2014-12-17T22:12:46Z
[SPARK-4821] [mllib] [python] [docs] Fix for pyspark.mllib.rand doc
+ small doc edit
+ include edit to make IntelliJ happy
CC: davies mengxr
Note to davies -- this does not fix the "WARNING: Literal block expected;
none found." warnings since that seems to involve spacing which IntelliJ does
not like. (Those warnings occur when generating the Python docs.)
Author: Joseph K. Bradley <[email protected]>
Closes #3669 from jkbradley/python-warnings and squashes the following
commits:
4587868 [Joseph K. Bradley] fixed warning
8cb073c [Joseph K. Bradley] Updated based on davies recommendation
c51eca4 [Joseph K. Bradley] Updated rst file for pyspark.mllib.rand doc.
Small doc edit. Small include edit to make IntelliJ happy.
Conflicts:
python/docs/pyspark.streaming.rst
python/pyspark/mllib/feature.py
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
