Xiangrui Meng created SPARK-9487:
------------------------------------
Summary: Use the same num. worker threads in Scala/Python unit
tests
Key: SPARK-9487
URL: https://issues.apache.org/jira/browse/SPARK-9487
Project: Spark
Issue Type: Improvement
Components: PySpark, Spark Core, SQL, Tests
Affects Versions: 1.5.0
Reporter: Xiangrui Meng
In Python we use `local[4]` for unit tests, while in Scala/Java we use
`local[2]` and `local` for some unit tests in SQL, MLLib, and other components.
If the operation depends on partition IDs, e.g., random number generator, this
will lead to different result in Python and Scala/Java. It would be nice to use
the same number in all unit tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
