GitHub user SaintBacchus reopened a pull request:
https://github.com/apache/spark/pull/2874
[SPARK-4033][Examples]Input of the SparkPi too big causes the emption
exception
If input of the SparkPi args is larger than the 25000, the integer 'n'
inside the code will be overflow, and may be a negative number.
And it causes the (0 until n) Seq as an empty seq, then doing the action
'reduce' will throw the UnsupportedOperationException("empty collection").
The max size of the input of sc.parallelize is Int.MaxValue - 1, not the
Int.MaxValue.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SaintBacchus/spark SparkPi
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2874.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2874
----
commit 9a2fb7b7d59e3dbe2193112dd328e521cab3a9d9
Author: huangzhaowei <[email protected]>
Date: 2014-10-21T09:18:27Z
Input of the SparkPi is too big
if the Input of the SparkPi is too big, it will throw new
UnsupportedOperationException("empty collection"), because the n may be a
negative int number.
commit 4cdc388780180f0403832c904f611d2ac581a4c4
Author: huangzhaowei <[email protected]>
Date: 2014-10-21T15:21:35Z
Update SparkPi.scala
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]