Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/2874#issuecomment-60014782
Yes, that would also fix the overflow, or at least, push it much farther
away. Does it ever make sense to have tens of billions of elements in the RDD
for this computation though? Allowing quardillions doesn't probably work
anyway. I'd just cap the number of slices to something sane.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
