Github user mengxr commented on the pull request:
https://github.com/apache/spark/pull/4200#issuecomment-71545716
@hhbyyh It really depends on whether you need the full SVD or the truncated
SVD. For the 60k * 70k problem, do you need its full SVD? If that is the case,
I would say this is not yet supported. If you only need top 50 singular values,
the dist mode should be able to handle it.
The limit is correct but I agree with @srowen that this is already beyond
what we can really handle on a single machine. I would keep this hard limit and
set a soft limit at `5000` (or `10000`). We output a warning message if n is
greater than this soft limit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
