Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/8246#issuecomment-139337078
@NathanHowell The real problem for MLlib is having a large number of
features; I'd say a few thousand is a reasonable limit right now. I'm working
on a new implementation which should be much faster for more features,
scheduled for 1.6.
Curious: When you ran MLlib, did you try useNodeIdCache, with checkpointing
turned on? That should help a little.
Also, do you know if the 2 implementations learned similarly sized trees?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
