Github user MLnick commented on the pull request:
https://github.com/apache/spark/pull/10607#issuecomment-198317091
Sure makes sense - it was my impression that the ML impl would be improved
and I assumed part of that may involve using more of DF/DS, hence my
comment
On Thu, 17 Mar 2016 at 20:58, jkbradley <[email protected]> wrote:
> Thanks for doing this migration. I checked the PR and it LGTM
>
> Your tests look good to me. The tests all seem fairly close, except for a
> couple of outliers, but even those seem within a standard deviation or so
> (the 2nd value in spark-perf results). Thanks for running them!
>
> Also @MLnick <https://github.com/MLnick>
>
> As part of those tickets, I think we can clean up this ML impl and
> interfaces if required (e.g. we could look at removing theprivate [ml]
> train method in favour of one in MLLIb that converts RDDs to DataFrame and
> calls ML, we can make more stuff private where possible, etc). But I think
> it'll be a lot easier to clean things up once everything is in ML.
>
> If the ML implementation uses RDDs underneath, it will be nice to call
> directly into that implementation from spark.mllib in order to avoid
> serialization overhead.
>
> â
> You are receiving this because you were mentioned.
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/10607#issuecomment-198033902>
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
