Re: ALS checkpoint performance

2014-08-15 Thread Xiangrui Meng
Guoqiang reported some results in his PRs
https://github.com/apache/spark/pull/828 and
https://github.com/apache/spark/pull/929 . But this is really
problem-dependent. -Xiangrui

On Fri, Aug 15, 2014 at 12:30 PM, Debasish Das  wrote:
> Hi,
>
> Are there any experiments detailing the performance hit due to HDFS
> checkpoint in ALS ?
>
> As we scale to large ranks with more ratings, I believe we have to cut the
> RDD lineage to safe guard against the lineage issue...
>
> Thanks.
> Deb
>

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org



ALS checkpoint performance

2014-08-15 Thread Debasish Das
Hi,

Are there any experiments detailing the performance hit due to HDFS
checkpoint in ALS ?

As we scale to large ranks with more ratings, I believe we have to cut the
RDD lineage to safe guard against the lineage issue...

Thanks.
Deb