Re: Checkpoint bugs in GraphX

2014-11-10 Thread GuoQiang Li
: "Xu Lijie";; Date: Tue, Nov 11, 2014 11:40 AM To: "GuoQiang Li"; Cc: "user"; "dev"; Subject: Re: Checkpoint bugs in GraphX Nice, we currently encounter a stackoverflow error caused by this bug. We also found that "val partitionsRDD: R

Re: Checkpoint bugs in GraphX

2014-11-10 Thread Xu Lijie
e: * Tue, Nov 11, 2014 10:19 AM > *To: * "user"; "dev"; > *Subject: * Checkpoint bugs in GraphX > > Hi, all. I'm not sure whether someone has reported this bug: > > > There should be a checkpoint() method in EdgeRDD and VertexRDD as follows: > > ov

Re: Checkpoint bugs in GraphX

2014-11-10 Thread GuoQiang Li
I have been trying to fix this bug.‍ The related PR: https://github.com/apache/spark/pull/2631‍ -- Original -- From: "Xu Lijie";; Date: Tue, Nov 11, 2014 10:19 AM To: "user"; "dev"; Subject: Checkpoint bugs in GraphX Hi, all

Checkpoint bugs in GraphX

2014-11-10 Thread Xu Lijie
Hi, all. I'm not sure whether someone has reported this bug: There should be a checkpoint() method in EdgeRDD and VertexRDD as follows: override def checkpoint(): Unit = { partitionsRDD.checkpoint() } Current EdgeRDD and VertexRDD use *RDD.checkpoint()*, which only checkpoint the edges/vertice