Github user neggert commented on the issue:
https://github.com/apache/spark/pull/12576
Not sure what's going on regarding the multiple PRs for this issue, but I
cherry-picked this PR on top of 1.6.2 and it fixed the problem for me.
---
If your project is set up for it, you can reply
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/12576
Okay so can you update this PR according to the discussion, and add the
necessary tests as well?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user suyanNone commented on the issue:
https://github.com/apache/spark/pull/12576
@tdas agree, `isCheckpointed` should be final, in current code,
`isCheckpointed` exposed as public is for testing?
---
If your project is set up for it, you can reply to this email and have
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/12576
CC: @ankurdave just making you aware of this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/12576
Actually, correction. Here is a better fix. The key problem here is that
the RDD internal logic for computing from checkpoints, depends on a public,
override-able API. This is wrong. Either the
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/12576
also, ping @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/12576
This fix is roughly in the right direction. However there are two major
concerns.
- It changes the behavior of VertexRDD.isCheckpointed() which is a public
method.
- There must be new tests
