Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15125
merged to master/2.2
I talked to Ankur about this change offline.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76088/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #76088 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76088/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #76088 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76088/testReport)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76069/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #76069 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76069/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #76069 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76069/testReport)**
for PR 15125 at commit
Github user dding3 commented on the issue:
https://github.com/apache/spark/pull/15125
OK, agreed. If user didn't set checkpointer directory while we turn on
checkpoint in pregel by default, there may be exception. I will change
spark.graphx.pregel.checkpointInterval to -1 as default
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15125
@dding3 @mallman one last thing to consider, with this PR we are having
`spark.graphx.pregel.checkpointInterval`
defaulting to 10. This changes the existing behavior (where
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15125
thanks @mallman I'll digest this and see what improvements we could make in
gf.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/76003/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #76003 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76003/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #76003 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/76003/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #75968 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75968/testReport)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75968/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #75968 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75968/testReport)**
for PR 15125 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
> I think we want to bring GraphFrames to feature/performance parity with
GraphX - @mallman would love to understand the challenges you have run into.
Better yet, would be great to get some issues
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15125
I think we want to bring GraphFrames to feature/performance parity with
GraphX - @mallman would love to understand the challenges you have run into.
Better yet, would be great to get some
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15125
I didn't forget about this ;) I was hoping others could review.
I will do another pass today - I believe at this point we have quorum with
the number of reviewers who have looked at this PR.
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
> Yeah, catch-22. I'd also like to split out graphx, but I sorta think
that's what's already happened with GraphFrames. I don't feel strongly enough
to campaign for it, but think graphx should
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15125
Yeah, catch-22. I'd also like to split out graphx, but I sorta think that's
what's already happened with GraphFrames. I don't feel strongly enough to
campaign for it, but think graphx should
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
> I feel like i really don't know anything about graphx and can't evaluate
this. It seems reasonable. I don't know if graphx is really active at this
stage?
Understood. Let me respond to
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15125
I feel like i really don't know anything about graphx and can't evaluate
this. It seems reasonable. I don't know if graphx is really active at this
stage?
---
If your project is set up for it, you
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@srowen Can you merge this PR, please? It's been over a month since we've
heard from any of the reviewers.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@felixcheung ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@felixcheung We haven't heard from @jkbradley or @ankurdave in a week.
Should we give them more time or can we merge to master?
---
If your project is set up for it, you can reply to this email
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15125
@jkbradley @ankurdave would you like to review?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@felixcheung Can you take another look and merge if LGTY? I think we've
addressed all of the open reviewer requests.
---
If your project is set up for it, you can reply to this email and have your
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73390/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73390 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73390/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73390 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73390/testReport)**
for PR 15125 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@dding3, thank you for your continued patience and dedication to this PR,
despite the continued change requests. We are getting closer to a merge.
---
If your project is set up for it, you can
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73150/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73148/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73150 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73150/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73148 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73148/testReport)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73138/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73138 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73138/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73138 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73138/testReport)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73070/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73070 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73070/testReport)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73070 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73070/testReport)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73065 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73065/testReport)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73065/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73065 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73065/testReport)**
for PR 15125 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
LGTM. @felixcheung are we good to merge?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73019/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73019 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73019/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #73019 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73019/testReport)**
for PR 15125 at commit
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@dding3 I submitted a PR against your `cp2_pregel` branch. If you merge
that PR into your branch, it will be reflected in this PR. This is my PR:
https://github.com/dding3/spark/pull/1.
---
If
Github user dding3 commented on the issue:
https://github.com/apache/spark/pull/15125
Both are OK for me. Please let me know if any update needed from me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
> I think @mallman is saying he would merge changes to @dding3 branch
Yes, or I could do them in a follow up PR. Or @dding3 could do them without
my PR. I'm not hung up on getting credit.
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/15125
I think @mallman is saying he would merge changes to @dding3 branch
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/15125
@mallman Looks good to me. Wait for any committers to check this further.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
Our connected components computation completed successfully, with
performance as expected. I've created a PR against @dding3's PR branch to
incorporate a couple simple things. Then I think we're
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@dding3 These latest changes look great. I'll run our big connected
components job today and report back.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72748/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #72748 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72748/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #72748 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72748/testReport)**
for PR 15125 at commit
Github user dding3 commented on the issue:
https://github.com/apache/spark/pull/15125
Thank you guys for reviewing the code. I have updated it based on the
comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
@viirya @dding3 I'm going to rerun our big connected components computation
with the changes I've suggested to validate that it still performs and
completes as expected. Given the time required to
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/15125
@mallman Thanks for ping me on this. The changes looks good to me overall.
The most issue might be as @srowen said, I don't know if this is actively
maintained still.
I saw most recently
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15125
I don't know. @ankurdave ? I am not sure this is being maintained in any
meaningful sense now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
LGTM. @srowen, can you recommend an mllib committer to review these
changes? I'm not familiar with that team.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71556/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #71556 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71556/testReport)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #71556 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71556/testReport)**
for PR 15125 at commit
Github user dding3 commented on the issue:
https://github.com/apache/spark/pull/15125
Sure. I will work on the rebase and update the PR soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mallman commented on the issue:
https://github.com/apache/spark/pull/15125
Hi @dding3. Thanks for working on this! I was able to rebase and apply your
patch to our build of Spark 2.1 to successfully compute the connected
components of a graph with 5.2 billion vertices,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65523/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65523 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65523/consoleFull)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65523 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65523/consoleFull)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65522 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65522/consoleFull)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65522/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65522 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65522/consoleFull)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65521/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65521 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65521/consoleFull)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65521 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65521/consoleFull)**
for PR 15125 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65519 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65519/consoleFull)**
for PR 15125 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15125
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65519/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15125
**[Test build #65519 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65519/consoleFull)**
for PR 15125 at commit
98 matches
Mail list logo
