Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-202653414
OK, I will fill a JIRA later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11779#issuecomment-202649212
Sure. I will submit a patch against branch-1.6
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/11779
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-202632164
Also about the history server test, yes it's OK to just generate a new
event log. The old tests were written in such a way that's difficult to
maintain anyway.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11779#issuecomment-202632288
Thanks merging into master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-202631390
OK, thanks for the explanation. Would you mind filing a JIRA for making
sure block manager correctly updates `updatedBlockStatuses`?
---
If your project is set up
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197727915
**[Test build #53398 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53398/consoleFull)**
for PR 11679 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11779#issuecomment-197836978
**[Test build #53413 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53413/consoleFull)**
for PR 11779 at commit
GitHub user jeanlyn opened a pull request:
https://github.com/apache/spark/pull/11779
[SPARK-13845][CORE]Using onBlockUpdated to replace onTaskEnd avioding
driver OOM
## What changes were proposed in this pull request?
We have a streaming job using `FlumePollInputStream`
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11779#issuecomment-197837954
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197753298
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11779#issuecomment-197785111
**[Test build #53413 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53413/consoleFull)**
for PR 11779 at commit
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11779#issuecomment-197786170
This PR is the same as #11679 , but i came across with some accidents when
rebasing the PR. So i create a new one.
/cc @andrewor14
---
If your project is set up
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197674341
All test failure is relevant with `HistoryServerSuite`, the reason is we
remove the `onTaskEnd`, and it's used to replay the storage page of history
server from the
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197841436
**[Test build #53411 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53411/consoleFull)**
for PR 11679 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197753302
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197778717
close this for accident
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197842015
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197752963
**[Test build #53398 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53398/consoleFull)**
for PR 11679 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197778519
**[Test build #53411 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53411/consoleFull)**
for PR 11679 at commit
Github user jeanlyn closed the pull request at:
https://github.com/apache/spark/pull/11679
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11779#issuecomment-197837947
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197842009
Build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-197127323
I think the the metrics of `updatedBlockStatuses` does not updated using
code like
```
c.taskMetrics().incUpdatedBlockStatuses(Seq((blockId, status)))
```
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196983237
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196983240
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196982953
**[Test build #53207 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53207/consoleFull)**
for PR 11679 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196941095
**[Test build #53207 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53207/consoleFull)**
for PR 11679 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196940321
I agree that this fixes the solution, but it also masks the problem that
`metrics.updatedBlockStatuses` is updated correctly. I think before we proceed
with the
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196939899
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196617986
@andrewor14 It seems the MIMA failure do not relevant with this patch. Do i
need to fix it in this patch?
---
If your project is set up for it, you can reply to this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196597574
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196597573
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196597542
**[Test build #53142 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53142/consoleFull)**
for PR 11679 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196592483
**[Test build #53142 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53142/consoleFull)**
for PR 11679 at commit
Github user jeanlyn commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196591027
Thanks @andrewor14 for review. We encounter this issue in branch-1.5, and I
had noticed recent changes of metrics. If i understand correctly, I think the
root cause of
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196440993
@jeanlyn Thanks for submitting the patch. It seems strange that
`metrics.updatedBlockStatuses` is not being updated. On what Spark versions did
you observe this
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196434358
add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196438375
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196438360
**[Test build #53082 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53082/consoleFull)**
for PR 11679 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196438386
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196436662
**[Test build #53082 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53082/consoleFull)**
for PR 11679 at commit
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/11679#issuecomment-196132369
cc @andrewor14
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
43 matches
Mail list logo
