Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/15407
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15407
Thanks for the explanation. Merging in master/branch-2.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15407
@rxin @tdas right now, items to be committed can be added to the queue, but
they will never actually be removed from the queue. poll() removes, iterator()
does not. I updated the description of
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15407
@koeninger can you put more information into the description of the pull
request? At the very least we should talk about the current implementation
causes memory leaks.
---
If your project is set
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/15407
Can you explain what is the memory that can currently leak with the
iterator?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15407
This doesn't affect correctness (only the highest offset for a given
partition is used in any case), just memory leaks. I'm not sure what a good
way to unit test memory leaks is, short of
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15407
Is there a test we can add to cover this?
I realized "How was this patch tested?" was confusing. That was meant to
ask how is the change tested itself, and a bug fix obviously can't be tested
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15407
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15407
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66609/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15407
**[Test build #66609 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66609/consoleFull)**
for PR 15407 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15407
**[Test build #66609 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66609/consoleFull)**
for PR 15407 at commit
11 matches
Mail list logo
