Github user stp008 commented on the issue:
https://github.com/apache/spark/pull/16629
actually if u turn it off then for each batch new consumer will be created.
For me the solution is to introduce pool of consumers equal to the total count
of concurrent job for example
---
If your
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/16629
I don't think it's a problem to make disabling the cache configurable, as
long as it's on by default. I don't think the additional static constructors in
kafka utils are necessary, are they?
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16629
@srowen Yes, this pr does not provide a way to support `ConsumerCache` in
multi-thread, but give users a more clear hint to this issue. I think it may be
more complex before we achieve a better
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/16629
I don't think this resolves the problem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16629
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16629
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71562/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16629
**[Test build #71562 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71562/testReport)**
for PR 16629 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16629
**[Test build #71562 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71562/testReport)**
for PR 16629 at commit
Github user uncleGen commented on the issue:
https://github.com/apache/spark/pull/16629
cc @zsxwing and @koeninger
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
