Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15737
Thanks, merging to master and 2.1!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68087/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68087 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68087/consoleFull)**
for PR 15737 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68085/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68085 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68085/consoleFull)**
for PR 15737 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68087 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68087/consoleFull)**
for PR 15737 at commit
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15737
Yeah, I'd rather just keep it the same. Failures here should be uncommon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68085 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68085/consoleFull)**
for PR 15737 at commit
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15737
Good catch, I just mistakenly changed to AsMS in one place but not both.
On the test changes, do you want tests waiting up to 2 minutes * however
many kafka calls are being made? If so I
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15737
Two comments:
- I think `sc.conf.getTimeAsSeconds("spark.network.timeout",
"120s").toString` is still wrong. The suggestion is to make it the following:
```scala
val
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68019/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68019 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68019/consoleFull)**
for PR 15737 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68016/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68016 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68016/consoleFull)**
for PR 15737 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68019 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68019/consoleFull)**
for PR 15737 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #68016 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68016/consoleFull)**
for PR 15737 at commit
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15737
Ok, I'll update the default in both places to use spark.network.timeout and
leave the test config at 10 seconds
---
If your project is set up for it, you can reply to this email and have your
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15737
I don't think these cases are comparable, silently swallowing offset out of
range is dropping data on the floor. In contrast, networks _are_ transiently
flakey and we should be robust to that
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15737
I will always choose "fail in an obvious way that I can start debugging"
versus "start behaving poorly in non-obvious ways". Similar reason I
thought it was a really poor idea for the
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15737
If we have prefetched the timeout should never kick in (I hope?), so we
probably only care about the network here. If my choices are "fail the job" or
"wait a little bit longer", I don't see why
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15737
In most cases poll should be returning prefetched data from the buffer, not
waiting to talk to kafka over the network.
I could see increasing it a little bit, but I don't think it
Github user marmbrus commented on the issue:
https://github.com/apache/spark/pull/15737
Is there are reason to not change the user facing default? Its failing our
tests and it was failing for me for real with the current default. When we are
poling on the executors, the offsets
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67993/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #67993 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67993/consoleFull)**
for PR 15737 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #67993 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67993/consoleFull)**
for PR 15737 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #67990 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67990/consoleFull)**
for PR 15737 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15737
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67990/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15737
**[Test build #67990 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67990/consoleFull)**
for PR 15737 at commit
34 matches
Mail list logo
