Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/15737
I will always choose "fail in an obvious way that I can start debugging"
versus "start behaving poorly in non-obvious ways". Similar reason I
thought it was a really poor idea for the original PRs for structured
streaming kafka to silently swallow offset out of range: something is
wrong, and I want to know about it.
Ultimately this particular thing is tunable, so if you really want to
increase the default to however many seconds let me know - it won't affect
my ability to do my job.
On Wed, Nov 2, 2016 at 1:42 PM, Michael Armbrust <notificati...@github.com>
wrote:
> If we have prefetched the timeout should never kick in (I hope?), so we
> probably only care about the network here. If my choices are "fail the
job"
> or "wait a little bit longer", I don't see why you would ever chose "fail
> the job" for transient network issues. If its not a transient issue, do
you
> really think that knowing about it 9.5 seconds earlier outweighs that cost
> of false negatives?
>
> I guess I don't understand what kind of Kafka issues you are optimizing
> for?
>
> â
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/spark/pull/15737#issuecomment-257960644>, or
mute
> the thread
>
<https://github.com/notifications/unsubscribe-auth/AAGAB9_f5Jaae1inqEK4uTw45cgPdQ1Hks5q6NmfgaJpZM4KnW2M>
> .
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
