[
https://issues.apache.org/jira/browse/SPARK-20251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960671#comment-15960671
]
Sean Owen commented on SPARK-20251:
-----------------------------------
This depends on too many things, like how you've set up your app and what
recovery semantics you've implemented. I'd close this as something that should
start with more research and a question on the user@ list.
> Spark streaming skips batches in a case of failure
> --------------------------------------------------
>
> Key: SPARK-20251
> URL: https://issues.apache.org/jira/browse/SPARK-20251
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.0
> Reporter: Roman Studenikin
>
> We are experiencing strange behaviour of spark streaming application.
> Sometimes it just skips batch in a case of job failure and starts working on
> the next one.
> We expect it to attempt to reprocess batch, but not to skip it. Is it a bug
> or we are missing any important configuration params?
> Screenshots from spark UI:
> http://pasteboard.co/1oRW0GDUX.png
> http://pasteboard.co/1oSjdFpbc.png
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
