[
https://issues.apache.org/jira/browse/SPARK-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728680#comment-14728680
]
Andrei Dragomir commented on SPARK-8630:
----------------------------------------
Totally agree with [~asimjalis] comments; We have some integration tests set up
that use QueueInputDStream, as well as a processing topology that has
updateStateByKey (which implies checkpoints). This patch means that that test
suite no longer runs. Is there a work around ? What is the preferred way to
push test data into a spark job which has checkpointing ?
> Prevent from checkpointing QueueInputDStream
> --------------------------------------------
>
> Key: SPARK-8630
> URL: https://issues.apache.org/jira/browse/SPARK-8630
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Reporter: Shixiong Zhu
> Assignee: Shixiong Zhu
> Fix For: 1.4.1, 1.5.0
>
>
> It's better to prevent from checkpointing QueueInputDStream rather than
> failing the application when recovering `QueueInputDStream`, so that people
> can find the issue as soon as possible. See SPARK-8553 for example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
