[
https://issues.apache.org/jira/browse/SPARK-10052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699482#comment-14699482
]
Sean Owen commented on SPARK-10052:
-----------------------------------
You can't be sure the stage doesn't have side-effects, even if the input size
is 0. I believe we've had similar discussions before. You still need to launch
a job, but, the job can be a no-op.
> KafKaDirectDstream should filter empty partition task or rdd
> ------------------------------------------------------------
>
> Key: SPARK-10052
> URL: https://issues.apache.org/jira/browse/SPARK-10052
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.4.1
> Reporter: SuYan
>
> We run spark 1.4.0 spark direct streaming, found it submit stages and tasks
> for input event = 0 events.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
