[ https://issues.apache.org/jira/browse/SPARK-7942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen updated SPARK-7942: ----------------------------- Assignee: Tathagata Das > Receiver's life cycle is inconsistent with streaming job. > --------------------------------------------------------- > > Key: SPARK-7942 > URL: https://issues.apache.org/jira/browse/SPARK-7942 > Project: Spark > Issue Type: Bug > Components: Streaming > Affects Versions: 1.4.0 > Reporter: SaintBacchus > Assignee: Tathagata Das > Fix For: 1.5.0 > > > Streaming consider the receiver as a common spark job, thus if an error > occurs in the receiver's logical(after 4 times(default) retries ), streaming > will no longer get any data but the streaming job is still running. > A general scenario is that: we config the > `spark.streaming.receiver.writeAheadLog.enable` as true to use the > `ReliableKafkaReceiver` but do not set the checkpoint dir. Then the receiver > will soon be shut down but the streaming is alive. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org