[
https://issues.apache.org/jira/browse/SPARK-10955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-10955:
----------------------------------
Description: Spark streaming can be tricky with dynamic allocation and can
lose data if not used properly (with WAL, or with WAL-free solutions like
Direct Kafka and Kinesis since 1.5). If dynamic allocation is enabled, we
should issue a log4j warning. (was: Spark streaming can be tricky with dynamic
allocation and can lose dataWe should disable dynamic allocation or at least
log that it is dangerous.)
> Warn if dynamic allocation is enabled for Streaming jobs
> --------------------------------------------------------
>
> Key: SPARK-10955
> URL: https://issues.apache.org/jira/browse/SPARK-10955
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Reporter: Hari Shreedharan
> Assignee: Hari Shreedharan
> Fix For: 1.5.2, 1.6.0
>
>
> Spark streaming can be tricky with dynamic allocation and can lose data if
> not used properly (with WAL, or with WAL-free solutions like Direct Kafka and
> Kinesis since 1.5). If dynamic allocation is enabled, we should issue a log4j
> warning.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
