[
https://issues.apache.org/jira/browse/SPARK-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das resolved SPARK-1592.
----------------------------------
Resolution: Fixed
https://github.com/apache/spark/pull/512
> Old streaming input blocks not removed automatically from the BlockManagers
> ---------------------------------------------------------------------------
>
> Key: SPARK-1592
> URL: https://issues.apache.org/jira/browse/SPARK-1592
> Project: Spark
> Issue Type: Bug
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Priority: Blocker
>
> The raw input data is stored as blocks in BlockManagers. Earlier they were
> cleared by cleaner ttl. Now since streaming does not require cleaner TTL to
> be set, the block would not get cleared. This increases up the Spark's memory
> usage, which is not even accounted and shown in the Spark storage UI. It may
> cause the data blocks to spill over to disk, which eventually slows down the
> receiving of data (persisting to memory become bottlenecked by writing to
> disk).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
