Ahmed Mahran created SPARK-16160: ------------------------------------ Summary: Last remembered metadata window per dstream is not cleared upon context graceful stop Key: SPARK-16160 URL: https://issues.apache.org/jira/browse/SPARK-16160 Project: Spark Issue Type: Bug Components: Streaming Reporter: Ahmed Mahran Priority: Minor
When stopping a streaming context gracefully, the last remembered window for each dstream is not cleared. If the streaming context is stopped without stopping the spark context, the persisted garbage rdds would still be there especially when there is a relatively large remember duration in case of windowing or checkpointing. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org