jeanlyn created SPARK-13845: ------------------------------- Summary: Driver OOM after few days when running streaming Key: SPARK-13845 URL: https://issues.apache.org/jira/browse/SPARK-13845 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.6.1, 1.5.2 Reporter: jeanlyn
We have a streaming job using *FlumePollInputStream* always driver OOM after few days, here is some driver heap dump before OOM {noformat} num #instances #bytes class name ---------------------------------------------- 1: 13845916 553836640 org.apache.spark.storage.BlockStatus 2: 14020324 336487776 org.apache.spark.storage.StreamBlockId 3: 13883881 333213144 scala.collection.mutable.DefaultEntry 4: 8907 89043952 [Lscala.collection.mutable.HashEntry; 5: 62360 65107352 [B 6: 163368 24453904 [Ljava.lang.Object; 7: 293651 20342664 [C ... {noformat} *BlockStatus* and *StreamBlockId* keep on growing, and the driver OOM in the end. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org