[ https://issues.apache.org/jira/browse/SPARK-25594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17150729#comment-17150729 ]
Mridul Muralidharan commented on SPARK-25594: --------------------------------------------- Given regression in functionality if this is merged, closing bug. See comment: https://github.com/apache/spark/pull/22609#issuecomment-426405757 > OOM in long running applications even with UI disabled > ------------------------------------------------------ > > Key: SPARK-25594 > URL: https://issues.apache.org/jira/browse/SPARK-25594 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.3.0, 2.4.0 > Reporter: Mridul Muralidharan > Assignee: Mridul Muralidharan > Priority: Major > > Typically for long running applications with large number of tasks it is > common to disable UI to minimize overhead at driver. > Earlier, with spark ui disabled, only stage/job information was kept as part > of JobProgressListener. > As part of history server scalability fixes, particularly SPARK-20643, > inspite of disabling UI - task information continues to be maintained in > memory. > In our long running tests against spark thrift server, this eventually > results in OOM. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org