[ https://issues.apache.org/jira/browse/SPARK-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-5899. ------------------------------ Resolution: Duplicate > Viewing specific stage information which contains thousands of tasks will > freak out the driver and CPU cores from where it runs > ------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-5899 > URL: https://issues.apache.org/jira/browse/SPARK-5899 > Project: Spark > Issue Type: Bug > Components: Web UI > Affects Versions: 1.3.0, 1.2.1 > Environment: CentOS 7, Spark Standalone > Reporter: Mark Khaitman > Priority: Critical > > If a user ever tries to view specific stage stats, for example, a repartition > stage which involves 30000 partitions, the Web UI attempts to load every > single task result onto a single webpage, which is completely destroying CPU > usage on the driver, which subsequently causes the remaining tasks / jobs to > be nearly impossible to complete. > Ideally the task results should be paged (if not too much trouble) to prevent > this from happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org