[ https://issues.apache.org/jira/browse/SPARK-25448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625093#comment-16625093 ]
shahid edited comment on SPARK-25448 at 9/23/18 12:51 PM: ---------------------------------------------------------- I am not able to reproduce the issue with spark2.3.1 Test steps: 1) add configuration "spark.ui.retainedJobs=1400" in the spark-defaults.conf 2) re-start history server sbin/start-historyserver.sh 3) launch spark-shell bin/spark-shell {code:java} val rdd = sc.parallelize(1 to 5, 5) for(i <- 1 to 2000){ rdd.count } {code} 4) Open application UI from the historyserver page !image-2018-09-23-18-01-35-554.png! So, mostly it is a configuration related issue. was (Author: shahid): I am not able to reproduce the issue with spark2.3.1 Test steps: 1) add configuration "spark.ui.retainedJobs=1400" in the spark-defaults.conf 2) re-start history server sbin/start-historyserver.sh 3) launch spark-shell bin/spark-shell {code:java} val rdd = sc.parallelize(1 to 5, 5) for(i <- 1 to 2000){ rdd.count } {code} 4) Open application UI from the historyserver page !image-2018-09-23-18-01-35-554.png! > [Spark Job History] Job Staged page shows 1000 Jobs only > -------------------------------------------------------- > > Key: SPARK-25448 > URL: https://issues.apache.org/jira/browse/SPARK-25448 > Project: Spark > Issue Type: Bug > Components: Web UI > Affects Versions: 2.3.1 > Environment: Server OS :-SUSE 11 > No. of Cluster Node:- 6 > Spark Version:- 2.3.1 > Reporter: ABHISHEK KUMAR GUPTA > Priority: Major > Attachments: image-2018-09-23-18-01-35-554.png > > > 1. configure spark.ui.retainedJobs = 100000 in spark-default.conf file of Job > History > 2. Submit 1 Lakh job from beeline > 3. Go to the application ID from Job History Page " Incomplete Application > Link" > 4. Job tab will list only max 1000 jobs under the application > Actual output > Completed Jobs: 24952, only showing 952 > Staged page should list all completed Jobs in this case 24952 -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org