Hello All, I am running the history server for a completed application. This application was run with the following parameters
bin/spark-submit --class <class_name> --master local[2] --conf spark.local.dir=/mnt/ --conf spark.eventLog.dir=/mnt/sparklog/ --conf spark.eventLog.enabled=true --conf spark.ui.retainedJobs=10000 --conf spark.ui.retainedStages=10000 <jar_name> In the Spark Web UI (http://localhost:18080/), the DAG visualization of only the most recent job is available. For rest of the jobs, I get the following message No visualization information available for this job! If this is an old job, its visualization metadata may have been cleaned up over time. You may consider increasing the value of spark.ui.retainedJobs and spark.ui.retainedStages. I did increase the retainedJobs and retainedStages value to 10,000. What else should be done to retain the visualization information of all the jobs/stages? Thanks in advance. Raghava. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/DAG-visualization-no-visualization-information-available-with-history-server-tp26117.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org