I am not sure whether you can copy the log files from Spark workers to your local machine and view it from the Web UI. In fact, if you are able to copy the log files locally, you can just view them directly in any text editor.
I suspect what you really want to see is the application history. Here is the relevant information from Spark’s monitoring page (http://spark.apache.org/docs/latest/monitoring.html) To view the web UI after the fact, set spark.eventLog.enabled to true before starting the application. This configures Spark to log Spark events that encode the information displayed in the UI to persisted storage. Mohammed Author: Big Data Analytics with Spark<http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/> From: Philip Lee [mailto:philjj...@gmail.com] Sent: Monday, January 25, 2016 9:51 AM To: user@spark.apache.org Subject: Re: a question about web ui log As I mentioned before, I am tryint to see the spark log on a cluster via ssh-tunnel 1) The error on application details UI is probably from monitoring porting 4044. Web UI port is 8088, right? so how could I see job web ui view and application details UI view in the web ui on my local machine? 2) still wondering how to see the log after copyting log file to my local. The error was metioned in previous mail. Thanks, Phil On Mon, Jan 25, 2016 at 5:36 PM, Philip Lee <philjj...@gmail.com<mailto:philjj...@gmail.com>> wrote: Hello, a questino about web UI log. I could see web interface log after forwarding the port on my cluster to my local and click completed application, but when I clicked "application detail UI" [Inline image 1] It happened to me. I do not know why. I also checked the specific log folder. It has a log file in it. Actually, that's why I could click the completed application link, right? So is it okay for me to copy the log file in my cluster to my local machine. And after turning on spark Job Manger on my local by myself, I could see application deatils UI in my local machine? Best, Phil
