Bhanuday Birla created SPARK-18486:
--------------------------------------
Summary: Access of SparkUI after task completion
Key: SPARK-18486
URL: https://issues.apache.org/jira/browse/SPARK-18486
Project: Spark
Issue Type: Question
Components: Optimizer, Structured Streaming, Web UI
Affects Versions: 2.0.1
Environment: Windows server 2008 R2
Spark Version: 2.0.1
Running locally and accessing sparkUI on por 4040
Reporter: Bhanuday Birla
Priority: Blocker
Fix For: 2.0.1
Hi,
I am running spark scripts in on local machine. In order to see completion
times of different jobs, I want to access SparkUI on port 4040. But i can
access it while the script is running. After completion of the script, it's not
accessible. To resolve this I have set spark.eventLog.enabled property to true
inside my application.
conf
=SparkConf().setMaster("local[4]").setAppName("IAP").set("spark.eventLog.enabled",
'true')
Also I have created a folder named "spark-events" storing history logs which
can be picked up by SparkUI. But still no luck. I am working on a project in my
firm and get stuck here. Please help me in any possible way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
