Liang-Chi Hsieh created SPARK-4913:
--------------------------------------
Summary: Fix incorrect event log path
Key: SPARK-4913
URL: https://issues.apache.org/jira/browse/SPARK-4913
Project: Spark
Issue Type: Bug
Reporter: Liang-Chi Hsieh
SPARK-2261 uses a single file to log events for an app. `eventLogDir` in
`ApplicationDescription` is replaced with `eventLogFile`. However,
`ApplicationDescription` in `SparkDeploySchedulerBackend` is initialized with
`SparkContext`'s `eventLogDir`. It is just the log directory, not the actual
log file path. `Master.rebuildSparkUI` can not correctly rebuild a new SparkUI
for the app.
Because the `ApplicationDescription` is remotely registered with `Master` and
the app's id is then generated in `Master`, we can not get the app id in
advance before registration. So the received description needs to be modified
with correct `eventLogFile` value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
