Josh Rosen created SPARK-6716:
---------------------------------
Summary: Change SparkContext.DRIVER_IDENTIFIER from '<driver>' to
'driver'
Key: SPARK-6716
URL: https://issues.apache.org/jira/browse/SPARK-6716
Project: Spark
Issue Type: Bug
Components: Spark Core
Reporter: Josh Rosen
Assignee: Josh Rosen
Currently, the driver's executorId is set to {{<driver>}}. This choice of ID
was present in older Spark versions, but it has started to cause problems now
that executorIds are used in more contexts, such as Ganglia metric names or
driver thread-dump links the web UI. The angle brackets must be escaped when
embedding this ID in XML or as part of URLs and this has led to multiple
problems:
- https://issues.apache.org/jira/browse/SPARK-6484
- https://issues.apache.org/jira/browse/SPARK-4313
The simplest solution seems to be to change this id to something that does not
contain any special characters, such as {{driver}}.
I'm not sure whether we can perform this change in a patch release, since this
ID may be considered a stable API by metrics users, but it's probably okay to
do this in a major release as long as we document it in the release notes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
