GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/5372
[SPARK-6716] Change SparkContext.DRIVER_IDENTIFIER from <driver> to driver
Currently, the driver's executorId is set to `<driver>`. This choice of ID
was present in older Spark versions, but it has started to cause problems now
that executorIds are used in more contexts, such as Ganglia metric names or
driver thread-dump links the web UI. The angle brackets must be escaped when
embedding this ID in XML or as part of URLs and this has led to multiple
problems:
- https://issues.apache.org/jira/browse/SPARK-6484
- https://issues.apache.org/jira/browse/SPARK-4313
The simplest solution seems to be to change this id to something that does
not contain any special characters, such as `driver`.
I'm not sure whether we can perform this change in a patch release, since
this ID may be considered a stable API by metrics users, but it's probably okay
to do this in a major release as long as we document it in the release notes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark driver-id-fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5372.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5372
----
commit 7ff12e049377f0bcfe6491a33eea2f020cde77d9
Author: Josh Rosen <[email protected]>
Date: 2015-04-05T20:56:34Z
Change SparkContext.DRIVER_IDENTIFIER from <driver> to driver
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
