GitHub user tdas opened a pull request:
https://github.com/apache/spark/pull/3455
[SPARK-4601][Streaming] Set correct call site for streaming jobs so that it
is displayed correctly on the Spark UI
When running the NetworkWordCount, the description of the word count jobs
are set as "getCallsite at DStream:xxx" . This should be set to the line number
of the streaming application that has the output operation that led to the job
being created. This is because the callsite is incorrectly set in the thread
launching the jobs. This PR fixes that.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tdas/spark streaming-callsite-fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3455.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3455
----
commit 69fc26fd870361c345f6482737d6192949ab46b4
Author: Tathagata Das <[email protected]>
Date: 2014-11-25T13:08:32Z
Set correct call site for streaming jobs so that it is displayed correctly
on the Spark UI
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
