GitHub user punya opened a pull request:
https://github.com/apache/spark/pull/5535
[SPARK-6952] Handle long args when detecting PID reuse
sbin/spark-daemon.sh used
ps -p "$TARGET_PID" -o args= to figure out
whether the process running with the expected PID is actually a Spark
daemon. When running with a large classpath, the output of ps gets
truncated and the check fails spuriously.
This weakens the check to see if it's a java command (which is something
we do in other parts of the script) rather than looking for the specific
main class name. This means that SPARK-4832 might happen under a
slightly broader range of circumstances (a java program happened to
reuse the same PID), but it seems worthwhile compared to failing
consistently with a large classpath.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/punya/spark feature/SPARK-6952
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5535.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5535
----
commit 7ea12d1e6746d9cdf57fb657a015200267649ca7
Author: Punya Biswal <[email protected]>
Date: 2015-04-15T23:55:41Z
Handle long args when detecting PID reuse
sbin/spark-daemon.sh used
ps -p "$TARGET_PID" -o args= to figure out
whether the process running with the expected PID is actually a Spark
daemon. When running with a large classpath, the output of ps gets
truncated and the check fails spuriously.
This weakens the check to see if it's a java command (which is something
we do in other parts of the script) rather than looking for the specific
main class name. This means that SPARK-4832 might happen under a
slightly broader range of circumstances (a java program happened to
reuse the same PID), but it seems worthwhile compared to failing
consistently with a large classpath.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
