GitHub user devaraj-kavali opened a pull request:
https://github.com/apache/spark/pull/21996
[SPARK-24888][CORE] spark-submit --master spark://host:port --status
driver-id does not work
## What changes were proposed in this pull request?
In `SparkSubmit.scala` (`val uninitLog = initializeLogIfNecessary(true,
silent = true)`) -> `Logging.scala` (`val replLevel =
Option(replLogger.getLevel()).getOrElse(Level.WARN)`), the log level for
`rootLogger ` is overiri to `WARN ` but the status of the driver and kill
driver commands status have been logging with `INFO ` log level, so there is
nothing printing on the console for status and kill commands. This PR overrides
the `logInfo()` for `RestSubmissionClient` and redirects the msg to
`printStream`.
## How was this patch tested?
I verified it manually by running status and kill commands, these are the
results with and without the PR change.
- Without the PR change
```
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --status driver-20180803165641-0000
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --kill driver-20180803165641-0000
```
- With the PR change
```
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --kill driver-20180803165641-0000
Submitting a request to kill submission driver-20180803165641-0000 in
spark://user1-work-pc:6066.
Server responded with KillSubmissionResponse:
{
"action" : "KillSubmissionResponse",
"message" : "Driver driver-20180803165641-0000 has already finished or
does not exist",
"serverSparkVersion" : "2.4.0-SNAPSHOT",
"submissionId" : "driver-20180803165641-0000",
"success" : false
}
[user1@user1-work-pc bin]$ ./spark-submit --master
spark://user1-work-pc:6066 --status driver-20180803165641-0000
Submitting a request for the status of submission
driver-20180803165641-0000 in spark://user1-work-pc:6066.
Server responded with SubmissionStatusResponse:
{
"action" : "SubmissionStatusResponse",
"driverState" : "FINISHED",
"serverSparkVersion" : "2.4.0-SNAPSHOT",
"submissionId" : "driver-20180803165641-0000",
"success" : true,
"workerHostPort" : "xx.x.xx.xxx:42040",
"workerId" : "worker-20180803165615-10.3.66.149-42040"
}
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/devaraj-kavali/spark SPARK-24888
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21996.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21996
----
commit 1e8000ffacabac742d16efee72ec1e421225a272
Author: Devaraj K <devaraj@...>
Date: 2018-08-04T00:47:23Z
[SPARK-24888][CORE] spark-submit --master spark://host:port --status
driver-id does not work
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
