Github user drcrallen commented on the pull request:
https://github.com/apache/spark/pull/10319#issuecomment-165875292
After more thorough testing, it seems that there is still a race for
getting a `FINISHED` vs `KILLED` final status with this patch.
here's an example log where it was reported as killed:
STDERR
```
15/12/18 19:01:34 INFO CoarseGrainedExecutorBackend: Driver commanded a
shutdown
15/12/18 19:01:34 INFO MemoryStore: MemoryStore cleared
15/12/18 19:01:34 INFO BlockManager: BlockManager stopped
15/12/18 19:01:34 INFO RemoteActorRefProvider$RemotingTerminator: Shutting
down remote daemon.
15/12/18 19:01:34 INFO RemoteActorRefProvider$RemotingTerminator: Remote
daemon shut down; proceeding with flushing remote transports.
15/12/18 19:01:34 INFO RemoteActorRefProvider$RemotingTerminator: Remoting
shut down.
15/12/18 19:01:34 INFO ShutdownHookManager: Shutdown hook called
15/12/18 19:01:34 INFO ShutdownHookManager: Deleting directory
/mesos/slaves/355857ee-1069-480d-baa8-e89427bd5840-S13/frameworks/355857ee-1069-480d-baa8-e89427bd5840-0079/executors/3/runs/b1cb5f55-43f3-4d1d-8ada-18fd29319868/tmp/java/spark-76c38206-57df-46aa-bc2a-4ad143d65959
I1218 19:01:44.844650 94385 exec.cpp:381] Executor asked to shutdown
```
STDOUT
```
2015-12-18T19:01:29.228+0000: Total time for which application threads were
stopped: 0.0000720 seconds, Stopping threads took: 0.0000280 seconds
2015-12-18T19:01:34.418+0000: Total time for which application threads were
stopped: 0.0202110 seconds, Stopping threads took: 0.0170400 seconds
2015-12-18T19:01:34.895+0000: Total time for which application threads were
stopped: 0.0003310 seconds, Stopping threads took: 0.0002190 seconds
2015-12-18T19:01:34.896+0000: Total time for which application threads were
stopped: 0.0001710 seconds, Stopping threads took: 0.0000940 seconds
2015-12-18T19:01:34.907+0000: Total time for which application threads were
stopped: 0.0002100 seconds, Stopping threads took: 0.0001420 seconds
2015-12-18T19:01:34.907+0000: Total time for which application threads were
stopped: 0.0002470 seconds, Stopping threads took: 0.0001740 seconds
2015-12-18T19:01:34.945+0000: Total time for which application threads were
stopped: 0.0004670 seconds, Stopping threads took: 0.0000420 seconds
2015-12-18T19:01:34.961+0000: Total time for which application threads were
stopped: 0.0001110 seconds, Stopping threads took: 0.0000720 seconds
Heap
PSYoungGen total 39748608K, used 18046635K [0x00007f7749000000,
0x00007f8149000000, 0x00007f8149000000)
eden space 37492224K, 46% used
[0x00007f7749000000,0x00007f7b7afe63e8,0x00007f8039580000)
from space 2256384K, 19% used
[0x00007f80bf480000,0x00007f80dac44b78,0x00007f8149000000)
to space 2194432K, 0% used
[0x00007f8039580000,0x00007f8039580000,0x00007f80bf480000)
ParOldGen total 83886080K, used 70680448K [0x00007f6349000000,
0x00007f7749000000, 0x00007f7749000000)
object space 83886080K, 84% used
[0x00007f6349000000,0x00007f7422fe0010,0x00007f7749000000)
PSPermGen total 106496K, used 70405K [0x00007f6339000000,
0x00007f633f800000, 0x00007f6349000000)
object space 106496K, 66% used
[0x00007f6339000000,0x00007f633d4c16e8,0x00007f633f800000)
Shutting down
Sending SIGTERM to process tree at pid 94404
Killing the following process trees:
[
-+- 94404 sh -c "/mnt/tmp/spark/bin/spark-class"
org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url
akka.tcp://sparkDriver@REDACTED/user/CoarseGrainedScheduler --executor-id
355857ee-1069-480d-baa8-e89427bd5840-S13 --hostname REDACTED --cores 13
--app-id 355857ee-1069-480d-baa8-e89427bd5840-0079
\--- 94405 ()
]
Command terminated with signal Terminated (pid: 94404)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
