Github user BryanCutler commented on the pull request:
https://github.com/apache/spark/pull/12481#issuecomment-211601590
LGTM. I noticed this problem in testing out the DriverWrapper for another
issue and was looking for a fix. After applying this patch, the Driver will
complete with a FINISHED status instead of FAILED (tested on a local cluster).
I did notice some new warnings in the Driver log though, but doesn't seem
to cause a problem
```
WARN Dispatcher: Message
RemoteProcessDisconnected(oc7258617548.ibm.com:7077) dropped. RpcEnv already
stopped.
```
And the app log still has an error that the worker watcher was disconnect,
but maybe that is a separate issue and it doesn't seem to affect the
application final state
```
16/04/18 14:55:57 INFO CoarseGrainedExecutorBackend: Driver commanded a
shutdown
16/04/18 14:55:57 INFO MemoryStore: MemoryStore cleared
16/04/18 14:55:57 INFO BlockManager: BlockManager stopped
16/04/18 14:55:57 ERROR WorkerWatcher: Lost connection to worker rpc
endpoint spark://[email protected]:39166. Exiting.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
