Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-167711102
@srowen, thanks for the reply. I tested it in my yarn environment, it looks
more stable now.
In current implementation, restart will start new socket in new thread in a
fixed sleep interval and old socket resource will be released in finally block
in old thread, in some rare cicumastance old socket resource may not be
released properly(not sure why it happened, after 10 hours or sometimes 36 hous
continous running).
In this PR, I suggest to move the socket release to Receive.onStop(), it
will be explictly invoked and release the socket resource before start new
Socket connection, should be safer and keep the same behaviour like other
Receivers and release resource in onStop() method.
@zsxwing, @jerryshao
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
