Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/15961
This looks good to me pending @rxins proposed wording change. I think
restarting a stopped SparkContext which has been killed by an OOM killer or
other issue is probably not a good thing to generally do and the warning text
makes it clear enough that the machine may be in a bad state so the user can
investigate if necessary.
While I'm unlikely to use this, it sounds like it could make life easier
for some notebook users to avoid having to restart their notebook kernel.
Thanks for taking the time on this PR @kxepal :)
Note: that if you do have automatic retry logic that likely should not be
implemented this way - there is a chance this might lead to a very bad state
(hence the warning message). [Just mentioning this since it was mentioned as
one of the possible uses in our discussion].
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
