This would be useful to us, so I've created a JIRA ticket for this
discussion: https://issues.apache.org/jira/browse/SPARK-24122
On Wed, Mar 28, 2018 at 10:28 AM, Anirudh Ramanathan <
ramanath...@google.com.invalid> wrote:
> We discussed this early on in our fork and I think we should have this
We discussed this early on in our fork and I think we should have this in a
JIRA and discuss it further. It's something we want to address in the
future.
One proposed method is using a StatefulSet of size 1 for the driver. This
ensures recovery but at the same time takes away from the completion
A carry-over from the apache-spark-on-k8s project, it would be useful to
have a configurable restart policy for submitted jobs with the Kubernetes
resource manager. See the following issues:
https://github.com/apache-spark-on-k8s/spark/issues/133
