Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21884#discussion_r206999720
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala
---
@@ -203,4 +212,12
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r207016169
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r207073149
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -669,6 +686,34 @@ private[spark] class AppStatusListener
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21923#discussion_r207072577
--- Diff: core/src/main/java/org/apache/spark/AbstractExecutorPlugin.java
---
@@ -0,0 +1,39 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r207004997
--- Diff:
core/src/main/scala/org/apache/spark/executor/ExecutorMetrics.scala ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21884
Thanks @shaneknapp, I'll merge this now.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21884
Thanks @adelbertc for the contribution!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r207002859
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -669,6 +686,34 @@ private[spark] class AppStatusListener
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r205849187
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/ClientModeTestsSuite.scala
---
@@ -0,0
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21900
@skonto can you try this with spark-rbacyaml?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user mccheah opened a pull request:
https://github.com/apache/spark/pull/21900
[SPARK-23146][K8S][TESTS] Don't set service account name for client mode
test
## What changes were proposed in this pull request?
Don't set service account name for the pod created
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21900
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21884
This needs a Spark ticket. I also don't think pods specifically have to
expose ports. My understanding was that the ports field on the pod was only to
assign ports names, so that the ports can
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209712159
--- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala
---
@@ -180,6 +180,26 @@ private[spark] abstract class MemoryManager
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209717523
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -669,6 +686,31 @@ private[spark] class AppStatusListener
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209714883
--- Diff:
core/src/main/scala/org/apache/spark/executor/ExecutorMetrics.scala ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209716819
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -669,6 +686,31 @@ private[spark] class AppStatusListener
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209714796
--- Diff:
core/src/main/scala/org/apache/spark/executor/ExecutorMetrics.scala ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209715208
--- Diff:
core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ---
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209715001
--- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala
---
@@ -180,6 +180,26 @@ private[spark] abstract class MemoryManager
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209711557
--- Diff:
core/src/main/scala/org/apache/spark/executor/ExecutorMetrics.scala ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21923#discussion_r209805442
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -130,6 +130,12 @@ private[spark] class Executor(
private val
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21221#discussion_r209773404
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -669,6 +686,31 @@ private[spark] class AppStatusListener
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21221
@edwinalu I think this is almost ready, can you fix merge conflicts?
@squito @felixcheung any other comments
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21584
Let's merge at the end of the day pacific time (~5PM-ish) on Friday, August
17, pending any additional feedback on the mailing list thread discussing the
subject of including this in 2.4
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22146
@skonto @erikerlandson
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21221
We're going to delay on merging this until after the 2.4 branch is cut. We
can include this in Spark 2.5.
---
-
To unsubscribe
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211802430
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/TemplateVolumeStep.scala
---
@@ -0,0 +1,51
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211800379
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -96,3 +112,25
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211800127
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/TemplateVolumeStep.scala
---
@@ -0,0 +1,51
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211800821
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -51,7 +57,13 @@ private
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211799242
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala
---
@@ -59,5 +65,23 @@ private[spark] object
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211801962
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/TemplateVolumeStep.scala
---
@@ -0,0 +1,51
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211800415
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -16,11 +16,17
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211800326
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -96,3 +112,25
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r211800176
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/TemplateVolumeStep.scala
---
@@ -0,0 +1,51
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212126595
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -51,7 +57,13 @@ private
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212396719
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala
---
@@ -0,0 +1,71
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212396608
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala
---
@@ -0,0 +1,71
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212400281
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -81,13 +88,17
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212395959
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -225,6 +225,18 @@ private[spark] object Config
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212396387
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala
---
@@ -59,5 +65,21 @@ private[spark] object
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212448189
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -225,6 +225,18 @@ private[spark] object Config
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r212722242
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -225,6 +225,18 @@ private[spark] object Config
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21584
I filed https://issues.apache.org/jira/browse/SPARK-25152 for the
integration tests.
---
-
To unsubscribe, e-mail: reviews
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21584
Ok I am merging this to master now. Thanks for the work on this!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22146
@yifeih can we also modify `docs/running-on-kubernetes.md` and have a
section specific to using this feature? Mark it as experimental also
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22146
Jenkins, ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
> Yes, the service gets its endpoints by matching its label selector
against labels on the pods so it's critical to have matching labels. Another
tenable solution is for the driver backend c
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
I think taking a step back, it seems unwise more so to be making any
assumptions about the location in which a driver is running in client mode.
Client mode is simply just saying
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Anyone know what's happening with this:
```
[error]
/home/jenkins/workspace/testing-k8s-prb-make-spark-distribution-unified/external/avro/src/test/scala/org/apache/spark/sql/avro
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
We discussed this offline. After some experimentation, we concluded that
it's not actually straightforward to set up the headless service in the
Kubernetes scheduler code in client mode, which
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Never mind, think it's recovering now.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@liyinan926 did some of my own edits on top of your suggestions for docs
wording on the latest patch.
---
-
To unsubscribe, e
GitHub user mccheah opened a pull request:
https://github.com/apache/spark/pull/21800
[SPARK-24825][K8S][TEST] Kubernetes integration tests build the whole
reactor
## What changes were proposed in this pull request?
Make the integration test script build all modules
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21800
test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21800
Addressed comments
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@echarles it's highly likely your pods cannot communicate to your driver's
host because of some network partition or a firewall issue. The specific steps
you would have to take to fix that issue
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
> agreed with @echarles that it would be great if the submission client
will still create a headless service for the driver if the driver is running in
a pod in client mode.
The g
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
> Can you point in the fork where the submission client is create the
headless service? (just to help me understand the internals)
> Btw If we stick to this manual approach, th
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@liyinan926 I wonder if we want to have the pod name owner reference still
be a thing, if you will, in client mode. For example what if the pod name that
is given is accidentally one
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
> there was a change in Spark recently in how the driver self-discovered
its hostname by default, if I am not mistaken. Can't recall the exact patch. I
remember that change specifically prompt
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@echarles then you'd probably want more information such as the logs of the
executors, though I'd imagine that one would have trouble getting those given
that the executor exits so quickly
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Also make sure your driver can actually allocate the port specified by
`spark.driver.port`?
---
-
To unsubscribe, e-mail
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@echarles there was a change in Spark recently in how the driver
self-discovered its hostname by default, if I am not mistaken. Can't recall the
exact patch. I remember that change specifically
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
My guess is that the executor is failing to contact the driver - I don't
think you can use the pod name as the hostname in minikube. Can you create a
headless service to bind your pod to a stable
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Comments are addressed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r202801155
--- Diff: docs/running-on-kubernetes.md ---
@@ -120,8 +120,8 @@ This URI is the location of the example jar that is
already in the Docker image
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Something's misbehaving in the integration tests, as the integration test
build isn't able to track dependencies from other modules in the Spark project.
Instead, the integration test tries
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Filed https://issues.apache.org/jira/browse/SPARK-24825 to track the test
problem.
---
-
To unsubscribe, e-mail: reviews
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r202772018
--- Diff: docs/running-on-kubernetes.md ---
@@ -399,18 +426,18 @@ specific to Spark on Kubernetes.
Path to the CA cert file for connecting
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Merging in a few hours if no additional comments are raised.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r205177861
--- Diff: docs/running-on-kubernetes.md ---
@@ -117,6 +117,45 @@ If the local proxy is running at localhost:8001,
`--master k8s://http://127.0.0
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r205178092
--- Diff: docs/running-on-kubernetes.md ---
@@ -117,6 +117,45 @@ If the local proxy is running at localhost:8001,
`--master k8s://http://127.0.0
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21748#discussion_r205178769
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -35,26 +35,39
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Ok after the next build passes I'm going to merge immediately. Thanks for
the review.
---
-
To unsubscribe, e-mail: reviews
GitHub user mccheah opened a pull request:
https://github.com/apache/spark/pull/21874
[SPARK-23146][K8S][TESTS] Enable client mode integration test.
## What changes were proposed in this pull request?
Enable client mode integration test after merging from master
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21874
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
@echarles I don't think we should be special-casing Kubernetes here as
being any different from the other cluster managers. The main point of client
mode is that the driver is running locally
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Though I suppose you could have the driver patch its own metadata fields to
assign itself a unique label. I could see that being confusing to users when
they observe that their driver pod metadata
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
> About selecting the pod with labels, another approach I have taken is
simply using the name of the driver pod, a bit like I have done with the
following deployment (so no need to ensure lab
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21748
Sounds fine. How does the documentation look now in that regard?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21862
We're holding on all additional pod modification feature requests in favor
of work on https://issues.apache.org/jira/browse/SPARK-24434, which would allow
the pods to be customized arbitrarily
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22256#discussion_r213771635
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/LocalDirsFeatureStep.scala
---
@@ -37,41 +40,99 @@ private
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22256#discussion_r214107006
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/LocalDirsFeatureStep.scala
---
@@ -37,41 +40,99 @@ private
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22112
@cloud-fan @tgravescs was wondering if we could get an ETA on this landing?
Also, I tried running something analogous to the example script from the
description of https://issues.apache.org
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22298#discussion_r214491727
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala
---
@@ -51,6 +51,7 @@ private
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22215#discussion_r214490999
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala
---
@@ -60,4 +64,81 @@ private[spark] object
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22215
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22298#discussion_r214492080
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/bindings/JavaDriverFeatureStep.scala
---
@@ -38,7 +39,8
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22215#discussion_r214491177
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala
---
@@ -60,4 +64,81 @@ private[spark] object
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22215#discussion_r214491272
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManager.scala
---
@@ -151,13
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22298#discussion_r214492297
--- Diff:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PythonTestsSuite.scala
---
@@ -72,12
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22257
Looks good, thanks. Merging.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22298
https://issues.apache.org/jira/browse/SPARK-25291 looks like a real issue
with the way the tests are written. So we don't necessarily want to ignore it
for this patch, but we're still thinking
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21669#discussion_r215446604
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -51,7 +51,11 @@ private
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/22146
I think we still need some consensus on the driver container API.
Personally I think it's clearer to use a configuration option to specifically
select the container that is the driver
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22323#discussion_r215456338
--- Diff: docs/running-on-kubernetes.md ---
@@ -215,6 +215,19 @@
spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/22146#discussion_r215452489
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -81,9 +95,12 @@ private
601 - 700 of 946 matches
Mail list logo
