Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190964276
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingEventSource.scala
---
@@ -0,0 +1,65
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190963140
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -56,17 +56,43
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190962726
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleEventHandler.scala
---
@@ -0,0
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21366
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21366
Alright, this should be good for review now, with all cleanups and
appropriate test coverage in place. Please take a look. Thanks
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190769478
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingEventSource.scala
---
@@ -0,0 +1,58
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190762965
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/DeterministicExecutorPodsEventQueue.scala
---
@@ -0,0
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21366
jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21366
@liyinan926 I still have a few more tests to write and some tweaks, but the
architecture is now along the lines of what we discussed. Any feedback would be
appreciated
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190389782
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,229
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190386369
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,229
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190385827
--- Diff: pom.xml ---
@@ -760,6 +760,12 @@
1.10.19
test
+
--- End diff --
We always add to the
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190367677
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,229
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190367420
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,229
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190366267
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsWatchEventSource.scala
---
@@ -0,0 +1,63
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190365981
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,229
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r190365332
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,229
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r189739169
--- Diff: pom.xml ---
@@ -150,6 +150,7 @@
4.5.4
4.4.8
+3.0.1
--- End diff --
Noted, will remove in the next
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r189642632
--- Diff: pom.xml ---
@@ -150,6 +150,7 @@
4.5.4
4.4.8
+3.0.1
--- End diff --
These are data structures
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21366
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21366
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r189401135
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingEventSource.scala
---
@@ -0,0 +1,58
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r189400912
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingEventSource.scala
---
@@ -0,0 +1,58
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r189400286
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,212
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21366#discussion_r189399432
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsEventHandler.scala
---
@@ -0,0 +1,212
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21366
Needs tests. @foxish @liyinan926 for initial comments on the design.
---
-
To unsubscribe, e-mail: reviews-unsubscr
GitHub user mccheah opened a pull request:
https://github.com/apache/spark/pull/21366
[SPARK-24248][K8S][WIP] Use the Kubernetes API to populate an event queue
for scheduling
## What changes were proposed in this pull request?
Previously, the scheduler backend was
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21238
What would make this difficult to backport is the fact that this patch was
built on top of the big refactor PR that only went in after 2.3. So we'd need
to rewrite this with the old archite
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21238
I think we can afford to hold off here.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21238
Requesting sign off and merge from @foxish
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21238
@rxin @liyinan926 @foxish addressed comments.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21238
Also #21260 currently only supports hostPath and PVCs but you definitely
want emptyDir for isolation (though that looks like a trivial enough change
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21238
@andrusha I don't think it's entirely analogous - for the simple reason
that the hostPath volumes PR doesn't take into account `SPARK_LOCAL_DIRS`. That
environment variable is used t
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186876513
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -320,50
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186864312
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -320,50
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186853331
--- Diff: docs/running-on-kubernetes.md ---
@@ -561,6 +561,13 @@ specific to Spark on Kubernetes.
This is distinct from spark.executor.cores: it is
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186853242
--- Diff: docs/running-on-kubernetes.md ---
@@ -561,6 +561,13 @@ specific to Spark on Kubernetes.
This is distinct from spark.executor.cores: it is
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186852387
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -320,50
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186825312
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -320,50
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186793785
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -320,50
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186790719
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -320,50
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21241#discussion_r186789848
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -320,50
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186591946
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -29,18 +31,36
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186591608
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -29,18 +31,36
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186591836
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilder.scala
---
@@ -29,18 +31,36
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186591534
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/bindings/PythonDriverFeatureStep.scala
---
@@ -0,0 +1,72
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186244105
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -63,10 +67,17 @@ private[spark] case
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186244157
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -101,17 +112,29 @@ private[spark
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186244580
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/bindings/PythonDriverFeatureStep.scala
---
@@ -0,0 +1,59
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r18629
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala
---
@@ -88,15 +94,22
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r186244541
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/bindings/PythonDriverFeatureStep.scala
---
@@ -0,0 +1,59
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21241
@foxish @liyinan926 @eje please take a look.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user mccheah opened a pull request:
https://github.com/apache/spark/pull/21241
[SPARK-24135][K8s] Resilience to init-container errors on executors.
## What changes were proposed in this pull request?
Spark doesn't attach init-containers. But if a custom web ho
GitHub user mccheah opened a pull request:
https://github.com/apache/spark/pull/21238
[SPARK-24137] Mount local directories as empty dir volumes.
## What changes were proposed in this pull request?
Dramatically improves performance and won't cause Spark applicatio
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21238
@foxish @liyinan926 please take a look, thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r183161726
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala
---
@@ -88,15 +94,22
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21092
Integration tests are meant to be in this repository but we haven't gotten
there yet. See https://github.com/apache/spark/pull/
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r182607048
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala
---
@@ -88,15 +94,22
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21092#discussion_r182606366
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/KubernetesDriverBuilderSuite.scala
---
@@ -89,6 +97,29
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21067
> We don't have a solid story for checkpointing streaming computation right
now, and even if we did, you'll certainly lose all progress from batch jobs.
Should probab
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21067#discussion_r181475185
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala
---
@@ -92,6 +93,12 @@ private
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/21067#discussion_r181474442
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -59,15
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/21067
Looks like there's a lot of conflicts from the refactor that was just
merged.
In general though I don't think this buys us too much. The problem is that
when the driver fails, y
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r180784491
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesSpec.scala
---
@@ -14,17 +14,18 @@
* See the
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r180782338
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesSpec.scala
---
@@ -14,17 +14,18 @@
* See the
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
Latest commits address @kimoonkim and @liyinan926's comments. Also fixed
merge conflicts related to https://github.com/apache/spark/pull/20811 @and
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r179297367
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala
---
@@ -175,49 +138,41
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r179297415
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala
---
@@ -175,49 +138,41
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r179296929
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala
---
@@ -14,104 +14,66
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r179296427
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala
---
@@ -0,0 +1,135
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
@vanzin can you take a look? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20553#discussion_r178371360
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -85,6 +85,12 @@ private[spark] object Config
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20553#discussion_r178371345
--- Diff: docs/running-on-kubernetes.md ---
@@ -576,14 +576,21 @@ specific to Spark on Kubernetes.
spark.kubernetes.driver.limit.cores
(none
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20553
Ah never mind sorry - thought we were referring to changing thread count
with `spark.kubernetes.executor.request.cores`. Think this config key makes
sense to communicate the exact K8s semantics
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20553
`spark.executor.cores` is the standard used by all the other cluster
managers, so we have to use that.
---
-
To unsubscribe, e
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20943
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
Rerun integration tests please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
Rerun kubernetes integration tests please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177545717
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -0,0 +1,158 @@
+/*
+ * Licensed
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177507771
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -0,0 +1,164 @@
+/*
+ * Licensed
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177502836
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -0,0 +1,164 @@
+/*
+ * Licensed
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
@echarles I don't think we should be making this configurable, as it would
not fit the precedent of the other cluster managers. We would allow the user to
write the bulk of the spark-submit
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177277920
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -0,0 +1,158 @@
+/*
+ * Licensed
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177277876
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala
---
@@ -0,0 +1,164 @@
+/*
+ * Licensed
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177273537
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala
---
@@ -180,20 +167,17
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177273458
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkPod.scala
---
@@ -14,17 +14,16 @@
* See the License for
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177265297
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesSpec.scala
---
@@ -14,25 +14,18 @@
* See the
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20910#discussion_r177264954
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala
---
@@ -211,34 +195,34
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
Requesting review from @vanzin , @foxish, @ifilonenko, @liyinan926, @eje.
Any other feedback is welcome!
---
-
To unsubscribe
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20910
For all reviewers - this change is very large. Github's interpretation of
the diff also doesn't present the changes in the most easily consumed manner.
To account for this, the pu
GitHub user mccheah opened a pull request:
https://github.com/apache/spark/pull/20910
[SPARK-22839] [K8s] Refactor to unify driver and executor pod builder APIs
## What changes were proposed in this pull request?
Breaks down the construction of driver pods and executor pods
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20669
Thanks - merging shortly.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20669
Merge button doesn't appear for me in the UI =( will need to look into that.
---
-
To unsubscribe, e-mail: reviews-uns
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20669
Think someone with permissions to merge has to do so here.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20669
@vanzin anything other feedback before merging this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user mccheah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20669#discussion_r175522981
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh ---
@@ -53,14 +53,10 @@ fi
case "$SPARK_K8S_CM
401 - 500 of 947 matches
Mail list logo
