Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r158008588
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
GitHub user foxish opened a pull request:
https://github.com/apache/spark/pull/20032
[SPARK-22845] [Scheduler] Modify spark.kubernetes.allocation.batch.delay to
take time instead of int
## What changes were proposed in this pull request?
Fixing configuration
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19954
> I don't think they are independent as architecturally they make sense
together and represent a single concern: enabling use of remote dependencies
through init-containers. Missing any
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157967544
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157966640
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,502 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157965524
--- Diff: docs/building-spark.md ---
@@ -49,7 +49,7 @@ To create a Spark distribution like those distributed by
the
to be runnable, use `./dev/make
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19717#discussion_r157643149
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/spark-base/Dockerfile
---
@@ -0,0 +1,47 @@
+#
+# Licensed to the Apache Software
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20007
@vanzin PTAL
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20007
700 might break openshift that doesn't run these containers as root.
@erikerlandson can you verify?
---
-
To unsubscribe, e
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/20007#discussion_r157531263
--- Diff: dev/make-distribution.sh ---
@@ -168,12 +168,18 @@ echo "Build flags: $@" >> "$DISTDIR/RELEASE"
# Copy jars
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19995
@ueshin, can this and https://github.com/apache/spark/pull/20007 be merged
now?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20007
Yes, I'll be updating the docs. The '-a' tries to preserve permissions and
all other attributes when copying those directories (
http://man7.org/linux/man-pages/man1/cp.1.html). It's
GitHub user foxish opened a pull request:
https://github.com/apache/spark/pull/20007
[SPARK-22777][Scheduler] Kubernetes mode dockerfile permission and
distribution
## What changes were proposed in this pull request?
1. entrypoint.sh for Kubernetes spark-base image is marked
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19995
@HyukjinKwon done. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19995
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19995
@ueshin @vanzin SparkR failure seems unrelated to me. Any ideas?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157327721
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -133,30 +132,78 @@ private[spark] object Config
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157327476
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -133,30 +132,78 @@ private[spark] object Config
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19995
Done, did another pass over them. Thanks @erikerlandson
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19995
cc/ @vanzin @ueshin @jiangxb1987 @erikerlandson @liyinan926
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
GitHub user foxish opened a pull request:
https://github.com/apache/spark/pull/19995
[SPARK-22807] [Scheduler] Remove config that says docker and replace with
container
## What changes were proposed in this pull request?
Changes discussed in
https://github.com/apache/spark
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157293530
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,502 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157269442
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,502 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157111248
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,502 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157095536
--- Diff: docs/building-spark.md ---
@@ -49,7 +49,7 @@ To create a Spark distribution like those distributed by
the
to be runnable, use `./dev/make
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157093891
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,502 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157093878
--- Diff: docs/running-on-yarn.md ---
@@ -18,7 +18,8 @@ Spark application's configuration (driver, executors, and
the AM when running
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157093475
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,502 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157093212
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,502 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19954
@mridulm We did have some discussions on resource localization at some
point. This is a powerful mechanism when coupled with a resource staging server
(in the future). There is a cost per-pod when
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19954
@rxin @mateiz @vanzin @mridulm @jiangxb1987 @felixcheung, if you can help
take a look at this, it would add a lot of value to Kubernetes mode in 2.3.
Granted we already have a way to supply
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r157006909
--- Diff: docs/building-spark.md ---
@@ -49,7 +49,7 @@ To create a Spark distribution like those distributed by
the
to be runnable, use `./dev/make
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19972
LGTM.
Will independently verify the fix as well. Thanks @mccheah and @liyinan926
for debugging.
---
-
To unsubscribe, e
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19954
@liyinan926, can you update the PR title please?
Maybe "Enable submission of GCS (or S3) and HTTP dependencies to Kubernetes
Scheduler Ba
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156693078
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156692510
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156691165
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156688079
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156687169
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156690501
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156687477
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156686786
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156682354
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156680716
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156679585
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156680213
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156682749
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156683182
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156678936
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156680776
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156682918
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156679094
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156679614
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156680505
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156679949
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156683316
--- Diff: docs/submitting-applications.md ---
@@ -155,6 +165,12 @@ The master URL passed to Spark can be in one of the
following formats
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156679055
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156678238
--- Diff: docs/index.md ---
@@ -112,7 +113,7 @@ options for deployment:
* [Mesos](running-on-mesos.html): deploy a private cluster using
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156679300
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,498 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19946
@jiangxb1987, thanks for the round of review - I updated some links and
pointers to the docs as well. PTAL.
---
-
To unsubscribe
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156460094
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,496 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156460137
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,496 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156460053
--- Diff: sbin/build-push-docker-images.sh ---
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156460169
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,496 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156457660
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,496 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156459761
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,496 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156457806
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,496 @@
+---
+layout: global
+title: Running Spark on Kubernetes
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r156457426
--- Diff: sbin/build-push-docker-images.sh ---
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under
GitHub user foxish opened a pull request:
https://github.com/apache/spark/pull/19946
[SPARK-22648] [Scheduler] Spark on Kubernetes - Documentation
What changes were proposed in this pull request?
This PR contains documentation on the usage of Kubernetes scheduler in
Spark
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19717
> Yes, I agree that we eventually should allow client mode including use
cases that directly create SparkContext
Sorry, in your previous comment, it seemed like you were suggest
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19717
@liyinan926:
> Actually given that in our implementations the executors talk to the
driver through a Kubernetes headless service that is only created by the
submission client. N
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19717
I understand the concern here. I think we can guard against accidental use
in client mode to post a nicer error message. @liyinan926, how about testing if
`kubernetes.default.svc` is accessible
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19717
Thanks to all the reviewers, @liyinan926 and @mccheah for the comments &
fixes. This looks real close!
@ifilonenko - if we could get our integration tests to run once against
this PR this
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19717#discussion_r154955367
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/Client.scala
---
@@ -0,0 +1,234 @@
+/*
+ * Licensed
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19717#discussion_r154952728
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -590,6 +600,11 @@ private[deploy] class SparkSubmitArguments(args
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19717#discussion_r154952559
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -119,5 +139,60 @@ private[spark] object Config
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19717
@liyinan926, please change PR title to match sub-task:
https://issues.apache.org/jira/browse/SPARK-22646
---
-
To unsubscribe, e
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19468
Done, created subtasks.
Thanks everyone for helping review this PR!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r153341501
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r153297620
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -0,0 +1,115 @@
+/*
+ * Licensed
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r153296815
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19468
retest this please
Failure looks unrelated.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19468
@mridulm @vanzin @mateiz @rxin This is waiting for another LGTM I think
(aside from the one by @jiangxb1987). If all parties have no major changes to
suggest, may I propose we merge this before
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r150754184
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r150752598
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r150576157
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala
---
@@ -0,0 +1,229
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r150570562
--- Diff: resource-managers/kubernetes/core/pom.xml ---
@@ -0,0 +1,102 @@
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19468
@vanzin @mridulm @jiangxb1987 https://github.com/apache/spark/pull/19717 is
the second part of this PR which has the client-side and spark-submit changes.
The two in combination should make a lot
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19468
@mridulm @jiangxb1987 @vanzin thanks for taking the time to review this. Is
there another round of reviews that you guys would want to do here? We're
hoping to get this and just **one other** PR
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r149172795
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r147211956
--- Diff: resource-managers/kubernetes/core/pom.xml ---
@@ -0,0 +1,102 @@
+
+
+http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146439253
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146439099
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146437447
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146429823
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146427690
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146426033
--- Diff: pom.xml ---
@@ -2649,6 +2649,13 @@
+ kubernetes
+
+resource-managers/kubernetes/core
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146074603
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/19468
@vanzin, you were right, the YARN constants were left overs and made no
sense wrt k8s. We discussed it in our weekly meeting - it was simply dead code.
I've addressed most of the style comments
Github user foxish commented on a diff in the pull request:
https://github.com/apache/spark/pull/19468#discussion_r146072553
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
---
@@ -0,0
201 - 300 of 331 matches
Mail list logo
