GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/265
Added support for accessing secured HDFS
Also changed the way task run so tasks always run under the user who submit
the tasks. This replaces the old approach of using a environment variable
Github user liyinan926 commented on the pull request:
https://github.com/apache/spark/pull/265#issuecomment-38970674
This PR replaces https://github.com/apache/incubator-spark/pull/467.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157327812
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/FileFetcher.scala
---
@@ -0,0 +1,27
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@vanzin Addressed your comments so far. PTAL. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157328327
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -133,30 +132,78 @@ private[spark] object
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157328425
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/KubernetesSparkDependencyDownloadInitContainer.scala
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157351657
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/KubernetesSparkDependencyDownloadInitContainer.scala
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157531197
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/steps/DriverInitContainerBootstrapStepSuite.scala
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@ueshin addressed your comments. PTAL. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@mccheah @foxish I gave you push access to the fork I used for this PR.
Feel free to push commits if you want. Please do let me know if you plan to
address comments from @vanzin and make changes
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@vanzin @mccheah @foxish Variables names have been shortened in
https://github.com/apache/spark/pull/19954/commits/d50c61e46c43049b9ede615ffabd89a55c4adb0c,
and traits with only a single
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157815526
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/BaseInitContainerConfigurationStep.scala
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@vanzin additionally to shortening names and removing unnecessary traits,
we have further improved the code by removing unused/unnecessary class
arguments and enhancing the Scaladocs on core
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
> @liyinan926 we might be able to make the review smaller, though this
makes some of the code we contribute on each push unusable on its own.
@mccheah I think the biggest thing t
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
> @liyinan926 the code structure is the main issue, yes, but we can tackle
the code structure more effectively by having a better decomposition of the
review process as well. We have these th
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157898666
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkConfPropertiesParser.scala
---
@@ -0,0 +1,50
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157909168
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkConfPropertiesParser.scala
---
@@ -0,0 +1,50
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911449
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala
---
@@ -209,9 +213,33
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911709
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/InitContainerBootstrap.scala
---
@@ -0,0 +1,119
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911785
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/BaseInitContainerConfigurationStep.scala
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911743
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/MountSecretsBootstrap.scala
---
@@ -0,0 +1,62
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911819
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala
---
@@ -0,0 +1,118
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911768
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/InitContainerUtil.scala
---
@@ -0,0 +1,37
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911698
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/InitContainerBootstrap.scala
---
@@ -0,0 +1,119
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911827
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala
---
@@ -0,0 +1,118
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157911840
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -45,6 +45,59
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157912757
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/DriverConfigOrchestrator.scala
---
@@ -116,10 +127,45
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157912841
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/InitContainerConfigOrchestrator.scala
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157913485
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -45,6 +45,59
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157914419
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala
---
@@ -0,0 +1,118
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@vanzin Addressed your comments in
https://github.com/apache/spark/pull/19954/commits/3407d7af68c44b100558e49e4012a27e41b29dda.
PTAL. Thanks a lot
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157942418
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/MountSecretsBootstrap.scala
---
@@ -0,0 +1,62
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r157942449
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala
---
@@ -209,9 +213,33
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158074477
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverInitContainerBootstrapStep.scala
---
@@ -0,0
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19946#discussion_r158168783
--- Diff: docs/running-on-kubernetes.md ---
@@ -0,0 +1,573 @@
+---
+layout: global
+title: Running Spark on Kubernetes
+---
+* This
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@vanzin Thanks for the review! Addressed your comments in
https://github.com/apache/spark/pull/19954/commits/be5de00c9a75ef583759b931e3f8597b9f426f6c
GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20059
[SPARK-22648][Kubernetes] Update documentation to cover features in #19954
## What changes were proposed in this pull request?
This PR updates the Kubernetes documentation corresponding
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20059
@foxish addressed your comments.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20059#discussion_r158567222
--- Diff: docs/running-on-kubernetes.md ---
@@ -120,6 +120,23 @@ by their appropriate remote URIs. Also, application
dependencies can be pre-moun
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20059#discussion_r158568089
--- Diff: docs/running-on-kubernetes.md ---
@@ -120,6 +120,23 @@ by their appropriate remote URIs. Also, application
dependencies can be pre-moun
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20059
Updated in
https://github.com/apache/spark/pull/20059/commits/fbb21121447fe131358042ce05454f88d6fb.
---
-
To unsubscribe
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20059#discussion_r158575135
--- Diff: docs/running-on-kubernetes.md ---
@@ -120,6 +120,57 @@ by their appropriate remote URIs. Also, application
dependencies can be pre-moun
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20059#discussion_r158575137
--- Diff: docs/running-on-kubernetes.md ---
@@ -528,51 +579,90 @@ specific to Spark on Kubernetes
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651909
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -132,30 +131,84 @@ private[spark] object
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651915
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -132,30 +131,84 @@ private[spark] object
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651930
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/DriverConfigOrchestrator.scala
---
@@ -98,28 +109,62
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651933
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -132,30 +131,84 @@ private[spark] object
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651945
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/k8s/SparkPodInitContainer.scala
---
@@ -0,0 +1,116
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651953
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -45,6 +45,59
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158651951
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala
---
@@ -45,6 +45,59
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19954#discussion_r158680033
--- Diff:
resource-managers/kubernetes/docker/src/main/dockerfiles/init-container/Dockerfile
---
@@ -0,0 +1,24 @@
+#
+# Licensed to the Apache
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20059#discussion_r158722622
--- Diff: docs/running-on-kubernetes.md ---
@@ -528,51 +576,91 @@ specific to Spark on Kubernetes
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20059#discussion_r158768975
--- Diff: docs/running-on-kubernetes.md ---
@@ -120,6 +120,54 @@ by their appropriate remote URIs. Also, application
dependencies can be pre-moun
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20059#discussion_r158768984
--- Diff: docs/running-on-kubernetes.md ---
@@ -528,51 +576,91 @@ specific to Spark on Kubernetes
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/19954
@vanzin @jiangxb1987 @ueshin Can this PR be merged now?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20148
[SPARK-22953][K8S] Avoids adding duplicated secret volumes when
init-container is used
## What changes were proposed in this pull request?
User-specified secrets are mounted into both
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159588468
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/MountSecretsBootstrap.scala
---
@@ -28,20 +28,26 @@ private
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159590150
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/MountSecretsBootstrap.scala
---
@@ -28,20 +28,26 @@ private
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159590877
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactorySuite.scala
---
@@ -172,10 +172,8
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159592335
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactorySuite.scala
---
@@ -172,10 +172,8
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20148
@jiangxb1987 Updated PR description.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20154#discussion_r159721343
--- Diff: docs/running-on-kubernetes.md ---
@@ -16,6 +16,8 @@ Kubernetes scheduler that has been added to Spark.
you may setup a test cluster on
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159729690
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/SecretVolumeUtils.scala
---
@@ -14,23 +14,20 @@
* See the
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159729720
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/SecretVolumeUtils.scala
---
@@ -14,23 +14,20 @@
* See the
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159739212
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/BasicDriverConfigurationStep.scala
---
@@ -119,7
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20156#discussion_r159751012
--- Diff: dev/sparktestsupport/modules.py ---
@@ -539,7 +539,7 @@ def __hash__(self):
kubernetes = Module(
name="kuber
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20148#discussion_r159751143
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/BasicDriverConfigurationStep.scala
---
@@ -119,7
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20156#discussion_r159752504
--- Diff: dev/sparktestsupport/modules.py ---
@@ -539,7 +539,7 @@ def __hash__(self):
kubernetes = Module(
name="kuber
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20156
LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20157
[SPARK-22757][K8S] Enable spark.jars and spark.files in KUBERNETES mode
## What changes were proposed in this pull request?
We missed enabling `spark.files` and `spark.jars` in
https
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20148
@vanzin Can this be merged?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20148
Why master?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20159
[SPARK-22953][K8S] Avoids adding duplicated secret volumes when
init-container is used
## What changes were proposed in this pull request?
User-specified secrets are mounted into both
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20148
Got it. @vanzin created https://github.com/apache/spark/pull/20159.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user liyinan926 closed the pull request at:
https://github.com/apache/spark/pull/20148
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20160
Enable spark.jars and spark.files in KUBERNETES mode
## What changes were proposed in this pull request?
We missed enabling `spark.files` and `spark.jars` in
https://github.com/apache
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20157
@vanzin created https://github.com/apache/spark/pull/20160. Closing this
one.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user liyinan926 closed the pull request at:
https://github.com/apache/spark/pull/20157
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20154
@vanzin it seems using `ARG` in `FROM` is only supported since Docker
version 17.06. It does not work for earlier versions of Docker. See
https://stackoverflow.com/questions/40273070/docker
GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20170
[SPARK-22960][K8S] Revert use of ARG base_image in images
## What changes were proposed in this pull request?
This PR reverts the `ARG base_image` before `FROM` in the images of driver
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20187
@foxish have you run the integration tests against this on both minikube
and GKE?
---
-
To unsubscribe, e-mail: reviews
GitHub user liyinan926 opened a pull request:
https://github.com/apache/spark/pull/20193
[SPARK-22998][K8S] Set missing value for SPARK_MOUNTED_CLASSPATH in the
executors
## What changes were proposed in this pull request?
The environment variable `SPARK_MOUNTED_CLASSPATH
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20193
@felixcheung so this change is still applicable if #20192 is merged.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20193
Yes, will review it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20192#discussion_r160323806
--- Diff: bin/docker-image-tool.sh ---
@@ -60,35 +42,53 @@ function image_ref {
}
function build {
- docker build \
---build
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20192#discussion_r160322848
--- Diff: bin/docker-image-tool.sh ---
@@ -60,35 +42,53 @@ function image_ref {
}
function build {
- docker build \
---build
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20192#discussion_r160323223
--- Diff:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/DriverConfigOrchestratorSuite.scala
---
@@ -75,8 +73,8
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20192#discussion_r160324036
--- Diff: docs/running-on-kubernetes.md ---
@@ -56,14 +56,13 @@ be run in a container runtime environment that
Kubernetes supports. Docker is a
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20192#discussion_r160324121
--- Diff: docs/running-on-kubernetes.md ---
@@ -56,14 +56,13 @@ be run in a container runtime environment that
Kubernetes supports. Docker is a
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20192#discussion_r160511180
--- Diff: docs/running-on-kubernetes.md ---
@@ -56,14 +56,13 @@ be run in a container runtime environment that
Kubernetes supports. Docker is a
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20192
LGTM.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20192#discussion_r160512758
--- Diff: docs/running-on-kubernetes.md ---
@@ -56,14 +56,13 @@ be run in a container runtime environment that
Kubernetes supports. Docker is a
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21095
My general comment is the code is centered around parsing and setting up
`hostPath` volumes even though the proposal is generic. I would suggest making
the code generic as well. Particularly I
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21095
Thanks @madanadit! I will take a look over the weekend or early net week.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/21095
@madanadit sorry I didn't get a chance to look into this. I will take a
detailed look once @foxish's comments on testing are
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21095#discussion_r185960630
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala
---
@@ -0,0 +1,142
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21095#discussion_r185960658
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala
---
@@ -0,0 +1,142
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21095#discussion_r185960149
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala
---
@@ -0,0 +1,142
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21095#discussion_r185980514
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala
---
@@ -109,7 +109,15
Github user liyinan926 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21095#discussion_r185961170
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala
---
@@ -0,0 +1,142
1 - 100 of 711 matches
Mail list logo
