[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743139 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743119 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743127 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743750 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150744643 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150745701 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150745794 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150747604 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -0,0 +1,227

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150748702 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150749210 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/ConfigurationUtils.scala --- @@ -0,0 +1,50

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150881619 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150948852 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150959150 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-15 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r151197505 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-15 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r151201092 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-15 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r151269007 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-15 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/19468 @vanzin @jiangxb1987 @mridulm We have addressed all the comments so far. Is this good to merge? --- - To unsubscribe, e-mail

[GitHub] spark pull request #22224: [SPARK-24090][K8S] Update running-on-kubernetes.m...

2018-08-24 Thread liyinan926
GitHub user liyinan926 opened a pull request: https://github.com/apache/spark/pull/4 [SPARK-24090][K8S] Update running-on-kubernetes.md ## What changes were proposed in this pull request? Updated documentation for Spark on Kubernetes for the upcoming 2.4.0

[GitHub] spark pull request #22215: [SPARK-25222][K8S] Improve container status loggi...

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22215#discussion_r212713203 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -60,4 +64,90 @@ private[spark

[GitHub] spark pull request #22215: [SPARK-25222][K8S] Improve container status loggi...

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22215#discussion_r212713446 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -60,4 +64,90 @@ private[spark

[GitHub] spark pull request #22215: [SPARK-25222][K8S] Improve container status loggi...

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22215#discussion_r212713726 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -60,4 +64,90 @@ private[spark

[GitHub] spark pull request #22212: [SPARK-25220] Seperate kubernetes node selector c...

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22212#discussion_r212717021 --- Diff: docs/running-on-kubernetes.md --- @@ -663,11 +663,21 @@ specific to Spark on Kubernetes

[GitHub] spark pull request #22212: [SPARK-25220] Seperate kubernetes node selector c...

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22212#discussion_r212717486 --- Diff: docs/running-on-kubernetes.md --- @@ -663,11 +663,21 @@ specific to Spark on Kubernetes

[GitHub] spark pull request #22212: [SPARK-25220] Seperate kubernetes node selector c...

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22212#discussion_r212717156 --- Diff: docs/running-on-kubernetes.md --- @@ -663,11 +663,21 @@ specific to Spark on Kubernetes

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212721068 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,18 @@ private[spark] object

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212724020 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,18 @@ private[spark] object

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212761597 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,18 @@ private[spark] object

[GitHub] spark pull request #22224: [SPARK-24090][K8S] Update running-on-kubernetes.m...

2018-08-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/4#discussion_r212867062 --- Diff: docs/running-on-kubernetes.md --- @@ -185,6 +185,35 @@ To use a secret through an environment variable use the following options to the

[GitHub] spark pull request #22224: [SPARK-24090][K8S] Update running-on-kubernetes.m...

2018-08-26 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/4#discussion_r212867059 --- Diff: docs/running-on-kubernetes.md --- @@ -299,21 +328,15 @@ RBAC authorization and how to configure Kubernetes service accounts for pods, pl

[GitHub] spark issue #22224: [SPARK-24090][K8S] Update running-on-kubernetes.md

2018-08-27 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/4 Can this be merged? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22256: [SPARK-25262][K8S][WIP] Better support configurability o...

2018-08-28 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22256 @mccheah. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22212: [SPARK-25220] Seperate kubernetes node selector config b...

2018-08-28 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22212 > I agree there's an argument for keeping this, but an alternative would be to leave the original for backward compatability, deprecate it, and recommend people make use of custom pod t

[GitHub] spark issue #22215: [SPARK-25222][K8S] Improve container status logging

2018-08-30 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22215 @mccheah can you give `ok to test` to this one and help merge it? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-04 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215040130 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-04 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215041417 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,15 @@ private[spark] object

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-04 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215055594 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-04 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215040572 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/LocalDirsFeatureStep.scala --- @@ -22,6 +22,7 @@ import

[GitHub] spark pull request #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be t...

2018-09-05 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22323#discussion_r215303844 --- Diff: docs/running-on-kubernetes.md --- @@ -215,6 +215,19 @@ spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.clai

[GitHub] spark issue #22323: [SPARK-25262][K8S] Allow SPARK_LOCAL_DIRS to be tmpfs ba...

2018-09-06 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22323 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-09-06 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22146 @mccheah @erikerlandson given that banch-2.4 has already been cut, we have more flexibility in terms of timing of getting this in, as well as more space for thoughts to get this right. I don&#

[GitHub] spark pull request #22405: [SPARK-25295][K8S]Fix executor names collision

2018-09-12 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22405#discussion_r217178781 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -254,3 +251,17

[GitHub] spark pull request #22405: [SPARK-25295][K8S]Fix executor names collision

2018-09-12 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22405#discussion_r217177996 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -254,3 +251,17

[GitHub] spark pull request #22405: [SPARK-25295][K8S]Fix executor names collision

2018-09-12 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22405#discussion_r217214721 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocatorSuite.scala --- @@ -167,13

[GitHub] spark issue #22405: [SPARK-25295][K8S]Fix executor names collision

2018-09-12 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22405 Will merge later today if there's no more comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark issue #22416: [SPARK-25291][K8S][BACKPORT] Fixing Flakiness of Executo...

2018-09-13 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22416 @ifilonenko so this is the same as #22415 except that it's for `branch-2.4`, right? --- - To unsubscribe, e-mail: re

[GitHub] spark pull request #22415: [SPARK-25291][K8S] Fixing Flakiness of Executor P...

2018-09-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22415#discussion_r217543984 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala

[GitHub] spark issue #22416: [SPARK-25291][K8S][BACKPORT] Fixing Flakiness of Executo...

2018-09-13 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22416 OK, so we can merge the other one to both master and branch-2.4. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22415: [SPARK-25291][K8S] Fixing Flakiness of Executor P...

2018-09-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22415#discussion_r217560548 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala

[GitHub] spark pull request #22415: [SPARK-25291][K8S] Fixing Flakiness of Executor P...

2018-09-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22415#discussion_r217583729 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala

[GitHub] spark pull request #22415: [SPARK-25291][K8S] Fixing Flakiness of Executor P...

2018-09-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22415#discussion_r217769122 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r217792152 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r217838764 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-17 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r218178934 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark issue #22433: [SPARK-25442][SQL][K8S] Support STS to run in k8s deploy...

2018-09-17 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22433 Agreed with @mridulm that the naming restriction is specific to k8s and should be handled in a k8s specific way, e.g., somewhere around https://github.com/apache/spark/blob/master/resource

[GitHub] spark issue #22415: [SPARK-25291][K8S] Fixing Flakiness of Executor Pod test...

2018-09-18 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22415 I'm gonna merge this to master and branch-2.4 before noon if there's no more comment. --- - To unsubscribe, e-mai

[GitHub] spark pull request #22392: [SPARK-23200] Reset Kubernetes-specific config on...

2018-09-18 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22392#discussion_r218616946 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -54,6 +54,10 @@ class Checkpoint(ssc: StreamingContext, val

[GitHub] spark issue #22722: [SPARK-24432][k8s] Add support for dynamic resource allo...

2018-10-15 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22722 @huafengw we are working on a new shuffle service + dynamic resource allocation design for k8s, and decided to not adopt the approach we used in the fork. @mccheah

[GitHub] spark pull request #22748: [SPARK-25745][K8S] Improve docker-image-tool.sh s...

2018-10-16 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22748#discussion_r225615239 --- Diff: bin/docker-image-tool.sh --- @@ -78,20 +91,38 @@ function build { docker build $NOCACHEARG "${BUILD_ARGS[@]}" \ -t $

[GitHub] spark pull request #22748: [SPARK-25745][K8S] Improve docker-image-tool.sh s...

2018-10-16 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22748#discussion_r225614967 --- Diff: bin/docker-image-tool.sh --- @@ -44,28 +44,41 @@ function image_ref { function build { local BUILD_ARGS local IMG_PATH

[GitHub] spark pull request #22608: [SPARK-23257][K8S][TESTS] Kerberos Support Integr...

2018-10-16 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r225618723 --- Diff: bin/docker-image-tool.sh --- @@ -71,18 +71,29 @@ function build { --build-arg base_img=$(image_ref spark

[GitHub] spark pull request #22608: [SPARK-23257][K8S][TESTS] Kerberos Support Integr...

2018-10-16 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r225620898 --- Diff: bin/docker-image-tool.sh --- @@ -71,18 +71,29 @@ function build { --build-arg base_img=$(image_ref spark

[GitHub] spark pull request #22608: [SPARK-23257][K8S][TESTS] Kerberos Support Integr...

2018-10-16 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r225634934 --- Diff: bin/docker-image-tool.sh --- @@ -71,18 +71,29 @@ function build { --build-arg base_img=$(image_ref spark

[GitHub] spark issue #22608: [SPARK-23257][K8S][TESTS] Kerberos Support Integration T...

2018-10-16 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22608 > I think the suggestion is to go to master & 2.4 The integration tests depend on the Kerberos support for Spark on k8s, which was merged into the master but not branch-2.4.

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-16 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r225718039 --- Diff: bin/docker-image-tool.sh --- @@ -71,18 +71,29 @@ function build { --build-arg base_img=$(image_ref spark

[GitHub] spark pull request #22760: [SPARK-25751][K8S][TEST] Unit Testing for Kerbero...

2018-10-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22760#discussion_r227065151 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/KubernetesFeaturesTestUtils.scala --- @@ -63,4 +63,40

[GitHub] spark pull request #22760: [SPARK-25751][K8S][TEST] Unit Testing for Kerbero...

2018-10-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22760#discussion_r227064253 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesConf.scala --- @@ -78,6 +79,12 @@ private[spark] case

[GitHub] spark pull request #22433: [SPARK-25442][SQL][K8S] Support STS to run in k8s...

2018-10-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22433#discussion_r227067136 --- Diff: docs/running-on-kubernetes.md --- @@ -340,6 +340,43 @@ RBAC authorization and how to configure Kubernetes service accounts for pods, pl

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-23 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r227554357 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/docker

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-23 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r227554814 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/ProcessUtils.scala --- @@ -40,7

[GitHub] spark pull request #22820: [SPARK-25828][K8S][BUILD] Bumping Kubernetes-Clie...

2018-10-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22820#discussion_r228030819 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -157,7 +157,10 @@ private[spark

[GitHub] spark pull request #22820: [SPARK-25828][K8S][BUILD] Bumping Kubernetes-Clie...

2018-10-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22820#discussion_r228030940 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala --- @@ -56,8 +56,10

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r228257003 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/Utils.scala --- @@ -27,4 +27,36

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r228258489 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/IntegrationTestBackend.scala

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-25 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r228256148 --- Diff: resource-managers/kubernetes/integration-tests/scripts/setup-integration-test-env.sh --- @@ -71,19 +71,36 @@ if [[ $IMAGE_TAG == &quo

[GitHub] spark issue #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration testin...

2018-10-25 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22805 @rvesse I want to take back my comment on bumping the version of the fabric8 client in the same PR. Based on the discussion so far, it seems more appropriate to do that in a separate PR, like in

[GitHub] spark issue #22146: [SPARK-24434][K8S] pod template files

2018-10-29 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22146 > Latest container selection w/ default to first LGTM +1. LGTM on container selection. --- - To unsubscribe, e-m

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r229367224 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/IntegrationTestBackend.scala

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r229366533 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/cloud/KubeConfigBackend.scala

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229370525 --- Diff: bin/docker-image-tool.sh --- @@ -53,7 +53,7 @@ function build { # contain a lot of duplicated jars with the main Spark directory. In a

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229373633 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/kerberos-deployment.yml --- @@ -0,0 +1,49 @@ +# +# Licensed to the Apache

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229373205 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/dn1-deployment.yml --- @@ -0,0 +1,49 @@ +# +# Licensed to the Apache

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229371556 --- Diff: resource-managers/kubernetes/docker/src/test/scripts/populate-data.sh --- @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +# +# Licensed to

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229370867 --- Diff: dev/make-distribution.sh --- @@ -191,7 +191,8 @@ fi # Only create and copy the dockerfiles directory if the kubernetes artifacts were

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229372327 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/data-populator-deployment.yml --- @@ -0,0 +1,49 @@ +# +# Licensed to the

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229373845 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/kerberos-test.yml --- @@ -0,0 +1,43 @@ +# +# Licensed to the Apache

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229372458 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/data-populator-service.yml --- @@ -0,0 +1,33 @@ +# +# Licensed to the

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229373938 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/nn-deployment.yml --- @@ -0,0 +1,54 @@ +# +# Licensed to the Apache

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229389845 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/data-populator-deployment.yml --- @@ -0,0 +1,49 @@ +# +# Licensed to the

[GitHub] spark pull request #22805: [WIP][SPARK-25809][K8S][TEST] New K8S integration...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22805#discussion_r229399189 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/IntegrationTestBackend.scala

[GitHub] spark issue #22805: [SPARK-25809][K8S][TEST] New K8S integration testing bac...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22805 @mccheah since he had some comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229523966 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/kerberos/KerberosPVWatcherCache.scala

[GitHub] spark pull request #22608: [SPARK-25750][K8S][TESTS] Kerberos Support Integr...

2018-10-30 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22608#discussion_r229523838 --- Diff: resource-managers/kubernetes/integration-tests/kerberos-yml/kerberos-set.yml --- @@ -0,0 +1,49 @@ +# +# Licensed to the Apache

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-11-01 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22904 Jenkins test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22897: [SPARK-25875][k8s] Merge code to set up driver co...

2018-11-02 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22897#discussion_r230429059 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverCommandFeatureStep.scala --- @@ -0,0 +1,134

[GitHub] spark pull request #22897: [SPARK-25875][k8s] Merge code to set up driver co...

2018-11-02 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22897#discussion_r230428297 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -58,16 +58,13

[GitHub] spark pull request #22897: [SPARK-25875][k8s] Merge code to set up driver co...

2018-11-02 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22897#discussion_r230469595 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverCommandFeatureStep.scala --- @@ -0,0 +1,134

[GitHub] spark pull request #22959: [SPARK-25876][k8s] Simplify kubernetes configurat...

2018-11-19 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22959#discussion_r234773904 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverCommandFeatureStep.scala --- @@ -124,7 +122,7

[GitHub] spark pull request #22959: [SPARK-25876][k8s] Simplify kubernetes configurat...

2018-11-19 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22959#discussion_r234774586 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/HadoopSparkUserExecutorFeatureStep.scala --- @@ -16,28

[GitHub] spark pull request #22959: [SPARK-25876][k8s] Simplify kubernetes configurat...

2018-11-19 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22959#discussion_r234774673 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -16,40

[GitHub] spark pull request #22959: [SPARK-25876][k8s] Simplify kubernetes configurat...

2018-11-19 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22959#discussion_r234774528 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/HadoopConfExecutorFeatureStep.scala --- @@ -28,21

<    2   3   4   5   6   7   8   >