[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes in...

2018-07-27 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 See #21748 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes in...

2018-07-11 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20451 See https://github.com/apache/spark/pull/21748 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/856/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/856/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/856/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20451 @echarles I'm planning to push a different change that takes a different approach from this. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 @liyinan926 Thx for you comments (see my answers). I have merged the last changes, added a tiny doc section and pushed again. ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/855/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-11 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20451 @mccheah who mentioned a plan to create a separate change for client mode support. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/821/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/821/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/821/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/820/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/820/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 @rayburgemeestre Thx for sharing you Toree conf. Are you sure you are running in client mode? (I don't see something similar to `--conf spark.submit.deployMode=client`). Maybe it just works in

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/820/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-07-10 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 @liyinan926 I pushed the merge with master and tested OutCluster (I didn't test InCluster but as mentioned previously, we would need something similar to the `DriverServiceBootstrapStep` of the

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-06-27 Thread rayburgemeestre
Github user rayburgemeestre commented on the issue: https://github.com/apache/spark/pull/20451 Yes, absolutely, but I made those changes in a hardcoded way though, just to try it out. So I'm not sure how helpful it is. Near the end of this file:

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-06-27 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 @rayburgemeestre This is cool! Apache Toree configuration for K8S is on my todo list but would be happy to copycat your conf... Any gist? IMHO both client and cluster mode

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-06-27 Thread rayburgemeestre
Github user rayburgemeestre commented on the issue: https://github.com/apache/spark/pull/20451 Just wanted to say this PR is really great, with this it's possible to for example use Jupyter with Spark on K8s:

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-05-31 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20451 @echarles I'm wondering do you have time to sync this with the master and make it to a shape that's ready for review? Thanks! ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-05-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2828/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-05-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2904/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-05-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-05-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2828/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-05-04 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 I have updated this branch and tested successfully client mode OutCluster. Happy to get confirmation of this from anyone here (cc/ @foxish) For InCluster (was working fine before

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-04-14 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 Now that #20910 has been merged, I will update this PR to take account the refactoring. @inetfuture Once these changes are pushed, there is the review process which needs to occur, so difficult

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-04-11 Thread inetfuture
Github user inetfuture commented on the issue: https://github.com/apache/spark/pull/20451 May I ask, what's the release plan for this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1761/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1761/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1784/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread echarles
Github user echarles commented on the issue: https://github.com/apache/spark/pull/20451 Current state works for out-cluster client mode. For in-cluster, we miss the `DriverServiceBootstrapStep` of the fork which "Allows the driver to be reachable by executor pods through a

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1756/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1756/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1779/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1735/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20451 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1735/ ---

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1757/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1756/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1755/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-01-31 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20451 /cc --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/422/

[GitHub] spark issue #20451: [SPARK-23146][WIP] Support client mode for Kubernetes cl...

2018-01-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20451 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional