[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 Thanks - merging shortly. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20669 There's a section explaining it at the bottom of https://spark.apache.org/committers.html --- - To unsubscribe, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20669 @mccheah you should have gotten an e-mail from Matei explaining the basics of how to merge PRs. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread erikerlandson
Github user erikerlandson commented on the issue: https://github.com/apache/spark/pull/20669 @mccheah workflow is to use `dev/merge_spark_pr.py` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 Merge button doesn't appear for me in the UI =( will need to look into that. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20669 @mccheah you're a committer... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 Think someone with permissions to merge has to do so here. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20669 I probably won't have time to review this carefully, so if you're happy with it, don't wait for me. --- - To unsubscribe,

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-19 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 @vanzin anything other feedback before merging this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1558/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1558/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1574/

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-16 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/20669 Results from integration testing: ``` Discovery starting. Discovery completed in 123 milliseconds. Run starting. Expected test count is: 8 KubernetesSuite: - Run SparkPi

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1495/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1495/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1511/

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1459/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1475/

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1459/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/20669 Newest push passes all tests (with this merged I will then merge in [this](https://github.com/apache-spark-on-k8s/spark-integration/pull/42/files)) ``` KubernetesSuite: - Run SparkPi

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 Hm, noted that we're making this tradeoff. We have an internal use case where we're pushing a custom logging properties file into the container using `spark.files`. Logging properties files need to

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20669 Oh, btw, if you think that is a really, really important feature, you still don't need an init container for that. You can just run the dependency download tool before you run spark-submit in the

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20669 That's one scenario where an init-container /might/ help. But be aware that your YARN example only works in a single one scenario - YARN cluster mode, and maybe even then may need some config

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-03-12 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20669 @vanzin I have a question regarding how this will interact with files that need to exist before the JVM starts. When we used the init-container approach, we were specifically fetching the

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1160/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1160/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1167/

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-28 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/20669 @vanzin @mccheah @foxish This is ready for your review as it passes unit tests and integration tests. At the moment, it is required that we modify the integration tests to call

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1028/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1028/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1027/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1034/

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread ssuchter
Github user ssuchter commented on the issue: https://github.com/apache/spark/pull/20669 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1027/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1033/

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1026/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread ssuchter
Github user ssuchter commented on the issue: https://github.com/apache/spark/pull/20669 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1032/

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20669 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1026/ ---

[GitHub] spark issue #20669: [SPARK-22839][K8S] Remove the use of init-container for ...

2018-02-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20669 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional