Github user ssuchter commented on the issue:
https://github.com/apache/spark/pull/20669
@vanzin I just opened this PR for putting the integration tests in the
Spark repo: https://github.com/apache/spark/pull/20697
---
Github user ssuchter commented on the issue:
https://github.com/apache/spark/pull/20669
It's just because we haven't PR'd them yet. There's some cleanups that I
wanted to do before doing that.
---
-
To unsubscribe,
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20669
BTW I wonder why are we still relying on integration tests that do not live
in the Spark repo...
---
-
To unsubscribe, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20669
`SparkFiles.get()` is the official way of retrieving anything that is
distributed using `--files`. So if the application is not using that, it's
relying on undocumented behavior.
In the
Github user ifilonenko commented on the issue:
https://github.com/apache/spark/pull/20669
@vanzin you are definitely right and that is what needs to happen. However,
one of the functions of the init container was resolving the file location
which atm isnât supported unless the
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20669
@ifilonenko you can't just remove the init container without replacing its
functionality. In my patch that was done by using spark-submit to run the
driver. But here you have nothing. That's why the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20669
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1105/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20669
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20669
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1105/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20669
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1112/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20669
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1066/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20669
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1066/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20669
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1072/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20669
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20669
I think tests are failing because you're missing changes to run the driver
through spark-submit in the entry point script:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20669
Kubernetes integration test status failure
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1061/
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20669
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1061/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20669
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20669
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1067/
Github user foxish commented on the issue:
https://github.com/apache/spark/pull/20669
+1, the benefits outweigh the desire to be k8s-like in the approach. We
should have tests that validate the behavior changes though. I'm wondering if
we should move the integration tests into the
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20669
If that's the consensus, then great! I thought some people still wanted to
keep the init container around for some reason.
---
-
Github user mccheah commented on the issue:
https://github.com/apache/spark/pull/20669
My impression was that we did reach a consensus that this was the right
idea from the mailing list discussion. Specifically
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20669
I'm not sure there was a consensus at the end of that discussion, and
@mccheah has been working on some internal refactoring here also. @foxish
---
23 matches
Mail list logo
