[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20812 Should we close this then? @jinxing64 @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/20812 @jerryshao Understood, `Ideally different udfs should be packaged in different jars with different name/version`. True. But we are faced with tons of udf/jars migrating from other engine. I made this pr not just for merging, but also want to know you experts comments. @jiangxb1987 @jerryshao Thanks again for your comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20812 @jinxing64 , I think using same name jars which contains different classes seems practically is not a best practice. Ideally different udfs should be packaged in different jars with different name/version. That will be easy for user to manage. Also same name jars could easily cause classpath issue usually. As you always has a workaround for this issue out of Spark. So I would suggest not to fix it, since this is a quite user specific issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88302/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20812 **[Test build #88302 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88302/testReport)** for PR 20812 at commit [`4ac1e8e`](https://github.com/apache/spark/commit/4ac1e8ecb5af688bc341e6256e9003244130fd25). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88303/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20812 **[Test build #88303 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88303/testReport)** for PR 20812 at commit [`4473878`](https://github.com/apache/spark/commit/4473878d4bfc457a10522248798429465310adaa). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/20812 @jerryshao Thanks for comment; Yes, this change is only for `sc.addJar` and the jars will be named with a prefix when executor `updateDependencies`. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20812 Does it only fix the jars added by `sc.addJar` or using non-yarn mode? Because yarn uses distributed cache at start, so it has a different code path, right? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20812 **[Test build #88303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88303/testReport)** for PR 20812 at commit [`4473878`](https://github.com/apache/spark/commit/4473878d4bfc457a10522248798429465310adaa). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1558/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20812 **[Test build #88302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88302/testReport)** for PR 20812 at commit [`4ac1e8e`](https://github.com/apache/spark/commit/4ac1e8ecb5af688bc341e6256e9003244130fd25). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1557/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/20812 @jiangxb1987 Thanks a lot for review. I will refine soon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20812 The idea looks good, just a few comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/20812 @vanzin @zsxwing @jerryshao How do you think about this ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88208/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20812 **[Test build #88208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88208/testReport)** for PR 20812 at commit [`f78c273`](https://github.com/apache/spark/commit/f78c273c6132f9cc226668590273836950c39b74). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1489/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20812 **[Test build #88208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88208/testReport)** for PR 20812 at commit [`f78c273`](https://github.com/apache/spark/commit/f78c273c6132f9cc226668590273836950c39b74). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1488/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
