[GitHub] [spark] AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573291506 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573291509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116525/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573291506 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
SparkQA removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573286858 **[Test build #116525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116525/testReport)** for PR 24939 at commit [`c198669`](https://github.com/apache/spark/commit/c198669123c05e2a195422147a3a22ac4960ea7e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573291509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116525/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
SparkQA commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573291449 **[Test build #116525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116525/testReport)** for PR 24939 at commit [`c198669`](https://github.com/apache/spark/commit/c198669123c05e2a195422147a3a22ac4960ea7e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size
gatorsmile commented on issue #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size URL: https://github.com/apache/spark/pull/26434#issuecomment-573290979 @JkSelf Could you update the PR description and explain what kind of skew join this PR is trying to resolve. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile edited a comment on issue #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size
gatorsmile edited a comment on issue #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size URL: https://github.com/apache/spark/pull/26434#issuecomment-573290979 @JkSelf Could you update the PR description and explain what kind of skew join this PR is trying to resolve? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289516 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116521/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289515 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289516 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116521/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289515 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274742 **[Test build #116521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116521/testReport)** for PR 26813 at commit [`82a8906`](https://github.com/apache/spark/commit/82a890676b1bb057cbb1c1070759c0aafceb5fce). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289450 **[Test build #116521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116521/testReport)** for PR 26813 at commit [`82a8906`](https://github.com/apache/spark/commit/82a890676b1bb057cbb1c1070759c0aafceb5fce). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289156 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116520/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289155 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289156 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116520/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289155 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274092 **[Test build #116520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116520/testReport)** for PR 26813 at commit [`82a8906`](https://github.com/apache/spark/commit/82a890676b1bb057cbb1c1070759c0aafceb5fce). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573289087 **[Test build #116520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116520/testReport)** for PR 26813 at commit [`82a8906`](https://github.com/apache/spark/commit/82a890676b1bb057cbb1c1070759c0aafceb5fce). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573288868 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116524/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573288867 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573288868 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116524/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
SparkQA removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573282284 **[Test build #116524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116524/testReport)** for PR 27164 at commit [`5f37b64`](https://github.com/apache/spark/commit/5f37b64010bf669d1426db53e9e4c35770cf37b4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573288867 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
SparkQA commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573288814 **[Test build #116524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116524/testReport)** for PR 27164 at commit [`5f37b64`](https://github.com/apache/spark/commit/5f37b64010bf669d1426db53e9e4c35770cf37b4). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573287589 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116523/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573287588 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573287588 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
SparkQA removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573280870 **[Test build #116523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116523/testReport)** for PR 26682 at commit [`cbdb2ed`](https://github.com/apache/spark/commit/cbdb2ed628c580602c4cc2124d9e0103a81aac04). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573287589 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116523/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573287534 **[Test build #116523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116523/testReport)** for PR 26682 at commit [`cbdb2ed`](https://github.com/apache/spark/commit/cbdb2ed628c580602c4cc2124d9e0103a81aac04). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573286957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21311/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573286957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21311/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins removed a comment on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573286955 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ozancicek commented on a change in pull request #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
ozancicek commented on a change in pull request #24939: [SPARK-18569][ML][R]
Support RFormula arithmetic, I() and spark functions
URL: https://github.com/apache/spark/pull/24939#discussion_r365503334
##
File path: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala
##
@@ -442,11 +488,17 @@ object RFormulaModel extends MLReadable[RFormulaModel] {
val metadata = DefaultParamsReader.loadMetadata(path, sc, className)
val dataPath = new Path(path, "data").toString
- val data = sparkSession.read.parquet(dataPath).select("label", "terms",
"hasIntercept").head()
+ val df = sparkSession.read.parquet(dataPath)
+ val columns = Seq("label", "terms", "hasIntercept", "evalExprs")
+ val data = df.select(columns.intersect(df.columns).map(df(_)): _*)
+.head()
val label = data.getString(0)
val terms = data.getAs[Seq[Seq[String]]](1)
val hasIntercept = data.getBoolean(2)
- val resolvedRFormula = ResolvedRFormula(label, terms, hasIntercept)
+ // Check for old saved models (spark version < 3), they do not have
evalExprs.
+ val (major, minor) =
VersionUtils.majorMinorVersion(metadata.sparkVersion)
+ val evalExprs = if (major.toInt < 3) Seq[String]() else
data.getAs[Seq[String]](3)
+ val resolvedRFormula = ResolvedRFormula(label, terms, hasIntercept,
evalExprs)
Review comment:
Thanks, added a testcase for loading a model from 2.4.4.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
AmplabJenkins commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573286955 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
SparkQA commented on issue #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions URL: https://github.com/apache/spark/pull/24939#issuecomment-573286858 **[Test build #116525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116525/testReport)** for PR 24939 at commit [`c198669`](https://github.com/apache/spark/commit/c198669123c05e2a195422147a3a22ac4960ea7e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ozancicek commented on a change in pull request #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
ozancicek commented on a change in pull request #24939: [SPARK-18569][ML][R]
Support RFormula arithmetic, I() and spark functions
URL: https://github.com/apache/spark/pull/24939#discussion_r365503223
##
File path: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala
##
@@ -344,29 +360,28 @@ class RFormulaModel private[feature](
private[ml] val pipelineModel: PipelineModel)
extends Model[RFormulaModel] with RFormulaBase with MLWritable {
+ private val foldExprs = (df: DataFrame) =>
resolvedFormula.evalExprs.foldLeft(df) _
+
@Since("2.0.0")
override def transform(dataset: Dataset[_]): DataFrame = {
checkCanTransform(dataset.schema)
-transformLabel(pipelineModel.transform(dataset))
+val withExprs = foldExprs(dataset.toDF) {
+ case(df, colname) => df.withColumn(colname, expr(colname))
+}
+val withFeatures = pipelineModel.transform(withExprs)
+val withLabel = transformLabel(withFeatures)
+foldExprs(withLabel) {
+ case(df, colname) => df.drop(col(s"`$colname`"))
Review comment:
Indeed, foldLeft is unnecessary. Switched to ```drop(colNames: String*)```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ozancicek commented on a change in pull request #24939: [SPARK-18569][ML][R] Support RFormula arithmetic, I() and spark functions
ozancicek commented on a change in pull request #24939: [SPARK-18569][ML][R]
Support RFormula arithmetic, I() and spark functions
URL: https://github.com/apache/spark/pull/24939#discussion_r365503203
##
File path: mllib/src/main/scala/org/apache/spark/ml/feature/RFormula.scala
##
@@ -344,29 +360,28 @@ class RFormulaModel private[feature](
private[ml] val pipelineModel: PipelineModel)
extends Model[RFormulaModel] with RFormulaBase with MLWritable {
+ private val foldExprs = (df: DataFrame) =>
resolvedFormula.evalExprs.foldLeft(df) _
+
@Since("2.0.0")
override def transform(dataset: Dataset[_]): DataFrame = {
checkCanTransform(dataset.schema)
-transformLabel(pipelineModel.transform(dataset))
+val withExprs = foldExprs(dataset.toDF) {
+ case(df, colname) => df.withColumn(colname, expr(colname))
+}
Review comment:
Thanks, I wasn't aware of that function. Switched to withColumns.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #27168: [SPARK-30377][ML] Make Regressors extend abstract class Regressor
viirya commented on a change in pull request #27168: [SPARK-30377][ML] Make
Regressors extend abstract class Regressor
URL: https://github.com/apache/spark/pull/27168#discussion_r365503175
##
File path:
mllib/src/main/scala/org/apache/spark/ml/regression/AFTSurvivalRegression.scala
##
@@ -208,8 +195,7 @@ class AFTSurvivalRegression @Since("1.6.0")
(@Since("1.6.0") override val uid: S
}
@Since("2.0.0")
Review comment:
We change from `fit` to `train`. Strictly it is not `since 2.0.0`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui
HeartSaVioR commented on issue #26201: [SPARK-29543][SS][UI] Init structured streaming ui URL: https://github.com/apache/spark/pull/26201#issuecomment-573282938 Yeah we realized the patch doesn't fix the issue, so we rolled back the patch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27146: [SPARK-21869][SS][DOCS][FOLLOWUP] Document Kafka producer pool configuration
HeartSaVioR commented on issue #27146: [SPARK-21869][SS][DOCS][FOLLOWUP] Document Kafka producer pool configuration URL: https://github.com/apache/spark/pull/27146#issuecomment-573282845 Yeah I sought a bit, and realized the published doc seems to be only broken for preview/preview2. If then may not be a big deal. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573282684 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116522/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573282681 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573282684 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116522/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573282681 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
SparkQA removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573276119 **[Test build #116522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116522/testReport)** for PR 26682 at commit [`cbdb2ed`](https://github.com/apache/spark/commit/cbdb2ed628c580602c4cc2124d9e0103a81aac04). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573282654 **[Test build #116522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116522/testReport)** for PR 26682 at commit [`cbdb2ed`](https://github.com/apache/spark/commit/cbdb2ed628c580602c4cc2124d9e0103a81aac04). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #27146: [SPARK-21869][SS][DOCS][FOLLOWUP] Document Kafka producer pool configuration
HeartSaVioR commented on a change in pull request #27146:
[SPARK-21869][SS][DOCS][FOLLOWUP] Document Kafka producer pool configuration
URL: https://github.com/apache/spark/pull/27146#discussion_r365500406
##
File path: docs/structured-streaming-kafka-integration.md
##
@@ -802,6 +802,31 @@ df.selectExpr("topic", "CAST(key AS STRING)", "CAST(value
AS STRING)") \
+### Producer Caching
+
+Given Kafka producer instance is designed to be thread-safe, Spark initializes
a Kafka producer instance and co-use across tasks for same caching key.
+
+The caching key is built up from the following information:
Review comment:
Hmm... would we like to guide how to tweak it, or would like to just hide
it? The reason I explain this is that end users may encounter an issue on
producer pool and would like to debug a bit, but if it feels us to be too
internal, sounds OK to hide it as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573282373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21310/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573282373 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21310/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins removed a comment on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573282369 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
AmplabJenkins commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573282369 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
SparkQA commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573282284 **[Test build #116524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116524/testReport)** for PR 27164 at commit [`5f37b64`](https://github.com/apache/spark/commit/5f37b64010bf669d1426db53e9e4c35770cf37b4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events
HeartSaVioR commented on issue #27164: [SPARK-30479][SQL] Apply compaction of event log to SQL events URL: https://github.com/apache/spark/pull/27164#issuecomment-573282228 OK Just rebased this. cc. @vanzin @gaborgsomogyi This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table
AmplabJenkins removed a comment on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table URL: https://github.com/apache/spark/pull/27173#issuecomment-573281677 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table
AmplabJenkins removed a comment on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table URL: https://github.com/apache/spark/pull/27173#issuecomment-573281678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116517/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table
AmplabJenkins commented on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table URL: https://github.com/apache/spark/pull/27173#issuecomment-573281677 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table
SparkQA removed a comment on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table URL: https://github.com/apache/spark/pull/27173#issuecomment-573267011 **[Test build #116517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116517/testReport)** for PR 27173 at commit [`454c71b`](https://github.com/apache/spark/commit/454c71b05261fda60008146b3fc4bd4751fd7636). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table
AmplabJenkins commented on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table URL: https://github.com/apache/spark/pull/27173#issuecomment-573281678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116517/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table
SparkQA commented on issue #27173: [SPARK-30312][SQL][2.4] Preserve path permission and acl when truncate table URL: https://github.com/apache/spark/pull/27173#issuecomment-573281628 **[Test build #116517 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116517/testReport)** for PR 27173 at commit [`454c71b`](https://github.com/apache/spark/commit/454c71b05261fda60008146b3fc4bd4751fd7636). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] manuzhang commented on a change in pull request #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size
manuzhang commented on a change in pull request #26434: [SPARK-29544] [SQL]
optimize skewed partition based on data size
URL: https://github.com/apache/spark/pull/26434#discussion_r365499728
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/ReduceNumShufflePartitions.scala
##
@@ -145,39 +147,36 @@ case class ReduceNumShufflePartitions(conf: SQLConf)
extends Rule[SparkPlan] {
distinctNumPreShufflePartitions.length == 1,
"There should be only one distinct value of the number pre-shuffle
partitions " +
"among registered Exchange operator.")
-val numPreShufflePartitions = distinctNumPreShufflePartitions.head
val partitionStartIndices = ArrayBuffer[Int]()
-// The first element of partitionStartIndices is always 0.
-partitionStartIndices += 0
-
-var postShuffleInputSize = 0L
-
-var i = 0
-while (i < numPreShufflePartitions) {
- // We calculate the total size of ith pre-shuffle partitions from all
pre-shuffle stages.
- // Then, we add the total size to postShuffleInputSize.
- var nextShuffleInputSize = 0L
- var j = 0
- while (j < mapOutputStatistics.length) {
-nextShuffleInputSize += mapOutputStatistics(j).bytesByPartitionId(i)
-j += 1
- }
-
- // If including the nextShuffleInputSize would exceed the target
partition size, then start a
- // new partition.
- if (i > 0 && postShuffleInputSize + nextShuffleInputSize >
targetPostShuffleInputSize) {
-partitionStartIndices += i
-// reset postShuffleInputSize.
-postShuffleInputSize = nextShuffleInputSize
- } else {
-postShuffleInputSize += nextShuffleInputSize
- }
-
- i += 1
+val partitionEndIndices = ArrayBuffer[Int]()
+val numPartitions = distinctNumPreShufflePartitions.head
+val includedPartitions = (0 until
numPartitions).filter(!excludedPartitions.contains(_))
+val firstStartIndex = includedPartitions(0)
+partitionStartIndices += firstStartIndex
+var postShuffleInputSize =
mapOutputStatistics.map(_.bytesByPartitionId(firstStartIndex)).sum
+var i = firstStartIndex
Review comment:
`i` is actually the (candidate) "partitionEndIndex" and would be better
inited to `firstStartIndex + 1` (also reset to `nextPartitionIndex + 1`). That
helps me to understand the logic here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] manuzhang commented on a change in pull request #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size
manuzhang commented on a change in pull request #26434: [SPARK-29544] [SQL]
optimize skewed partition based on data size
URL: https://github.com/apache/spark/pull/26434#discussion_r365499728
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/ReduceNumShufflePartitions.scala
##
@@ -145,39 +147,36 @@ case class ReduceNumShufflePartitions(conf: SQLConf)
extends Rule[SparkPlan] {
distinctNumPreShufflePartitions.length == 1,
"There should be only one distinct value of the number pre-shuffle
partitions " +
"among registered Exchange operator.")
-val numPreShufflePartitions = distinctNumPreShufflePartitions.head
val partitionStartIndices = ArrayBuffer[Int]()
-// The first element of partitionStartIndices is always 0.
-partitionStartIndices += 0
-
-var postShuffleInputSize = 0L
-
-var i = 0
-while (i < numPreShufflePartitions) {
- // We calculate the total size of ith pre-shuffle partitions from all
pre-shuffle stages.
- // Then, we add the total size to postShuffleInputSize.
- var nextShuffleInputSize = 0L
- var j = 0
- while (j < mapOutputStatistics.length) {
-nextShuffleInputSize += mapOutputStatistics(j).bytesByPartitionId(i)
-j += 1
- }
-
- // If including the nextShuffleInputSize would exceed the target
partition size, then start a
- // new partition.
- if (i > 0 && postShuffleInputSize + nextShuffleInputSize >
targetPostShuffleInputSize) {
-partitionStartIndices += i
-// reset postShuffleInputSize.
-postShuffleInputSize = nextShuffleInputSize
- } else {
-postShuffleInputSize += nextShuffleInputSize
- }
-
- i += 1
+val partitionEndIndices = ArrayBuffer[Int]()
+val numPartitions = distinctNumPreShufflePartitions.head
+val includedPartitions = (0 until
numPartitions).filter(!excludedPartitions.contains(_))
+val firstStartIndex = includedPartitions(0)
+partitionStartIndices += firstStartIndex
+var postShuffleInputSize =
mapOutputStatistics.map(_.bytesByPartitionId(firstStartIndex)).sum
+var i = firstStartIndex
Review comment:
`i` is actually "partitionEndIndex" and would be better inited to
`firstStartIndex + 1` (also reset to `nextPartitionIndex + 1`). That helps me
to understand the logic here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] manuzhang commented on a change in pull request #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size
manuzhang commented on a change in pull request #26434: [SPARK-29544] [SQL]
optimize skewed partition based on data size
URL: https://github.com/apache/spark/pull/26434#discussion_r365499728
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/ReduceNumShufflePartitions.scala
##
@@ -145,39 +147,36 @@ case class ReduceNumShufflePartitions(conf: SQLConf)
extends Rule[SparkPlan] {
distinctNumPreShufflePartitions.length == 1,
"There should be only one distinct value of the number pre-shuffle
partitions " +
"among registered Exchange operator.")
-val numPreShufflePartitions = distinctNumPreShufflePartitions.head
val partitionStartIndices = ArrayBuffer[Int]()
-// The first element of partitionStartIndices is always 0.
-partitionStartIndices += 0
-
-var postShuffleInputSize = 0L
-
-var i = 0
-while (i < numPreShufflePartitions) {
- // We calculate the total size of ith pre-shuffle partitions from all
pre-shuffle stages.
- // Then, we add the total size to postShuffleInputSize.
- var nextShuffleInputSize = 0L
- var j = 0
- while (j < mapOutputStatistics.length) {
-nextShuffleInputSize += mapOutputStatistics(j).bytesByPartitionId(i)
-j += 1
- }
-
- // If including the nextShuffleInputSize would exceed the target
partition size, then start a
- // new partition.
- if (i > 0 && postShuffleInputSize + nextShuffleInputSize >
targetPostShuffleInputSize) {
-partitionStartIndices += i
-// reset postShuffleInputSize.
-postShuffleInputSize = nextShuffleInputSize
- } else {
-postShuffleInputSize += nextShuffleInputSize
- }
-
- i += 1
+val partitionEndIndices = ArrayBuffer[Int]()
+val numPartitions = distinctNumPreShufflePartitions.head
+val includedPartitions = (0 until
numPartitions).filter(!excludedPartitions.contains(_))
+val firstStartIndex = includedPartitions(0)
+partitionStartIndices += firstStartIndex
+var postShuffleInputSize =
mapOutputStatistics.map(_.bytesByPartitionId(firstStartIndex)).sum
+var i = firstStartIndex
Review comment:
`i` is actually partitionEndIndex and would be better inited to
`firstStartIndex + 1` (also reset to `nextPartitionIndex + 1`). That helps me
to understand the logic here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573280870 **[Test build #116523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116523/testReport)** for PR 26682 at commit [`cbdb2ed`](https://github.com/apache/spark/commit/cbdb2ed628c580602c4cc2124d9e0103a81aac04). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573280468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21309/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573280465 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573280468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21309/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573280465 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tgravescs commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
tgravescs commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573280378 test this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on issue #27097: [SPARK-9478][ML][PYSPARK] Add sample weights to Random Forest
huaxingao commented on issue #27097: [SPARK-9478][ML][PYSPARK] Add sample weights to Random Forest URL: https://github.com/apache/spark/pull/27097#issuecomment-573277910 LGTM :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #27097: [SPARK-9478][ML][PYSPARK] Add sample weights to Random Forest
huaxingao commented on a change in pull request #27097:
[SPARK-9478][ML][PYSPARK] Add sample weights to Random Forest
URL: https://github.com/apache/spark/pull/27097#discussion_r365497142
##
File path:
mllib/src/test/scala/org/apache/spark/ml/classification/RandomForestClassifierSuite.scala
##
@@ -259,6 +263,37 @@ class RandomForestClassifierSuite extends MLTest with
DefaultReadWriteTest {
})
}
+ test("training with sample weights") {
+val df = binaryDataset
+val numClasses = 2
+// (numTrees, maxDepth, subsamplingRate, fractionInTol)
+val testParams = Seq(
+ (20, 5, 1.0, 0.96),
+ (20, 10, 1.0, 0.96),
+ (20, 10, 0.95, 0.96)
+)
Review comment:
The reason I suggested testing different impurities is because when
calculating best split, the impurity path (both ```entropy``` and ```gini```)
is affected by sample weight. However, after taking a look at the DecisionTree
test, I saw both ```entropy``` and ```gini``` are tested with sample weight
there, so this is already covered in DecisionTree test, no need to test here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27030: [SPARK-30244][SQL] Emit pre/post events for "Partition" methods in ExternalCatalogWithListener
dongjoon-hyun commented on a change in pull request #27030: [SPARK-30244][SQL]
Emit pre/post events for "Partition" methods in ExternalCatalogWithListener
URL: https://github.com/apache/spark/pull/27030#discussion_r365496650
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogEventSuite.scala
##
@@ -209,4 +209,65 @@ class ExternalCatalogEventSuite extends SparkFunSuite {
catalog.dropFunction("db5", "fn4")
checkEvents(DropFunctionPreEvent("db5", "fn4") :: DropFunctionEvent("db5",
"fn4") :: Nil)
}
+
+ testWithCatalog("partition") { (catalog, checkEvents) =>
+// CREATE
+val path1 = Files.createTempDirectory("db_")
+val uri1 = preparePath(path1)
+
+val dbDefinition = createDbDefinition()
+
+val storage = CatalogStorageFormat.empty.copy(
+ locationUri = Option(uri1))
Review comment:
Is this table storage pointing db path? Maybe, do you want the following?
```scala
val path1 = Files.createTempDirectory("db_")
val path2 = Files.createTempDirectory(path1, "tbl_")
val uri1 = preparePath(path1)
val uri2 = preparePath(path2)
// CREATE
val dbDefinition = createDbDefinition(uri1)
val storage = CatalogStorageFormat.empty.copy(
locationUri = Option(uri2))
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573276188 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins removed a comment on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573276193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21308/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573276193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21308/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
AmplabJenkins commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573276188 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with
SparkQA commented on issue #26682: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with URL: https://github.com/apache/spark/pull/26682#issuecomment-573276119 **[Test build #116522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116522/testReport)** for PR 26682 at commit [`cbdb2ed`](https://github.com/apache/spark/commit/cbdb2ed628c580602c4cc2124d9e0103a81aac04). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274742 **[Test build #116521 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116521/testReport)** for PR 26813 at commit [`82a8906`](https://github.com/apache/spark/commit/82a890676b1bb057cbb1c1070759c0aafceb5fce). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274216 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274218 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21307/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274218 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21307/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274216 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JkSelf commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
JkSelf commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274078 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573274092 **[Test build #116520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116520/testReport)** for PR 26813 at commit [`82a8906`](https://github.com/apache/spark/commit/82a890676b1bb057cbb1c1070759c0aafceb5fce). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273571 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116519/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273565 **[Test build #116519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116519/testReport)** for PR 26813 at commit [`5d1af66`](https://github.com/apache/spark/commit/5d1af66c543f9191c09dbb7ca3a290b623ea260a). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21306/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273514 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273570 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JkSelf commented on issue #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size
JkSelf commented on issue #26434: [SPARK-29544] [SQL] optimize skewed partition based on data size URL: https://github.com/apache/spark/pull/26434#issuecomment-573273539 @cloud-fan Can you help to retest and change the test machine? Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273514 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273570 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273400 **[Test build #116519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116519/testReport)** for PR 26813 at commit [`5d1af66`](https://github.com/apache/spark/commit/5d1af66c543f9191c09dbb7ca3a290b623ea260a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273515 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/21306/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273571 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116519/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
SparkQA commented on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573273400 **[Test build #116519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/116519/testReport)** for PR 26813 at commit [`5d1af66`](https://github.com/apache/spark/commit/5d1af66c543f9191c09dbb7ca3a290b623ea260a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573272983 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116518/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE
AmplabJenkins removed a comment on issue #26813: [SPARK-30188][SQL][WIP] Resolve the failed unit tests when enable AQE URL: https://github.com/apache/spark/pull/26813#issuecomment-573272922 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
