[GitHub] [spark] SparkQA removed a comment on pull request #28627: [SPARK-31756][WEBUI][test-maven] Add real headless browser support for UI test
SparkQA removed a comment on pull request #28627: URL: https://github.com/apache/spark/pull/28627#issuecomment-635621840 **[Test build #123247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123247/testReport)** for PR 28627 at commit [`2e805ec`](https://github.com/apache/spark/commit/2e805ec3276d820935b987861ab90a042c1a8638). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28627: [SPARK-31756][WEBUI][test-maven] Add real headless browser support for UI test
SparkQA commented on pull request #28627: URL: https://github.com/apache/spark/pull/28627#issuecomment-635776114 **[Test build #123247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123247/testReport)** for PR 28627 at commit [`2e805ec`](https://github.com/apache/spark/commit/2e805ec3276d820935b987861ab90a042c1a8638). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements
maropu commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-635774892 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28657: [SPARK-31840][ML] Add instance weight support in LogisticRegressionSummary
huaxingao commented on pull request #28657: URL: https://github.com/apache/spark/pull/28657#issuecomment-635770681 cc @srowen @zhengruifeng This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
AmplabJenkins removed a comment on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635769889 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
SparkQA removed a comment on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635766059 **[Test build #123263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123263/testReport)** for PR 28672 at commit [`0b3e765`](https://github.com/apache/spark/commit/0b3e765867f4f3ed33954bd31b62d602e9d28832). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
AmplabJenkins commented on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635769889 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
SparkQA commented on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635769798 **[Test build #123263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123263/testReport)** for PR 28672 at commit [`0b3e765`](https://github.com/apache/spark/commit/0b3e765867f4f3ed33954bd31b62d602e9d28832). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635767099 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635767099 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635720332 **[Test build #123259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123259/testReport)** for PR 28331 at commit [`70c3871`](https://github.com/apache/spark/commit/70c3871ed801ed2b5e964e321bcfaba33f8735af). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
AmplabJenkins removed a comment on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635764256 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635766357 **[Test build #123259 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123259/testReport)** for PR 28331 at commit [`70c3871`](https://github.com/apache/spark/commit/70c3871ed801ed2b5e964e321bcfaba33f8735af). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
SparkQA commented on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635766059 **[Test build #123263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123263/testReport)** for PR 28672 at commit [`0b3e765`](https://github.com/apache/spark/commit/0b3e765867f4f3ed33954bd31b62d602e9d28832). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #28664: [SPARK-31855][SQL][TESTS] Check reading date/timestamp from Avro files w/ and w/o Spark version
cloud-fan closed pull request #28664: URL: https://github.com/apache/spark/pull/28664 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28664: [SPARK-31855][SQL][TESTS] Check reading date/timestamp from Avro files w/ and w/o Spark version
cloud-fan commented on pull request #28664: URL: https://github.com/apache/spark/pull/28664#issuecomment-635765240 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
huaxingao commented on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635764888 Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference per @gatorsmile request. cc @maropu @dilipbiswal @ulysses-you @jzhuge @xuanyuanking This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors
AmplabJenkins removed a comment on pull request #28619: URL: https://github.com/apache/spark/pull/28619#issuecomment-635764290 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors
AmplabJenkins commented on pull request #28619: URL: https://github.com/apache/spark/pull/28619#issuecomment-635764290 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
AmplabJenkins commented on pull request #28672: URL: https://github.com/apache/spark/pull/28672#issuecomment-635764256 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors
SparkQA commented on pull request #28619: URL: https://github.com/apache/spark/pull/28619#issuecomment-635763920 **[Test build #123262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123262/testReport)** for PR 28619 at commit [`1cae338`](https://github.com/apache/spark/commit/1cae338342376b16c42e436b2a8fdb2240e9d9b9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao opened a new pull request #28672: [SPARK-31333][SQL][DOCS][FOLLOW-UP] Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference
huaxingao opened a new pull request #28672: URL: https://github.com/apache/spark/pull/28672 ### What changes were proposed in this pull request? Add Coalesce/Repartition/Repartition_By_Range Hints to SQL Reference ### Why are the changes needed? To make SQL reference complete ### Does this PR introduce _any_ user-facing change? https://user-images.githubusercontent.com/13592258/83223466-bc207500-a12f-11ea-8876-12eff13a3e72.png;> Only the the above page is changed. The following two pages are the same as before. https://user-images.githubusercontent.com/13592258/83223474-bfb3fc00-a12f-11ea-807a-824a618afa0b.png;> https://user-images.githubusercontent.com/13592258/83223478-c2165600-a12f-11ea-806e-a1e57dc35ef4.png;> ### How was this patch tested? Manually build and check This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635758324 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123261/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
SparkQA removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635757584 **[Test build #123261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123261/testReport)** for PR 28593 at commit [`852d730`](https://github.com/apache/spark/commit/852d7306320f110a16b4d627c9e547a3183a4ef2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635758317 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635757936 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635758309 **[Test build #123261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123261/testReport)** for PR 28593 at commit [`852d730`](https://github.com/apache/spark/commit/852d7306320f110a16b4d627c9e547a3183a4ef2). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635758317 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635757936 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635757584 **[Test build #123261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123261/testReport)** for PR 28593 at commit [`852d730`](https://github.com/apache/spark/commit/852d7306320f110a16b4d627c9e547a3183a4ef2). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default
cloud-fan commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-635757370 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28669: [SPARK-31864][SQL] Adjust AQE skew join trigger condition
cloud-fan commented on pull request #28669: URL: https://github.com/apache/spark/pull/28669#issuecomment-635751861 This makes sense to me. e.g. for partitions [1, 1, 1, ..., 1, 10], which are coalesced to `[9, 10]`, I don't think there are skewed partitions. @JkSelf what do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #28668: [SPARK-31862][SQL] Remove exception wrapping in AQE
cloud-fan closed pull request #28668: URL: https://github.com/apache/spark/pull/28668 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635749462 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123257/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28668: [SPARK-31862][SQL] Remove exception wrapping in AQE
cloud-fan commented on pull request #28668: URL: https://github.com/apache/spark/pull/28668#issuecomment-635749895 yea, it's better to make the error behavior consistent between AQE and non-AQE. Thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635749458 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635749458 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635704603 **[Test build #123257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123257/testReport)** for PR 28647 at commit [`9eacf1e`](https://github.com/apache/spark/commit/9eacf1e041dd0ace3efab2825d20549af0b4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635749256 **[Test build #123257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123257/testReport)** for PR 28647 at commit [`9eacf1e`](https://github.com/apache/spark/commit/9eacf1e041dd0ace3efab2825d20549af0b4). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #28670: [SPARK-31865][SQL] Fix complex AQE query stage not reused
cloud-fan closed pull request #28670: URL: https://github.com/apache/spark/pull/28670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28670: [SPARK-31865][SQL] Fix complex AQE query stage not reused
cloud-fan commented on pull request #28670: URL: https://github.com/apache/spark/pull/28670#issuecomment-635749108 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
cloud-fan closed pull request #28651: URL: https://github.com/apache/spark/pull/28651 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28670: [SPARK-31865][SQL] Fix complex AQE query stage not reused
AmplabJenkins commented on pull request #28670: URL: https://github.com/apache/spark/pull/28670#issuecomment-635747350 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28670: [SPARK-31865][SQL] Fix complex AQE query stage not reused
AmplabJenkins removed a comment on pull request #28670: URL: https://github.com/apache/spark/pull/28670#issuecomment-635747350 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
cloud-fan commented on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635747382 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28670: [SPARK-31865][SQL] Fix complex AQE query stage not reused
SparkQA removed a comment on pull request #28670: URL: https://github.com/apache/spark/pull/28670#issuecomment-635646684 **[Test build #123251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123251/testReport)** for PR 28670 at commit [`dd41b7d`](https://github.com/apache/spark/commit/dd41b7da80335017b26665bdbb58c2bf733ec167). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28670: [SPARK-31865][SQL] Fix complex AQE query stage not reused
SparkQA commented on pull request #28670: URL: https://github.com/apache/spark/pull/28670#issuecomment-635746931 **[Test build #123251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123251/testReport)** for PR 28670 at commit [`dd41b7d`](https://github.com/apache/spark/commit/dd41b7da80335017b26665bdbb58c2bf733ec167). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635746316 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123256/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635746311 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635746311 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635746196 **[Test build #123256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123256/testReport)** for PR 28647 at commit [`162627a`](https://github.com/apache/spark/commit/162627ad2f2bc249bfd11580821a0b5de4c5ff6a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635698603 **[Test build #123256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123256/testReport)** for PR 28647 at commit [`162627a`](https://github.com/apache/spark/commit/162627ad2f2bc249bfd11580821a0b5de4c5ff6a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28671: [SPARK-31859][SPARK-31861][SPARK-31863] Fix Thriftserver session timezone issues
cloud-fan commented on a change in pull request #28671:
URL: https://github.com/apache/spark/pull/28671#discussion_r432244219
##
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
##
@@ -811,6 +811,63 @@ class HiveThriftBinaryServerSuite extends
HiveThriftJdbcTest {
}
}
}
+
+ test("SPARK-31859 Thriftserver works with
spark.sql.datetime.java8API.enabled=true") {
+withJdbcStatement() { statement =>
+ withJdbcStatement() { st =>
+st.execute("set spark.sql.datetime.java8API.enabled=true")
+val rs = st.executeQuery("select date '2020-05-28', timestamp
'2020-05-28 00:00:00'")
+rs.next()
+assert(rs.getDate(1).toString() == "2020-05-28")
+assert(rs.getTimestamp(2).toString() == "2020-05-28 00:00:00.0")
+ }
+}
+ }
+
+ test("SPARK-31861 Thriftserver respects spark.sql.session.timeZone") {
+withJdbcStatement() { statement =>
+ withJdbcStatement() { st =>
+st.execute("set spark.sql.session.timeZone=+03:15") // different than
Thriftserver's JVM tz
+ val rs = st.executeQuery("select timestamp '2020-05-28 10:00:00'")
+rs.next()
+val ts = rs.getTimestamp(1)
+val tsString = rs.getString(1)
Review comment:
this is not used.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28671: [SPARK-31859][SPARK-31861][SPARK-31863] Fix Thriftserver session timezone issues
cloud-fan commented on a change in pull request #28671:
URL: https://github.com/apache/spark/pull/28671#discussion_r432243974
##
File path:
sql/hive-thriftserver/v1.2/src/main/java/org/apache/hive/service/cli/ColumnValue.java
##
@@ -178,9 +178,9 @@ public static TColumnValue toTColumnValue(Type type, Object
value) {
case VARCHAR_TYPE:
return stringValue((HiveVarchar)value);
case DATE_TYPE:
- return dateValue((Date)value);
case TIMESTAMP_TYPE:
- return timestampValue((Timestamp)value);
Review comment:
shall we remove the `dateValue` and `timestampValue` methods?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #27805: [SPARK-31056][SQL] Add CalendarIntervals division
yaooqinn commented on pull request #27805: URL: https://github.com/apache/spark/pull/27805#issuecomment-635741463 thanks for pinging me @maropu Using extract/date_part functions instead may be a good choice here for end-users. Supporting `/` for intervals goes against a contract that we are trying to follow - 'avoiding 1 month = 30 days'. BTW, I check presto which supports ANSI-like year-month/day-time intervals and it does not have such a feature. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28668: [SPARK-31862][SQL] Remove exception wrapping in AQE
AmplabJenkins removed a comment on pull request #28668: URL: https://github.com/apache/spark/pull/28668#issuecomment-635739504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28668: [SPARK-31862][SQL] Remove exception wrapping in AQE
AmplabJenkins commented on pull request #28668: URL: https://github.com/apache/spark/pull/28668#issuecomment-635739504 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28671: [SPARK-31859][SPARK-31861][SPARK-31863] Fix Thriftserver session timezone issues
cloud-fan commented on a change in pull request #28671:
URL: https://github.com/apache/spark/pull/28671#discussion_r432237754
##
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkOperationUtils.scala
##
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive.thriftserver
+
+import org.apache.hive.service.cli.session.HiveSession
+
+import org.apache.spark.SparkContext
+import org.apache.spark.sql.{SparkSession, SQLContext}
+import org.apache.spark.sql.catalyst.catalog.CatalogTableType
+import org.apache.spark.sql.catalyst.catalog.CatalogTableType.{EXTERNAL,
MANAGED, VIEW}
+import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.util.Utils
+
+/**
+ * Utils for Spark operations.
+ */
+private[hive] trait SparkOperationUtils {
Review comment:
how about
```
trait SparkOperationUtils extends Operation {
...
abstract override def run(): Unit = {
withLocalProperties {
super.run()
}
}
}
```
You can try this in scala shell
```
class AA {
def f(): Int = 1
}
trait BB extends AA {
abstract override def f(): Int = super.f() + 1
}
class CC extends AA with BB
new CC().f // print 2
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28668: [SPARK-31862][SQL] Remove exception wrapping in AQE
SparkQA commented on pull request #28668: URL: https://github.com/apache/spark/pull/28668#issuecomment-635738977 **[Test build #123249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123249/testReport)** for PR 28668 at commit [`b0f2ad0`](https://github.com/apache/spark/commit/b0f2ad0f5d657244a95cab8f05c2df4a827e0766). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28668: [SPARK-31862][SQL] Remove exception wrapping in AQE
SparkQA removed a comment on pull request #28668: URL: https://github.com/apache/spark/pull/28668#issuecomment-635630594 **[Test build #123249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123249/testReport)** for PR 28668 at commit [`b0f2ad0`](https://github.com/apache/spark/commit/b0f2ad0f5d657244a95cab8f05c2df4a827e0766). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28671: [SPARK-31859][SPARK-31861][SPARK-31863] Fix Thriftserver session timezone issues
cloud-fan commented on a change in pull request #28671:
URL: https://github.com/apache/spark/pull/28671#discussion_r432237754
##
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkOperationUtils.scala
##
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.hive.thriftserver
+
+import org.apache.hive.service.cli.session.HiveSession
+
+import org.apache.spark.SparkContext
+import org.apache.spark.sql.{SparkSession, SQLContext}
+import org.apache.spark.sql.catalyst.catalog.CatalogTableType
+import org.apache.spark.sql.catalyst.catalog.CatalogTableType.{EXTERNAL,
MANAGED, VIEW}
+import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.util.Utils
+
+/**
+ * Utils for Spark operations.
+ */
+private[hive] trait SparkOperationUtils {
Review comment:
how about
```
trait SparkOperationUtils { this: Operation =>
...
override def run(): Unit = {
withLocalProperties {
.run()
}
}
}
```
You can try this in scala shell
```
class AA {
def f(): Int = 1
}
trait BB { this: AA =>
override def f(): Int = 2
}
class CC extends AA with BB
new CC().f // print 2
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28669: [SPARK-31864][SQL] Adjust AQE skew join trigger condition
AmplabJenkins removed a comment on pull request #28669: URL: https://github.com/apache/spark/pull/28669#issuecomment-635736416 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28669: [SPARK-31864][SQL] Adjust AQE skew join trigger condition
AmplabJenkins commented on pull request #28669: URL: https://github.com/apache/spark/pull/28669#issuecomment-635736416 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28669: [SPARK-31864][SQL] Adjust AQE skew join trigger condition
SparkQA removed a comment on pull request #28669: URL: https://github.com/apache/spark/pull/28669#issuecomment-635639292 **[Test build #123250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123250/testReport)** for PR 28669 at commit [`3ed0f63`](https://github.com/apache/spark/commit/3ed0f6387729d254dc58bd3bca8865019dff0065). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28669: [SPARK-31864][SQL] Adjust AQE skew join trigger condition
SparkQA commented on pull request #28669: URL: https://github.com/apache/spark/pull/28669#issuecomment-635735795 **[Test build #123250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123250/testReport)** for PR 28669 at commit [`3ed0f63`](https://github.com/apache/spark/commit/3ed0f6387729d254dc58bd3bca8865019dff0065). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #27627: [WIP][SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow
AmplabJenkins removed a comment on pull request #27627: URL: https://github.com/apache/spark/pull/27627#issuecomment-635732936 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #27627: [WIP][SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow
AmplabJenkins commented on pull request #27627: URL: https://github.com/apache/spark/pull/27627#issuecomment-635732936 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #27627: [WIP][SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow
SparkQA commented on pull request #27627: URL: https://github.com/apache/spark/pull/27627#issuecomment-635732632 **[Test build #123260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123260/testReport)** for PR 27627 at commit [`59a00c4`](https://github.com/apache/spark/commit/59a00c4e1092579532c37569261fb830c194f891). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] siknezevic commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements
siknezevic commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-635732643 Could you please reopen this PR and remove the Stale tag? This PR brings significant performance improvements when spilling is enabled. Thank you This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] skambha commented on pull request #27627: [WIP][SPARK-28067][SQL] Fix incorrect results for decimal aggregate sum by returning null on decimal overflow
skambha commented on pull request #27627: URL: https://github.com/apache/spark/pull/27627#issuecomment-635730700 I wanted to summarize where we are at. The changes up to this point took care of whole stage enabled, disabled and ansi enabled/disabled scenarios etc consistently. But based on the discussion with @cloud-fan , we do not want to do the overflow checks for the intermediate results so the decision is to scale back the fix to - handle the case where it won’t return incorrect results when ansi enabled is false - for other codepaths that are not handled, we throw exceptions and not incorrect results which is similar behavior as before. - There is potential for non-deterministic behavior for sum when there is a overflow. Although not ideal, it is a step forward and it will not return incorrect results for some scenarios. Please see this pr thread https://github.com/skambha/spark/pull/1 for the discussion. I will push the following changes after this: - Merged @cloud-fan's changes at https://github.com/skambha/spark/pull/1 and fix for test failure. - The only changes on top of it is I have cleaned up the code comments to reflect the current state This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
SparkQA removed a comment on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635718288 **[Test build #123258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123258/testReport)** for PR 28651 at commit [`099d261`](https://github.com/apache/spark/commit/099d261c126496ecccd3900fa902fec60e272f10). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
AmplabJenkins removed a comment on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635728948 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
AmplabJenkins commented on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635728948 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
SparkQA commented on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635728749 **[Test build #123258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123258/testReport)** for PR 28651 at commit [`099d261`](https://github.com/apache/spark/commit/099d261c126496ecccd3900fa902fec60e272f10). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable
maropu commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-635724871 Could you update the PR description, too, based on the current PR approach? e.g., `Join is the equi-join.` -> `Join is the sort merge one with equality conditions` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column
AmplabJenkins removed a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-635723792 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable
maropu commented on a change in pull request #28123:
URL: https://github.com/apache/spark/pull/28123#discussion_r432206204
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInSortMergeJoin.scala
##
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.bucketing
+
+import org.apache.spark.sql.catalyst.catalog.BucketSpec
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.execution.{FileSourceScanExec, FilterExec,
ProjectExec, SparkPlan}
+import org.apache.spark.sql.execution.joins.SortMergeJoinExec
+import org.apache.spark.sql.internal.SQLConf
+
+/**
+ * This rule coalesces one side of the `SortMergeJoin` if the following
conditions are met:
+ * - Two bucketed tables are joined.
+ * - The larger bucket number is divisible by the smaller bucket number.
+ * - "spark.sql.bucketing.coalesceBucketsInJoin.enabled" is set to true.
Review comment:
nit: could you use `COALESCE_BUCKETS_IN_JOIN_ENABLED` instead of
`spark.sql.bucketing.coalesceBucketsInJoin.enabled`?
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -2586,6 +2586,26 @@ object SQLConf {
.checkValue(_ > 0, "The timeout value must be positive")
.createWithDefault(10L)
+ val COALESCE_BUCKETS_IN_JOIN_ENABLED =
+buildConf("spark.sql.bucketing.coalesceBucketsInJoin.enabled")
+ .doc("When true, if two bucketed tables with the different number of
buckets are joined, " +
+"the side with a bigger number of buckets will be coalesced to have
the same number " +
+"of buckets as the other side. This bucket coalescing can happen only
when the bigger " +
+"number of buckets is divisible by the smaller number of buckets.")
Review comment:
Could you describe that this rule is only applied to sort-merge joins?
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInSortMergeJoin.scala
##
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.bucketing
+
+import org.apache.spark.sql.catalyst.catalog.BucketSpec
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.execution.{FileSourceScanExec, FilterExec,
ProjectExec, SparkPlan}
+import org.apache.spark.sql.execution.joins.SortMergeJoinExec
+import org.apache.spark.sql.internal.SQLConf
+
+/**
+ * This rule coalesces one side of the `SortMergeJoin` if the following
conditions are met:
+ * - Two bucketed tables are joined.
+ * - The larger bucket number is divisible by the smaller bucket number.
+ * - "spark.sql.bucketing.coalesceBucketsInJoin.enabled" is set to true.
+ * - The difference in the number of buckets is less than the value set in
+ * "spark.sql.bucketing.coalesceBucketsInJoin.maxNumBucketsDiff".
+ */
+case class CoalesceBucketsInSortMergeJoin(conf: SQLConf) extends
Rule[SparkPlan] {
Review comment:
nit: unnecessary space found: `extends Rule[SparkPlan] {`
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInSortMergeJoin.scala
##
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work
[GitHub] [spark] AmplabJenkins commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column
AmplabJenkins commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-635723792 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column
SparkQA removed a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-635615292 **[Test build #123246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123246/testReport)** for PR 27066 at commit [`238f2f2`](https://github.com/apache/spark/commit/238f2f29ac913875d7f80884d89e0395fc468215). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column
SparkQA commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-635723180 **[Test build #123246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123246/testReport)** for PR 27066 at commit [`238f2f2`](https://github.com/apache/spark/commit/238f2f29ac913875d7f80884d89e0395fc468215). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635720616 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635720611 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635720332 **[Test build #123259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123259/testReport)** for PR 28331 at commit [`70c3871`](https://github.com/apache/spark/commit/70c3871ed801ed2b5e964e321bcfaba33f8735af). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
SparkQA commented on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635718288 **[Test build #123258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123258/testReport)** for PR 28651 at commit [`099d261`](https://github.com/apache/spark/commit/099d261c126496ecccd3900fa902fec60e272f10). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
AmplabJenkins removed a comment on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635716619 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
AmplabJenkins commented on pull request #28651: URL: https://github.com/apache/spark/pull/28651#issuecomment-635716619 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635715162 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123255/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635715157 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635715157 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635686780 **[Test build #123255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123255/testReport)** for PR 28647 at commit [`b30d62c`](https://github.com/apache/spark/commit/b30d62c884858cdfb065c19ef5dc3115aaa38a29). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
SparkQA commented on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635715025 **[Test build #123255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123255/testReport)** for PR 28647 at commit [`b30d62c`](https://github.com/apache/spark/commit/b30d62c884858cdfb065c19ef5dc3115aaa38a29). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28651: [SPARK-31833][SQL][test-hive1.2] Set HiveThriftServer2 with actual port while configured 0
yaooqinn commented on a change in pull request #28651:
URL: https://github.com/apache/spark/pull/28651#discussion_r432217803
##
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SharedThriftServer.scala
##
@@ -73,11 +68,19 @@ trait SharedThriftServer extends SharedSparkSession {
}
}
- private def startThriftServer(port: Int, attempt: Int): Unit = {
-logInfo(s"Trying to start HiveThriftServer2: port=$port, attempt=$attempt")
+ private def startThriftServer(attempt: Int): Unit = {
+logInfo(s"Trying to start HiveThriftServer2:, attempt=$attempt")
val sqlContext = spark.newSession().sqlContext
-sqlContext.setConf(ConfVars.HIVE_SERVER2_THRIFT_PORT.varname,
port.toString)
+// Set the HIVE_SERVER2_THRIFT_PORT to 0, so it could randomly pick any
free port to use.
+// It's much more robust than set a random port generated by ourselves
ahead
Review comment:
thanks.
the `HTTP` mode is not used here, so the `.._THRIFT_PORT` is enough
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635712678 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123254/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635712675 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635712675 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635712336 **[Test build #123254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123254/testReport)** for PR 28331 at commit [`fe265d7`](https://github.com/apache/spark/commit/fe265d7f576b4c3752bfb061060328a8c32ea2ad). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635673683 **[Test build #123254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123254/testReport)** for PR 28331 at commit [`fe265d7`](https://github.com/apache/spark/commit/fe265d7f576b4c3752bfb061060328a8c32ea2ad). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28661: [SPARK-31849][PYTHON][SQL] Make PySpark exceptions more Pythonic
HyukjinKwon commented on pull request #28661: URL: https://github.com/apache/spark/pull/28661#issuecomment-635710262 > if this is only removing "unhelpful" stack traces from the console, I can see it. I am pretty certain on this. Actually, this is why I am hiding JVM strackstrace on only whitelisted exception carefully pinked up, and why I added a switch just, just in case :-). I target to get rid of pretty meaningless JVM stacktrace here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28523: [SPARK-31706][SQL] add back the support of streaming update mode
HyukjinKwon commented on pull request #28523: URL: https://github.com/apache/spark/pull/28523#issuecomment-635705454 I guess we agree that the vote based clearly on misunderstanding should not stand in practice. Well, I agree with you all of it. What I would like to discuss about further is the fact that it's still being pointed out here as a violation of ASF policies - does it look clear to all of you? > Misunderstanding goes both ways, so I don't think it would be safe to simply ignore someone if you think they missed something. I am talking about the case that the vote is _clearly_ misunderstood - it was talking about a different thing what the proposal doesn't, and it blocked the RC, and people had to wait indefinitely just pinging. I assume the argument is that a justification clearly based on false is valid. > A veto without a justification is invalid and has no weight. which doesn't make sense to me. Maybe we should clarify it in the documentation if this looks violating the policy. If we're going to argue about the policy, it at least looks unclear to me if I read it literally from the documentation there - does it look clear to all of you? > PMC members have formally binding votes, but in general community members are encouraged to vote, even if their votes are only advisory. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28647: [SPARK-31828][SQL] Retain table properties at CreateTableLikeCommand
AmplabJenkins removed a comment on pull request #28647: URL: https://github.com/apache/spark/pull/28647#issuecomment-635704865 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
