[GitHub] [spark] AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603605702 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120290/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603605694 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603605694 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603605702 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120290/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Eric5553 commented on issue #27897: [SPARK-31113][SQL] Add SHOW VIEWS command
Eric5553 commented on issue #27897: [SPARK-31113][SQL] Add SHOW VIEWS command URL: https://github.com/apache/spark/pull/27897#issuecomment-603605768 @dongjoon-hyun @cloud-fan @HyukjinKwon Is there any other comments that I should update? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
SparkQA removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603570620 **[Test build #120290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120290/testReport)** for PR 28006 at commit [`a439977`](https://github.com/apache/spark/commit/a439977e980e65e98d9954e5804925c39ea57053). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #27949: [SPARK-31150][SQL][Followup] Handle ' as escape for text
gatorsmile commented on issue #27949: [SPARK-31150][SQL][Followup] Handle ' as escape for text URL: https://github.com/apache/spark/pull/27949#issuecomment-603605312 @yaooqinn Please fix the title next time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
SparkQA commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603605396 **[Test build #120290 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120290/testReport)** for PR 28006 at commit [`a439977`](https://github.com/apache/spark/commit/a439977e980e65e98d9954e5804925c39ea57053). * This patch **fails PySpark pip packaging tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin edited a comment on issue #28000: [SPARK-30494][SQL][2.4] Fix cached data leakage during replacing an existing view
LantaoJin edited a comment on issue #28000: [SPARK-30494][SQL][2.4] Fix cached data leakage during replacing an existing view URL: https://github.com/apache/spark/pull/28000#issuecomment-603604136 Thank you, @dongjoon-hyun @HyukjinKwon @maropu This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] LantaoJin commented on issue #28000: [SPARK-30494][SQL][2.4] Fix cached data leakage during replacing an existing view
LantaoJin commented on issue #28000: [SPARK-30494][SQL][2.4] Fix cached data leakage during replacing an existing view URL: https://github.com/apache/spark/pull/28000#issuecomment-603604136 Thank you, @dongjoon-hyun @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #27935: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution
gatorsmile commented on issue #27935: [SPARK-25121][SQL] Supports multi-part table names for broadcast hint resolution URL: https://github.com/apache/spark/pull/27935#issuecomment-603601802 @maropu Could you fix the title and PR description? Add the test cases for the other join hints? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] rednaxelafx commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments
rednaxelafx commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603599059 Thank you very much for linking this PR to the existing JIRA ticket and its PR! I vaguely remember this had been implemented before but didn't find it in master and got curious... now I know >_<||| It's highly likely that this PR won't be good enough to make the bar, but I'd like to hear other folks' opinions. Thanks. Basically this is a hurdle that we cannot get past until the APIs are clarified and the semantics of `ScalaUDF.deterministic` is strong enough for optimizations to depend on: https://github.com/apache/spark/pull/24593#pullrequestreview-237361795 I had the same concerns when sending out this PR, and I'm more than okay to withdraw and close this PR given the current state of the API/semantics. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments
SparkQA commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603598928 **[Test build #120297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120297/testReport)** for PR 28008 at commit [`ab15f5c`](https://github.com/apache/spark/commit/ab15f5cbd4befc78abb7e1cafdd9729ec29b7216). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
SparkQA commented on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603598943 **[Test build #120300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120300/testReport)** for PR 28005 at commit [`ba437f7`](https://github.com/apache/spark/commit/ba437f786ef95e6fc583878fc25c101f7dff386d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait
SparkQA commented on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait URL: https://github.com/apache/spark/pull/27943#issuecomment-603598937 **[Test build #120298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120298/testReport)** for PR 27943 at commit [`9ff5ca2`](https://github.com/apache/spark/commit/9ff5ca20be633c17ceeabbaad46693da001ed943). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
SparkQA commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603598939 **[Test build #120299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120299/testReport)** for PR 28006 at commit [`e5879d2`](https://github.com/apache/spark/commit/e5879d27fdefa07ad6ba1c842524c118cc54a6b5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27974: [SPARK-31211][SQL] Fix rebasing of 29 February of Julian leap years
HyukjinKwon commented on issue #27974: [SPARK-31211][SQL] Fix rebasing of 29 February of Julian leap years URL: https://github.com/apache/spark/pull/27974#issuecomment-603597722 +1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait
AmplabJenkins commented on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait URL: https://github.com/apache/spark/pull/27943#issuecomment-603597250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25010/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603597219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25009/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait
AmplabJenkins removed a comment on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait URL: https://github.com/apache/spark/pull/27943#issuecomment-603597250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25010/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins removed a comment on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603597132 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait
AmplabJenkins commented on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait URL: https://github.com/apache/spark/pull/27943#issuecomment-603597247 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603597207 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins removed a comment on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603597137 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25007/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603597215 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25008/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603597216 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait
AmplabJenkins removed a comment on issue #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait URL: https://github.com/apache/spark/pull/27943#issuecomment-603597247 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603597137 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25007/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins commented on issue #28008: [SPARK-27692][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603597132 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins commented on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603597216 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603597207 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins commented on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603597215 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25008/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins commented on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603597219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25009/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei commented on a change in pull request #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait
turboFei commented on a change in pull request #27943: [SPARK-31179] Fast fail
the connection while last connection failed in the last retry IO wait
URL: https://github.com/apache/spark/pull/27943#discussion_r397566176
##
File path:
common/network-common/src/main/java/org/apache/spark/network/client/TransportClientFactory.java
##
@@ -192,7 +193,20 @@ public TransportClient createClient(String remoteHost,
int remotePort)
logger.info("Found inactive connection to {}, creating a new one.",
resolvedAddress);
}
}
- clientPool.clients[clientIndex] = createClient(resolvedAddress);
+ double fastFailTimeWindow = conf.ioRetryWaitTimeMs() * 0.95;
Review comment:
thanks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins removed a comment on issue #28009: [SPARK-31235][YARN] Separates different categories of applications URL: https://github.com/apache/spark/pull/28009#issuecomment-603593057 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications URL: https://github.com/apache/spark/pull/28009#issuecomment-603594661 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27943: [SPARK-31179] Fast fail the connection while last connection failed in the last retry IO wait
Ngone51 commented on a change in pull request #27943: [SPARK-31179] Fast fail
the connection while last connection failed in the last retry IO wait
URL: https://github.com/apache/spark/pull/27943#discussion_r397563886
##
File path:
common/network-common/src/main/java/org/apache/spark/network/client/TransportClientFactory.java
##
@@ -192,7 +193,20 @@ public TransportClient createClient(String remoteHost,
int remotePort)
logger.info("Found inactive connection to {}, creating a new one.",
resolvedAddress);
}
}
- clientPool.clients[clientIndex] = createClient(resolvedAddress);
+ double fastFailTimeWindow = conf.ioRetryWaitTimeMs() * 0.95;
Review comment:
Shall we make it a global variable?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications URL: https://github.com/apache/spark/pull/28009#issuecomment-603593057 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wang-zhun opened a new pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
wang-zhun opened a new pull request #28009: [SPARK-31235][YARN] Separates different categories of applications URL: https://github.com/apache/spark/pull/28009 ### What changes were proposed in this pull request? This PR adds `spark.yarn.applicationType` to identify the application type ### Why are the changes needed? The current application defaults to the SPARK type. In fact, different types of applications have different characteristics and are suitable for different scenarios.For example: SPAKR-SQL, SPARK-STREAMING. I recommend distinguishing them by the parameter `spark.yarn.applicationType` so that we can more easily manage and maintain different types of applications. ### How was this patch tested? add ut This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
maropu commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603592436 I looked over the two jira and I feel both are the same, so I will close the newer one as `duplicated` and update the PR title. If there is any problem, please re-update them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27967: [SPARK-31201][SQL] Add an individual config for skewed partition threshold
Ngone51 commented on a change in pull request #27967: [SPARK-31201][SQL] Add an
individual config for skewed partition threshold
URL: https://github.com/apache/spark/pull/27967#discussion_r397560722
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala
##
@@ -67,7 +67,7 @@ case class OptimizeSkewedJoin(conf: SQLConf) extends
Rule[SparkPlan] {
*/
private def isSkewed(size: Long, medianSize: Long): Boolean = {
size > medianSize *
conf.getConf(SQLConf.SKEW_JOIN_SKEWED_PARTITION_FACTOR) &&
- size > conf.getConf(SQLConf.ADVISORY_PARTITION_SIZE_IN_BYTES)
+ size > conf.getConf(SQLConf.SKEW_JOIN_SKEWED_PARTITION_THRESHOLD)
Review comment:
Do we still guarantee this after having s separate conf?
https://github.com/apache/spark/blob/30d95356f1881c32eb39e51525d2bcb331fcf867/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala#L196
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on issue #27995: [SPARK-31231][BUILD] Explicitly setuptools version as 46.0.0 in pip package test
HyukjinKwon edited a comment on issue #27995: [SPARK-31231][BUILD] Explicitly setuptools version as 46.0.0 in pip package test URL: https://github.com/apache/spark/pull/27995#issuecomment-603558061 @srowen, seems this can be the problem in client side with the latest `setuptools` (see pypa/setuptools#2041). So, I think using lower version now fixed the test but the issue itself persists. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #27937: [SPARK-30127][SQL] Support case class parameter for typed Scala UDF
Ngone51 commented on issue #27937: [SPARK-30127][SQL] Support case class parameter for typed Scala UDF URL: https://github.com/apache/spark/pull/27937#issuecomment-603588619 thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
HyukjinKwon commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603588453 FYI, there was a similar try at https://github.com/apache/spark/pull/24593 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks
Ngone51 commented on a change in pull request #27972: [SPARK-31207][CORE]
Ensure the total number of blocks to fetch equals to the sum of
local/hostLocal/remote blocks
URL: https://github.com/apache/spark/pull/27972#discussion_r397556929
##
File path:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
##
@@ -312,19 +310,57 @@ final class ShuffleBlockFetcherIterator(
hostLocalBlocks ++= blocksForAddress.map(info => (info._1, info._3))
hostLocalBlockBytes += mergedBlockInfos.map(_.size).sum
} else {
-numRemoteBlocks += blockInfos.size
remoteBlockBytes += blockInfos.map(_._2).sum
collectFetchRequests(address, blockInfos, collectedRemoteRequests)
}
}
+val numRemoteBlocks = collectedRemoteRequests.map(_.blocks.size).sum
val totalBytes = localBlockBytes + remoteBlockBytes + hostLocalBlockBytes
+assert(numBlocksToFetch == localBlocks.size + hostLocalBlocks.size +
numRemoteBlocks,
+ s"The number of non-empty blocks $numBlocksToFetch doesn't equal to the
number of local " +
+s"blocks ${localBlocks.size} + the number of host-local blocks
${hostLocalBlocks.size} " +
+s"+ the number of remote blocks ${numRemoteBlocks}.")
logInfo(s"Getting $numBlocksToFetch (${Utils.bytesToString(totalBytes)})
non-empty blocks " +
s"including ${localBlocks.size}
(${Utils.bytesToString(localBlockBytes)}) local and " +
s"${hostLocalBlocks.size} (${Utils.bytesToString(hostLocalBlockBytes)})
" +
s"host-local and $numRemoteBlocks
(${Utils.bytesToString(remoteBlockBytes)}) remote blocks")
collectedRemoteRequests
}
+ def createFetchRequest(
+ blocks: Seq[FetchBlockInfo],
+ address: BlockManagerId,
+ curRequestSize: Long): FetchRequest = {
+logDebug(s"Creating fetch request of $curRequestSize at $address "
+ + s"with ${blocks.size} blocks")
+FetchRequest(address, blocks)
+ }
+
+ def createFetchRequests(
+ curBlocks: ArrayBuffer[FetchBlockInfo],
+ address: BlockManagerId,
+ curRequestSize: Long,
+ isLast: Boolean,
+ collectedRemoteRequests: ArrayBuffer[FetchRequest]):
ArrayBuffer[FetchBlockInfo] = {
+val mergedBlocks = mergeContinuousShuffleBlockIdsIfNeeded(curBlocks)
+var retBlocks = new ArrayBuffer[FetchBlockInfo]
+if (mergedBlocks.length <= maxBlocksInFlightPerAddress) {
+ collectedRemoteRequests += createFetchRequest(mergedBlocks, address,
curRequestSize)
+} else {
+ mergedBlocks.grouped(maxBlocksInFlightPerAddress).foreach { blocks =>
+if (blocks.length == maxBlocksInFlightPerAddress || isLast) {
+ collectedRemoteRequests += createFetchRequest(blocks, address,
curRequestSize)
Review comment:
ok
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603587754 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120295/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603587749 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
SparkQA removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603585076 **[Test build #120295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120295/testReport)** for PR 28008 at commit [`aed59d7`](https://github.com/apache/spark/commit/aed59d7c14557b2330fcbd95a1eb9bb451375c94). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603587749 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603587754 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120295/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
SparkQA commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603587732 **[Test build #120295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120295/testReport)** for PR 28008 at commit [`aed59d7`](https://github.com/apache/spark/commit/aed59d7c14557b2330fcbd95a1eb9bb451375c94). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-603587006 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120287/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-603586998 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
AmplabJenkins commented on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-603586998 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
AmplabJenkins commented on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-603587006 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120287/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
SparkQA removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-603480820 **[Test build #120287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120287/testReport)** for PR 28001 at commit [`c7ccdde`](https://github.com/apache/spark/commit/c7ccdde7d0fbc806138512bd880d22fb33ea47dd). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
SparkQA commented on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-603586379 **[Test build #120287 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120287/testReport)** for PR 28001 at commit [`c7ccdde`](https://github.com/apache/spark/commit/c7ccdde7d0fbc806138512bd880d22fb33ea47dd). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603585496 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25005/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-603585544 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins removed a comment on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603585492 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-603585552 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25006/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603585492 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-603585544 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
AmplabJenkins commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603585496 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25005/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-603585552 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25006/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments
SparkQA commented on issue #28008: [SPARK-31240][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008#issuecomment-603585076 **[Test build #120295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120295/testReport)** for PR 28008 at commit [`aed59d7`](https://github.com/apache/spark/commit/aed59d7c14557b2330fcbd95a1eb9bb451375c94). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
SparkQA commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-603585031 **[Test build #120296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120296/testReport)** for PR 27690 at commit [`073b2e5`](https://github.com/apache/spark/commit/073b2e5aa299225dae830542247be0f15488eba9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
maropu commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-603584398 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
maropu commented on a change in pull request #27991: [SPARK-31227][SQL]
Non-nullable null type in complex types should not coerce to nullable type
URL: https://github.com/apache/spark/pull/27991#discussion_r397552138
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
##
@@ -210,8 +206,15 @@ object Cast {
case _ => false // overflow
}
+ /**
+ * Returns `true` iff it should change the nullability of this type in map
type,
+ * array type, expressions such as cast, etc.
+ *
+ * Note that you should take the nullability context into account.
+ * For example, it should not force to nullable type when null type in array
+ * type is non-nullable which means an empty array of null type.
+ */
def forceNullable(from: DataType, to: DataType): Boolean = (from, to) match {
-case (NullType, _) => true
Review comment:
We can do this now?
```
case (NullType, _) => false // empty array or map case
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] rednaxelafx opened a new pull request #28008: [WIP][SPARK-XXXXX][SQL] Constant fold deterministic Scala UDFs with foldable arguments
rednaxelafx opened a new pull request #28008: [WIP][SPARK-X][SQL] Constant fold deterministic Scala UDFs with foldable arguments URL: https://github.com/apache/spark/pull/28008 ### What changes were proposed in this pull request? Constant fold deterministic Scala UDFs with foldable arguments, conservatively. `ScalaUDF`s that meet all following criteria are subject to constant folding in this PR: - deterministic - all arguments are foldable - does not throw an exception when evaluating the UDF for constant folding ### Why are the changes needed? This is an optimization that enables more constant folding, improving the performance of a special case of Scala UDFs. Catalyst already implements constant folding of expressions in a few places, through the pattern `if (e.foldable) e.eval(EmptyRow)`. One of the optimizer rules that specifically performs constant folding is `ConstantFolding`. `ScalaUDF` does not override `foldable`, so it isn't subject to constant folding right now. To enable constant folding of `ScalaUDF`s, it's tempting to make it override `foldable`: ```scala override lazy val foldable: Boolean = deterministic && children.forall(_.foldable) ``` But `ScalaUDF`s are declared as deterministic by default, so it's possible for users to mis-declare a potentially exception-throwing UDF as deterministic, so for callers of the "constant folding" code pattern that do not expect an exception, overriding `ScalaUDF.foldable` in general may have a wide impact. Instead, this PR tackles the problem in a conservative way, where the constant folding of `ScalaUDF`s is only implemented in the `ConstantFolding` rule, with special handling to catch non-fatal exceptions when evaluating a Scala UDF during constant folding and skip folding such UDFs. ### Does this PR introduce any user-facing change? No. Caveat: this PR assumes certain semantics of `Expression.deterministic`. Although it guards against potentially exception-throwing Scala UDFs being mis-declared as deterministic, it cannot guard against other kinds of mis-declarations, e.g. - UDFs that are only expected to be invoked on executors; - UDFs that return the same value but have other side-effects, such as modifying a static field. Users should explicitly mark such UDFs as `asNondeterministic`. ### How was this patch tested? Added a test case in `ConstantFoldingSuite`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds`
AmplabJenkins commented on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds` URL: https://github.com/apache/spark/pull/28007#issuecomment-603581243 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds
AmplabJenkins removed a comment on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds` URL: https://github.com/apache/spark/pull/28007#issuecomment-603581243 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds
AmplabJenkins removed a comment on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds` URL: https://github.com/apache/spark/pull/28007#issuecomment-603581249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25003/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks
AmplabJenkins commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks URL: https://github.com/apache/spark/pull/27972#issuecomment-603581198 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds`
AmplabJenkins commented on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds` URL: https://github.com/apache/spark/pull/28007#issuecomment-603581249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25003/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks
AmplabJenkins removed a comment on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks URL: https://github.com/apache/spark/pull/27972#issuecomment-603581202 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25004/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks
AmplabJenkins removed a comment on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks URL: https://github.com/apache/spark/pull/27972#issuecomment-603581198 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks
AmplabJenkins commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks URL: https://github.com/apache/spark/pull/27972#issuecomment-603581202 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25004/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks
SparkQA commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks URL: https://github.com/apache/spark/pull/27972#issuecomment-603580917 **[Test build #120294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120294/testReport)** for PR 27972 at commit [`4560610`](https://github.com/apache/spark/commit/45606109fee2e128840f09fe140c397298014a27). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds`
SparkQA commented on issue #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds` URL: https://github.com/apache/spark/pull/28007#issuecomment-603580916 **[Test build #120293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120293/testReport)** for PR 28007 at commit [`bb4df54`](https://github.com/apache/spark/commit/bb4df549c96d961b80a5d07470417f6309d340a5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks
jiangxb1987 commented on issue #27972: [SPARK-31207][CORE] Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks URL: https://github.com/apache/spark/pull/27972#issuecomment-603580489 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 opened a new pull request #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds`
jiangxb1987 opened a new pull request #28007: [SPARK-31239][CORE][TEST] Increase await duration in `WorkerDecommissionSuite`.`verify a task with all workers decommissioned succeeds` URL: https://github.com/apache/spark/pull/28007 ### What changes were proposed in this pull request? The test case has been flaky because the execution time sometimes exceeds the await duration. Increase the await duration to avoid flakiness. ### How was this patch tested? Tested locally and it didn't fail anymore. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on issue #28002: [SPARK-31233][Core] Enhance RpcTimeoutException Log Message
jiangxb1987 commented on issue #28002: [SPARK-31233][Core] Enhance RpcTimeoutException Log Message URL: https://github.com/apache/spark/pull/28002#issuecomment-603576473 OK to test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28002: [SPARK-31233][Core] Enhance RpcTimeoutException Log Message
AmplabJenkins removed a comment on issue #28002: [SPARK-31233][Core] Enhance RpcTimeoutException Log Message URL: https://github.com/apache/spark/pull/28002#issuecomment-603242586 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on issue #28002: [SPARK-31233][Core] Enhance RpcTimeoutException Log Message
jiangxb1987 commented on issue #28002: [SPARK-31233][Core] Enhance RpcTimeoutException Log Message URL: https://github.com/apache/spark/pull/28002#issuecomment-603576441 Please revert the unnecessary code reformat changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable URL: https://github.com/apache/spark/pull/27815#issuecomment-603573214 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603573076 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603573082 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25002/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
AmplabJenkins removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable URL: https://github.com/apache/spark/pull/27815#issuecomment-603573223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120288/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable URL: https://github.com/apache/spark/pull/27815#issuecomment-603573214 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
AmplabJenkins commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable URL: https://github.com/apache/spark/pull/27815#issuecomment-603573223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120288/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603573076 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603573082 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25002/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
SparkQA removed a comment on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable URL: https://github.com/apache/spark/pull/27815#issuecomment-603502417 **[Test build #120288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120288/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
SparkQA commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603572691 **[Test build #120292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120292/testReport)** for PR 27991 at commit [`05dc916`](https://github.com/apache/spark/commit/05dc91695139b1e5f5c94eb117ba34c76bd3a3ff). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable
SparkQA commented on issue #27815: [SPARK-31088][SQL] Add back HiveContext and createExternalTable URL: https://github.com/apache/spark/pull/27815#issuecomment-603572648 **[Test build #120288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120288/testReport)** for PR 27815 at commit [`1f1a570`](https://github.com/apache/spark/commit/1f1a570d5b4940841a7d2cb4eb2fd800b8573007). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `final class ANOVASelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)` * `final class VarianceThresholdSelector @Since(\"3.1.0\")(@Since(\"3.1.0\") override val uid: String)` * `// `case class Foo(i: Int) extends AnyVal` will return type `Int` instead of `Foo`.` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603570961 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603570966 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25001/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603570982 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25000/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28006: [DO-NOT-MERGE][SPARK-31231][BUILD][2.4][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28006#issuecomment-603570970 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
