[GitHub] [spark] AmplabJenkins removed a comment on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
AmplabJenkins removed a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623885390 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122307/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang edited a comment on pull request #28383: [SPARK-31590][SQL] Metadata-only queries should not include subquery in partition filters
gengliangwang edited a comment on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-623875587 > Shall we remove OptimizeMetadataOnlyQuery? IIRC it has a correcness issue and we disable it by default. cc @gengliangwang On second thought: I think we should keep it for two reasons: 1. when users are 100% sure about their data won't contain empty partition, they can still turn it on. 2. the future developers may come up with the same idea and create exactly the same rule and enable it by default... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
AmplabJenkins removed a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623885385 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
SparkQA removed a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623879122 **[Test build #122307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122307/testReport)** for PR 26624 at commit [`c13dada`](https://github.com/apache/spark/commit/c13dada80dbc80b72aec38f9746e52b131e222c5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
AmplabJenkins commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623885385 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
SparkQA commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623885348 **[Test build #122307 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122307/testReport)** for PR 26624 at commit [`c13dada`](https://github.com/apache/spark/commit/c13dada80dbc80b72aec38f9746e52b131e222c5). * This patch **fails to generate documentation**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623883859 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623883859 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623883544 **[Test build #122308 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122308/testReport)** for PR 28451 at commit [`5726a1f`](https://github.com/apache/spark/commit/5726a1f080f7bffc9a863d79f7ddb8b674716b4e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
cloud-fan commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419892413
##
File path: project/SparkBuild.scala
##
@@ -905,7 +905,7 @@ object Unidoc {
object Checkstyle {
lazy val settings = Seq(
-checkstyleSeverityLevel := Some(CheckstyleSeverityLevel.Error),
+checkstyleSeverityLevel := Some(CheckstyleSeverityLevel.Warning),
Review comment:
is this also a mistake?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
igreenfield commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419892755
##
File path: project/SparkBuild.scala
##
@@ -905,7 +905,7 @@ object Unidoc {
object Checkstyle {
lazy val settings = Seq(
-checkstyleSeverityLevel := Some(CheckstyleSeverityLevel.Error),
+checkstyleSeverityLevel := Some(CheckstyleSeverityLevel.Warning),
Review comment:
yes, reverted
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
igreenfield commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419891389
##
File path: project/SparkBuild.scala
##
@@ -172,7 +172,7 @@ object SparkBuild extends PomBuild {
val configV = (baseDirectory in ThisBuild).value /
scalaStyleOnCompileConfig
val configUrlV = scalastyleConfigUrl.in(config).value
val streamsV = streams.in(config).value
-val failOnErrorV = true
+val failOnErrorV = false
Review comment:
push revert
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
AmplabJenkins removed a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623879484 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
AmplabJenkins commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623879484 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28441: [SPARK-31630][SQL] Fix perf regression by skipping timestamps rebasing after some threshold
AmplabJenkins removed a comment on pull request #28441: URL: https://github.com/apache/spark/pull/28441#issuecomment-623879398 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28441: [SPARK-31630][SQL] Fix perf regression by skipping timestamps rebasing after some threshold
AmplabJenkins commented on pull request #28441: URL: https://github.com/apache/spark/pull/28441#issuecomment-623879398 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
cloud-fan commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419890587
##
File path: project/SparkBuild.scala
##
@@ -172,7 +172,7 @@ object SparkBuild extends PomBuild {
val configV = (baseDirectory in ThisBuild).value /
scalaStyleOnCompileConfig
val configUrlV = scalastyleConfigUrl.in(config).value
val streamsV = streams.in(config).value
-val failOnErrorV = true
+val failOnErrorV = false
Review comment:
why this change?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
igreenfield commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419890734
##
File path: project/SparkBuild.scala
##
@@ -172,7 +172,7 @@ object SparkBuild extends PomBuild {
val configV = (baseDirectory in ThisBuild).value /
scalaStyleOnCompileConfig
val configUrlV = scalastyleConfigUrl.in(config).value
val streamsV = streams.in(config).value
-val failOnErrorV = true
+val failOnErrorV = false
Review comment:
mistake
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
SparkQA commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623879122 **[Test build #122307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122307/testReport)** for PR 26624 at commit [`c13dada`](https://github.com/apache/spark/commit/c13dada80dbc80b72aec38f9746e52b131e222c5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28441: [SPARK-31630][SQL] Fix perf regression by skipping timestamps rebasing after some threshold
SparkQA commented on pull request #28441: URL: https://github.com/apache/spark/pull/28441#issuecomment-623879076 **[Test build #122306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122306/testReport)** for PR 28441 at commit [`eaca6a8`](https://github.com/apache/spark/commit/eaca6a86eb182fabeeb5db487724646ab631568c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
cloud-fan commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623877465 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
maropu commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-623877304 Ur, the jira ID is incorrect? @dilipbiswal This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
maropu commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-623876997 Thanks! Merged to master/3.0. If @srowen has more comments, please ping @huaxingao so that she can include more fixes in her PR (#28451). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
dilipbiswal commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-623876888 Thanks a lot @maropu @huaxingao @srowen @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #28441: [SPARK-31630][SQL] Fix perf regression by skipping timestamps rebasing after some threshold
MaxGekk commented on a change in pull request #28441:
URL: https://github.com/apache/spark/pull/28441#discussion_r419886514
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
##
@@ -286,6 +286,17 @@ object RebaseDateTime {
*/
private val gregJulianRebaseMap =
loadRebaseRecords("gregorian-julian-rebase-micros.json")
+ private def getLastSwitchTs(rebaseMap: AnyRefMap[String, RebaseInfo]): Long
= {
+val latestTs = rebaseMap.values.map(_.switches.last).max
+require(rebaseMap.values.forall(_.diffs.last == 0),
Review comment:
It uses `latestTs` in the error message. I am going to improve the
message by converting micros to Instant, so, toString should form nicer string:
```scala
require(rebaseMap.values.forall(_.diffs.last == 0),
s"Differences between Julian and Gregorian calendar after
${microsToInstant(latestTs)} " +
"are expected to be zero for available time zones.")
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
maropu commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-623875717 Looks fine and I'll merge this first. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
AmplabJenkins removed a comment on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-623875337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #28383: [SPARK-31590][SQL] Metadata-only queries should not include subquery in partition filters
gengliangwang commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-623875587 > Shall we remove OptimizeMetadataOnlyQuery? IIRC it has a correcness issue and we disable it by default. cc @gengliangwang I am +1 with that. The correctness issue is the one I reported in https://github.com/apache/spark/pull/23635 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
AmplabJenkins commented on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-623875337 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28224: [SPARK-31429][SQL][DOC] Automatically generates a SQL document for built-in functions
maropu commented on a change in pull request #28224:
URL: https://github.com/apache/spark/pull/28224#discussion_r419884736
##
File path: docs/sql-ref-functions-builtin.md
##
@@ -0,0 +1,77 @@
+---
+layout: global
+title: Built-in Functions
+displayTitle: Built-in Functions
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-agg-funcs-table.html' %}
+### Aggregate Functions
+{% include_relative generated-agg-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-window-funcs-table.html' %}
+### Window Functions
+{% include_relative generated-window-funcs-table.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-array-funcs-table.html' %}
+### Array Functions
+{% include_relative generated-array-funcs-table.html %}
+ Examples
+{% include_relative generated-array-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-map-funcs-table.html' %}
+### Map Functions
+{% include_relative generated-map-funcs-table.html %}
+ Examples
+{% include_relative generated-map-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-datetime-funcs-table.html' %}
+### Date and Timestamp Functions
+{% include_relative generated-datetime-funcs-table.html %}
+ Examples
+{% include_relative generated-datetime-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-json-funcs-table.html' %}
+### JSON Functions
+{% include_relative generated-json-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
Review comment:
Thanks, @huaxingao
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
dilipbiswal commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623875264 Nice @huaxingao . LGTM - had some very minor comments. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
SparkQA commented on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-623874913 **[Test build #122305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122305/testReport)** for PR 28366 at commit [`00b9d47`](https://github.com/apache/spark/commit/00b9d47702ae76fca3c7246155175cb42f75136f). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
maropu commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623874394 Thanks! Merged to master/3.0. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
igreenfield commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419884091
##
File path: docs/configuration.md
##
@@ -2670,6 +2670,9 @@ Spark uses [log4j](http://logging.apache.org/log4j/) for
logging. You can config
`log4j.properties` file in the `conf` directory. One way to start is to copy
the existing
`log4j.properties.template` located there.
+By default, Spark adds to the MDC 1 record: `taskName` you can add that to
your patternLayout `%X{taskName}` in order to print in the logs
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28224: [SPARK-31429][SQL][DOC] Automatically generates a SQL document for built-in functions
huaxingao commented on a change in pull request #28224:
URL: https://github.com/apache/spark/pull/28224#discussion_r419884345
##
File path: docs/sql-ref-functions-builtin.md
##
@@ -0,0 +1,77 @@
+---
+layout: global
+title: Built-in Functions
+displayTitle: Built-in Functions
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-agg-funcs-table.html' %}
+### Aggregate Functions
+{% include_relative generated-agg-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-window-funcs-table.html' %}
+### Window Functions
+{% include_relative generated-window-funcs-table.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-array-funcs-table.html' %}
+### Array Functions
+{% include_relative generated-array-funcs-table.html %}
+ Examples
+{% include_relative generated-array-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-map-funcs-table.html' %}
+### Map Functions
+{% include_relative generated-map-funcs-table.html %}
+ Examples
+{% include_relative generated-map-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-datetime-funcs-table.html' %}
+### Date and Timestamp Functions
+{% include_relative generated-datetime-funcs-table.html %}
+ Examples
+{% include_relative generated-datetime-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-json-funcs-table.html' %}
+### JSON Functions
+{% include_relative generated-json-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
Review comment:
Sure. I will fix this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623874343 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623874239 **[Test build #122303 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122303/testReport)** for PR 28451 at commit [`804d15d`](https://github.com/apache/spark/commit/804d15d9093e2da9721ea6b1505118f91e9eb1f1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623874343 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
SparkQA removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623870833 **[Test build #122303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122303/testReport)** for PR 28451 at commit [`804d15d`](https://github.com/apache/spark/commit/804d15d9093e2da9721ea6b1505118f91e9eb1f1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28393: [SPARK-31595][SQL] Spark sql should allow unescaped quote mark in quoted string
cloud-fan commented on a change in pull request #28393:
URL: https://github.com/apache/spark/pull/28393#discussion_r419883101
##
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
##
@@ -500,4 +500,13 @@ class CliSuite extends SparkFunSuite with
BeforeAndAfterAll with BeforeAndAfterE
| ;""".stripMargin -> "testcomment"
)
}
+
+ test("SPARK-31595 Should allow unescaped quote mark in quoted string") {
+runCliWithin(1.minute)(
+ """SELECT '"legal string a';select 1 + 234;""".stripMargin -> "235"
+)
+runCliWithin(1.minute)(
+ """SELECT "legal 'string b";select 2 + 1;""".stripMargin -> "3"
Review comment:
nit: let's not use the multiline string style for a single line string.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
AmplabJenkins removed a comment on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-623873145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
AmplabJenkins commented on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-623873145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28224: [SPARK-31429][SQL][DOC] Automatically generates a SQL document for built-in functions
maropu commented on a change in pull request #28224:
URL: https://github.com/apache/spark/pull/28224#discussion_r419882097
##
File path: docs/sql-ref-functions-builtin.md
##
@@ -0,0 +1,77 @@
+---
+layout: global
+title: Built-in Functions
+displayTitle: Built-in Functions
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-agg-funcs-table.html' %}
+### Aggregate Functions
+{% include_relative generated-agg-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-window-funcs-table.html' %}
+### Window Functions
+{% include_relative generated-window-funcs-table.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-array-funcs-table.html' %}
+### Array Functions
+{% include_relative generated-array-funcs-table.html %}
+ Examples
+{% include_relative generated-array-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-map-funcs-table.html' %}
+### Map Functions
+{% include_relative generated-map-funcs-table.html %}
+ Examples
+{% include_relative generated-map-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-datetime-funcs-table.html' %}
+### Date and Timestamp Functions
+{% include_relative generated-datetime-funcs-table.html %}
+ Examples
+{% include_relative generated-datetime-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-json-funcs-table.html' %}
+### JSON Functions
+{% include_relative generated-json-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
Review comment:
yea... @huaxingao @dilipbiswal Could you include this fix in your open
PRs? #28451
or 28433
##
File path: docs/sql-ref-functions-builtin.md
##
@@ -0,0 +1,77 @@
+---
+layout: global
+title: Built-in Functions
+displayTitle: Built-in Functions
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-agg-funcs-table.html' %}
+### Aggregate Functions
+{% include_relative generated-agg-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-window-funcs-table.html' %}
+### Window Functions
+{% include_relative generated-window-funcs-table.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-array-funcs-table.html' %}
+### Array Functions
+{% include_relative generated-array-funcs-table.html %}
+ Examples
+{% include_relative generated-array-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-map-funcs-table.html' %}
+### Map Fu
[GitHub] [spark] cloud-fan commented on pull request #28441: [SPARK-31630][SQL] Fix perf regression by skipping timestamps rebasing after some threshold
cloud-fan commented on pull request #28441: URL: https://github.com/apache/spark/pull/28441#issuecomment-623872965 LGTM, let's regenerate the benchmark result to fix conflicts. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
dilipbiswal commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419882287
##
File path: docs/sql-ref-syntax-aux-show-partitions.md
##
@@ -27,37 +27,28 @@ partition spec.
### Syntax
-{% highlight sql %}
+```sql
SHOW PARTITIONS table_identifier [ partition_spec ]
-{% endhighlight %}
+```
### Parameters
-
- table_identifier
-
-Specifies a table name, which may be optionally qualified with a database
name.
-Syntax:
-
-[ database_name. ] table_name
-
-
-
-
- partition_spec
-
+* **table_identifier**
+
+Specifies a table name, which may be optionally qualified with a database
name.
+
+**Syntax:** `[ database_name. ] table_name`
+
+* **partition_spec**
+
An optional parameter that specifies a comma separated list of key and
value pairs
-for partitions. When specified, the partitions that match the partition
spec are returned.
-Syntax:
-
-PARTITION ( partition_col_name [ = partition_col_val ] [ , ... ] )
-
-
-
+for partitions. When specified, the partitions that match the partition
spec are returned.
Review comment:
@huaxingao just for consistency lets change "partition spec" to
"partition specification" ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28441: [SPARK-31630][SQL] Fix perf regression by skipping timestamps rebasing after some threshold
cloud-fan commented on a change in pull request #28441:
URL: https://github.com/apache/spark/pull/28441#discussion_r419882017
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
##
@@ -286,6 +286,17 @@ object RebaseDateTime {
*/
private val gregJulianRebaseMap =
loadRebaseRecords("gregorian-julian-rebase-micros.json")
+ private def getLastSwitchTs(rebaseMap: AnyRefMap[String, RebaseInfo]): Long
= {
+val latestTs = rebaseMap.values.map(_.switches.last).max
+require(rebaseMap.values.forall(_.diffs.last == 0),
Review comment:
ideally `require` should be the first line in a method.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28224: [SPARK-31429][SQL][DOC] Automatically generates a SQL document for built-in functions
maropu commented on a change in pull request #28224:
URL: https://github.com/apache/spark/pull/28224#discussion_r419882097
##
File path: docs/sql-ref-functions-builtin.md
##
@@ -0,0 +1,77 @@
+---
+layout: global
+title: Built-in Functions
+displayTitle: Built-in Functions
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-agg-funcs-table.html' %}
+### Aggregate Functions
+{% include_relative generated-agg-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-window-funcs-table.html' %}
+### Window Functions
+{% include_relative generated-window-funcs-table.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-array-funcs-table.html' %}
+### Array Functions
+{% include_relative generated-array-funcs-table.html %}
+ Examples
+{% include_relative generated-array-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-map-funcs-table.html' %}
+### Map Functions
+{% include_relative generated-map-funcs-table.html %}
+ Examples
+{% include_relative generated-map-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-datetime-funcs-table.html' %}
+### Date and Timestamp Functions
+{% include_relative generated-datetime-funcs-table.html %}
+ Examples
+{% include_relative generated-datetime-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-json-funcs-table.html' %}
+### JSON Functions
+{% include_relative generated-json-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
Review comment:
yea... @huaxingao @dilipbiswal Could you include this fix in your open
PRs?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
SparkQA commented on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-623872783 **[Test build #122304 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122304/testReport)** for PR 28366 at commit [`aa32dcc`](https://github.com/apache/spark/commit/aa32dcc303877caa6de070143c222cc4b9eeb4db). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
cloud-fan commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419881372
##
File path: docs/configuration.md
##
@@ -2670,6 +2670,9 @@ Spark uses [log4j](http://logging.apache.org/log4j/) for
logging. You can config
`log4j.properties` file in the `conf` directory. One way to start is to copy
the existing
`log4j.properties.template` located there.
+By default, Spark adds to the MDC 1 record: `taskName` you can add that to
your patternLayout `%X{taskName}` in order to print in the logs
Review comment:
```
By default, Spark adds 1 record to the MDC: `taskName`, which shows something
like `task 1.0 in stage 0.0`. You can add `%X{taskName}` to your
patternLayout in
order to print it in the logs.
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
cloud-fan commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419881372
##
File path: docs/configuration.md
##
@@ -2670,6 +2670,9 @@ Spark uses [log4j](http://logging.apache.org/log4j/) for
logging. You can config
`log4j.properties` file in the `conf` directory. One way to start is to copy
the existing
`log4j.properties.template` located there.
+By default, Spark adds to the MDC 1 record: `taskName` you can add that to
your patternLayout `%X{taskName}` in order to print in the logs
Review comment:
```
By default, Spark adds 1 record to the MDC: `taskName`, which shows
something like `task 1.0 in stage 0.0`. You can add `%X{taskName}` to your
patternLayout in order to print it in the logs.
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623871603 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623871603 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623870833 **[Test build #122303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122303/testReport)** for PR 28451 at commit [`804d15d`](https://github.com/apache/spark/commit/804d15d9093e2da9721ea6b1505118f91e9eb1f1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28442: [SPARK-31631][TESTS] Fix test flakiness caused by MiniKdc which throws 'address in use' BindException with retry
yaooqinn commented on a change in pull request #28442:
URL: https://github.com/apache/spark/pull/28442#discussion_r419878281
##
File path:
external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala
##
@@ -131,11 +130,7 @@ class KafkaTestUtils(
}
private def setUpMiniKdc(): Unit = {
-val kdcDir = Utils.createTempDir()
Review comment:
Do I still need to address this comment？
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned
AmplabJenkins commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-623869047 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned
AmplabJenkins removed a comment on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-623869047 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned
SparkQA commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-623868721 **[Test build #122302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122302/testReport)** for PR 28370 at commit [`c645582`](https://github.com/apache/spark/commit/c645582e2df06fe4736dc3b1673b377d4baf96f0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
cloud-fan commented on a change in pull request #28366:
URL: https://github.com/apache/spark/pull/28366#discussion_r419877636
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -2063,16 +2063,17 @@ object SQLConf {
.booleanConf
.createWithDefault(true)
- val NESTED_PREDICATE_PUSHDOWN_ENABLED =
-buildConf("spark.sql.optimizer.nestedPredicatePushdown.enabled")
- .internal()
- .doc("When true, Spark tries to push down predicates for nested columns
and or names " +
-"containing `dots` to data sources. Currently, Parquet implements both
optimizations " +
-"while ORC only supports predicates for names containing `dots`. The
other data sources" +
-"don't support this feature yet.")
+ val NESTED_PREDICATE_PUSHDOWN_V1_SOURCE_LIST =
+buildConf("spark.sql.optimizer.nestedPredicatePushdown.supportedV1Sources")
Review comment:
`supportedV1Sources` -> `supportedFileSources`?
DS v1 and file source are different APIs and have different planner
rules/physical nodes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources
cloud-fan commented on a change in pull request #28366:
URL: https://github.com/apache/spark/pull/28366#discussion_r419877190
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
##
@@ -179,15 +179,22 @@ class DataSourceV2Strategy(session: SparkSession) extends
Strategy with Predicat
case OverwriteByExpression(r: DataSourceV2Relation, deleteExpr, query,
writeOptions, _) =>
// fail if any filter cannot be converted. correctness depends on
removing all matching data.
- val filters = splitConjunctivePredicates(deleteExpr).map {
-filter => DataSourceStrategy.translateFilter(deleteExpr).getOrElse(
- throw new AnalysisException(s"Cannot translate expression to source
filter: $filter"))
- }.toArray
+ val filters = splitConjunctivePredicates(deleteExpr)
+ def transferFilters =
+(filters: Seq[Expression], supportNestedPredicatePushdown: Boolean) =>
{
Review comment:
Do we need the `supportNestedPredicatePushdown` parameter here as the
caller side always pass true?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623867741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
SparkQA removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865188 **[Test build #122301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122301/testReport)** for PR 28451 at commit [`289e5ae`](https://github.com/apache/spark/commit/289e5aea19b2b027efa37fbfe7bd723824b02b92). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623867673 **[Test build #122301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122301/testReport)** for PR 28451 at commit [`289e5ae`](https://github.com/apache/spark/commit/289e5aea19b2b027efa37fbfe7bd723824b02b92). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623867741 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] igreenfield commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
igreenfield commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623867478 I also ok with removing the default appId, appName. User will add what he needs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
dilipbiswal commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419874633
##
File path: docs/sql-ref-identifier.md
##
@@ -27,41 +27,34 @@ An identifier is a string used to identify a database
object such as a table, vi
Regular Identifier
-{% highlight sql %}
+```sql
{ letter | digit | '_' } [ , ... ]
-{% endhighlight %}
+```
Note: If `spark.sql.ansi.enabled` is set to true, ANSI SQL reserved keywords
cannot be used as identifiers. For more details, please refer to [ANSI
Compliance](sql-ref-ansi-compliance.html).
Review comment:
@huaxingao Should we bold "Note" ? I see that in other places we do bold
it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28431: [SPARK-31623][SQL][TESTS] Benchmark rebasing of INT96 and TIMESTAMP_MILLIS timestamps in read/write
cloud-fan commented on pull request #28431: URL: https://github.com/apache/spark/pull/28431#issuecomment-623865659 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865497 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865497 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865188 **[Test build #122301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122301/testReport)** for PR 28451 at commit [`289e5ae`](https://github.com/apache/spark/commit/289e5aea19b2b027efa37fbfe7bd723824b02b92). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27710: [SPARK-30960][SQL] add back the legacy date/timestamp format support in CSV/JSON parser
cloud-fan commented on a change in pull request #27710:
URL: https://github.com/apache/spark/pull/27710#discussion_r419872705
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala
##
@@ -239,7 +246,23 @@ class JacksonParser(
case DateType =>
(parser: JsonParser) => parseJsonToken[java.lang.Integer](parser,
dataType) {
case VALUE_STRING if parser.getTextLength >= 1 =>
- dateFormatter.parse(parser.getText)
+ try {
+dateFormatter.parse(parser.getText)
+ } catch {
+case NonFatal(e) =>
+ // If fails to parse, then tries the way used in 2.0 and 1.x for
backwards
+ // compatibility.
+ val str =
UTF8String.fromString(DateTimeUtils.cleanLegacyTimestampStr(parser.getText))
+ DateTimeUtils.stringToDate(str, options.zoneId).getOrElse {
+// In Spark 1.5.0, we store the data as number of days since
epoch in string.
+// So, we just convert it to Int.
+try {
+ parser.getText.toInt
Review comment:
good catch! I think we should.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
huaxingao commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419872224
##
File path: docs/sql-ref-literals.md
##
@@ -71,128 +68,114 @@ SELECT 'it\'s $10.' AS col;
+-+
|It's $10.|
+-+
-{% endhighlight %}
+```
### Binary Literal
A binary literal is used to specify a byte sequence value.
Syntax
-{% highlight sql %}
+```sql
X { 'c [ ... ]' | "c [ ... ]" }
-{% endhighlight %}
+```
+
+ Parameters
- Parameters
+* **c**
-
- c
-
One character from the character set.
Review comment:
seems to be hexadecimal. Changed to the following:
```
Syntax
X { 'num [ ... ]' | "num [ ... ]" }
Parameters
* **num**
Any hexadecimal number from 0 to F.
```
cc @yaooqinn
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #28310: [SPARK-31527][SQL] date add/subtract interval only allow those day precision in ansi mode
cloud-fan commented on a change in pull request #28310:
URL: https://github.com/apache/spark/pull/28310#discussion_r419872162
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##
@@ -618,6 +618,22 @@ object DateTimeUtils {
instantToMicros(resultTimestamp.toInstant)
}
+ /**
+ * Add the date and the interval's months and days.
+ * Returns a date value, expressed in days since 1.1.1970.
+ *
+ * @throws DateTimeException if the result exceeds the supported date range
+ * @throws IllegalArgumentException if the interval has `microseconds` part
+ */
+ def dateAddInterval(
+ start: SQLDate,
+ interval: CalendarInterval): SQLDate = {
+require(interval.microseconds == 0,
+ "Cannot add hours, minutes or seconds, milliseconds, microseconds to a
date")
+val ld =
LocalDate.ofEpochDay(start).plusMonths(interval.months).plusDays(interval.days)
Review comment:
FYI, in snowflake `internal '1 month 1 day'` is different from `internal
'1 day 1 month'`. We should at least document our own behavior.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
huaxingao commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419872224
##
File path: docs/sql-ref-literals.md
##
@@ -71,128 +68,114 @@ SELECT 'it\'s $10.' AS col;
+-+
|It's $10.|
+-+
-{% endhighlight %}
+```
### Binary Literal
A binary literal is used to specify a byte sequence value.
Syntax
-{% highlight sql %}
+```sql
X { 'c [ ... ]' | "c [ ... ]" }
-{% endhighlight %}
+```
+
+ Parameters
- Parameters
+* **c**
-
- c
-
One character from the character set.
Review comment:
seems to be hexadecimal. Changed to the following:
```
Syntax
```sql
X { 'num [ ... ]' | "num [ ... ]" }
```
Parameters
* **num**
Any hexadecimal number from 0 to F.
```
cc @yaooqinn
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
huaxingao commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419871927
##
File path: docs/sql-ref-literals.md
##
@@ -35,22 +35,19 @@ A string literal is used to specify a character string
value.
Syntax
-{% highlight sql %}
+```sql
'c [ ... ]' | "c [ ... ]"
Review comment:
changed to ```char```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
huaxingao commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419871991
##
File path: docs/sql-ref-identifier.md
##
@@ -27,54 +27,47 @@ An identifier is a string used to identify a database
object such as a table, vi
Regular Identifier
-{% highlight sql %}
+```sql
{ letter | digit | '_' } [ , ... ]
-{% endhighlight %}
+```
Note: If `spark.sql.ansi.enabled` is set to true, ANSI SQL reserved keywords
cannot be used as identifiers. For more details, please refer to [ANSI
Compliance](sql-ref-ansi-compliance.html).
Delimited Identifier
-{% highlight sql %}
+```sql
`c [ ... ]`
-{% endhighlight %}
+```
### Parameters
-
- letter
-
+* **letter**
+
Any letter from A-Z or a-z.
-
-
-
- digit
-
+
+* **digit**
+
Any numeral from 0 to 9.
-
-
-
- c
-
+
+* **c**
+
Any character from the character set. Use ` to escape special
characters (e.g., `).
-
-
### Examples
-{% highlight sql %}
+```sql
-- This CREATE TABLE fails with ParseException because of the illegal
identifier name a.b
CREATE TABLE test (a.b int);
-org.apache.spark.sql.catalyst.parser.ParseException:
-no viable alternative at input 'CREATE TABLE test (a.'(line 1, pos 20)
+ org.apache.spark.sql.catalyst.parser.ParseException:
Review comment:
Fixed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
huaxingao commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419871848
##
File path: docs/sql-ref-ansi-compliance.md
##
@@ -66,7 +66,7 @@ This means that in case an operation causes overflows, the
result is the same wi
On the other hand, Spark SQL returns null for decimal overflows.
When `spark.sql.ansi.enabled` is set to `true` and an overflow occurs in
numeric and interval arithmetic operations, it throws an arithmetic exception
at runtime.
-{% highlight sql %}
+```sql
-- `spark.sql.ansi.enabled=true`
Review comment:
I don't have a strong opinion on this. seems to me comment is OK too.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28445: [SPARK-31212][SQL][2.4] Fix Failure of casting the '1000-02-29' string to the date type
cloud-fan commented on pull request #28445: URL: https://github.com/apache/spark/pull/28445#issuecomment-623863062 @MaxGekk what's your opinion? I'm fine with this fix but I won't encourage people to spend much time fixing datetime related bugs in 2.4. The datetime part is completely rewritten in 3.0. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
igreenfield commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419869698
##
File path: docs/configuration.md
##
@@ -2670,6 +2670,9 @@ Spark uses [log4j](http://logging.apache.org/log4j/) for
logging. You can config
`log4j.properties` file in the `conf` directory. One way to start is to copy
the existing
`log4j.properties.template` located there.
+By default, Spark adds to the MDC 3 records: `appId`, `appName` and `taskName`
you can add that to your patternLayout `%X{appId}` in order to print in the logs
Review comment:
Maybe in both places?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28383: [SPARK-31590][SQL] Metadata-only queries should not include subquery in partition filters
cloud-fan commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-623861615 Shall we remove `OptimizeMetadataOnlyQuery`? IIRC it has a correcness issue and we disable it by default. cc @gengliangwang This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623857362 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
SparkQA removed a comment on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623778576 **[Test build #122295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122295/testReport)** for PR 28239 at commit [`453c5a5`](https://github.com/apache/spark/commit/453c5a5e0717d2681fc2e0ed4f48ca093d4020a0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623857362 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic allocation
dongjoon-hyun commented on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623857249 Thank you all! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
SparkQA commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623856851 **[Test build #122295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122295/testReport)** for PR 28239 at commit [`453c5a5`](https://github.com/apache/spark/commit/453c5a5e0717d2681fc2e0ed4f48ca093d4020a0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
cloud-fan commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623855672 LGTM except for the app id/name. I'm still not convinced that it's working, at least @Ngone51 reported he can't see app id/name by local testing. Can you clearly point out the code that sets app id/name? You mentioned it's in DAG scheduler, can you point out which line? It's even better if you can add a test. BTW I think it's OK to ask users to set app id/name themselves by `mdc.appId/Name`. I'm good with this patch if we just remove the handling of app id/name. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor
cloud-fan commented on a change in pull request #26624:
URL: https://github.com/apache/spark/pull/26624#discussion_r419860787
##
File path: docs/configuration.md
##
@@ -2670,6 +2670,9 @@ Spark uses [log4j](http://logging.apache.org/log4j/) for
logging. You can config
`log4j.properties` file in the `conf` directory. One way to start is to copy
the existing
`log4j.properties.template` located there.
+By default, Spark adds to the MDC 3 records: `appId`, `appName` and `taskName`
you can add that to your patternLayout `%X{appId}` in order to print in the logs
Review comment:
I think it's better to put the doc in `conf/log4j.properties.template`,
where users use this feature.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference
huaxingao commented on a change in pull request #28451:
URL: https://github.com/apache/spark/pull/28451#discussion_r419857329
##
File path: docs/sql-ref-functions-udf-aggregate.md
##
@@ -113,26 +102,26 @@ OPTIONS (
);
SELECT * FROM employees;
--- +---+--+
--- | name|salary|
--- +---+--+
--- |Michael| 3000|
--- | Andy| 4500|
--- | Justin| 3500|
--- | Berta| 4000|
--- +---+--+
++---+--+
+| name|salary|
++---+--+
+|Michael| 3000|
+| Andy| 4500|
+| Justin| 3500|
+| Berta| 4000|
++---+--+
SELECT myAverage(salary) as average_salary FROM employees;
--- +--+
--- |average_salary|
--- +--+
--- |3750.0|
--- +--+
-{% endhighlight %}
++--+
+|average_salary|
++--+
+|3750.0|
++--+
+```
Review comment:
This is for examples ``. I prefer to keep this
since we use this format for all the examples.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #28224: [SPARK-31429][SQL][DOC] Automatically generates a SQL document for built-in functions
gatorsmile commented on a change in pull request #28224:
URL: https://github.com/apache/spark/pull/28224#discussion_r419857270
##
File path: docs/sql-ref-functions-builtin.md
##
@@ -0,0 +1,77 @@
+---
+layout: global
+title: Built-in Functions
+displayTitle: Built-in Functions
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-agg-funcs-table.html' %}
+### Aggregate Functions
+{% include_relative generated-agg-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-window-funcs-table.html' %}
+### Window Functions
+{% include_relative generated-window-funcs-table.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-array-funcs-table.html' %}
+### Array Functions
+{% include_relative generated-array-funcs-table.html %}
+ Examples
+{% include_relative generated-array-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-map-funcs-table.html' %}
+### Map Functions
+{% include_relative generated-map-funcs-table.html %}
+ Examples
+{% include_relative generated-map-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-datetime-funcs-table.html' %}
+### Date and Timestamp Functions
+{% include_relative generated-datetime-funcs-table.html %}
+ Examples
+{% include_relative generated-datetime-funcs-examples.html %}
+{% break %}
+{% endif %}
+{% endfor %}
+
+{% for static_file in site.static_files %}
+{% if static_file.name == 'generated-json-funcs-table.html' %}
+### JSON Functions
+{% include_relative generated-json-funcs-table.html %}
+ Examples
+{% include_relative generated-agg-funcs-examples.html %}
Review comment:
generated-agg-funcs-examples.html -> generated-json-funcs-examples.html ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
SparkQA removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623844664 **[Test build #122299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122299/testReport)** for PR 28009 at commit [`b762753`](https://github.com/apache/spark/commit/b762753d9642d7c5b1faa8d5dcaa6402c95730c1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28164: [SPARK-31393][SQL] Show the correct alias in schema for expression
SparkQA commented on pull request #28164: URL: https://github.com/apache/spark/pull/28164#issuecomment-623848821 **[Test build #122300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122300/testReport)** for PR 28164 at commit [`cc4ee4c`](https://github.com/apache/spark/commit/cc4ee4c7b09ee9c09f40ac1d4f714db6a83838d4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623848806 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
SparkQA commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623848721 **[Test build #122299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122299/testReport)** for PR 28009 at commit [`b762753`](https://github.com/apache/spark/commit/b762753d9642d7c5b1faa8d5dcaa6402c95730c1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623848806 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28164: [SPARK-31393][SQL] Show the correct alias in schema for expression
AmplabJenkins removed a comment on pull request #28164: URL: https://github.com/apache/spark/pull/28164#issuecomment-623847685 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28164: [SPARK-31393][SQL] Show the correct alias in schema for expression
AmplabJenkins commented on pull request #28164: URL: https://github.com/apache/spark/pull/28164#issuecomment-623847685 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
SparkQA removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623821922 **[Test build #122298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122298/testReport)** for PR 28009 at commit [`4599e18`](https://github.com/apache/spark/commit/4599e18141efce8cc241fd1f9f4d5b84e3a297e7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623846376 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623846376 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
SparkQA commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623846317 **[Test build #122298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122298/testReport)** for PR 28009 at commit [`4599e18`](https://github.com/apache/spark/commit/4599e18141efce8cc241fd1f9f4d5b84e3a297e7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623845229 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
