[GitHub] [spark] yaooqinn commented on a change in pull request #28442: [SPARK-31631][TESTS] Fix test flakiness caused by MiniKdc which throws 'address in use' BindException with retry

2020-05-04 Thread GitBox
yaooqinn commented on a change in pull request #28442: URL: https://github.com/apache/spark/pull/28442#discussion_r419878281 ## File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala ## @@ -131,11 +130,7 @@ class KafkaTestUtils(

[GitHub] [spark] AmplabJenkins commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-623869047 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-623869047 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-05-04 Thread GitBox
cloud-fan commented on a change in pull request #28366: URL: https://github.com/apache/spark/pull/28366#discussion_r419877636 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2063,16 +2063,17 @@ object SQLConf { .booleanConf

[GitHub] [spark] SparkQA commented on pull request #28370: [SPARK-20732][CORE] Decommission cache blocks to other executors when an executor is decommissioned

2020-05-04 Thread GitBox
SparkQA commented on pull request #28370: URL: https://github.com/apache/spark/pull/28370#issuecomment-623868721 **[Test build #122302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122302/testReport)** for PR 28370 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28366: [SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-05-04 Thread GitBox
cloud-fan commented on a change in pull request #28366: URL: https://github.com/apache/spark/pull/28366#discussion_r419877190 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -179,15 +179,22 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623867741 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
SparkQA removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865188 **[Test build #122301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122301/testReport)** for PR 28451 at commit

[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623867673 **[Test build #122301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122301/testReport)** for PR 28451 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623867741 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] igreenfield commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
igreenfield commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623867478 I also ok with removing the default appId, appName. User will add what he needs. This is an automated

[GitHub] [spark] dilipbiswal commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
dilipbiswal commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419874633 ## File path: docs/sql-ref-identifier.md ## @@ -27,41 +27,34 @@ An identifier is a string used to identify a database object such as a table, vi

[GitHub] [spark] cloud-fan commented on pull request #28431: [SPARK-31623][SQL][TESTS] Benchmark rebasing of INT96 and TIMESTAMP_MILLIS timestamps in read/write

2020-05-04 Thread GitBox
cloud-fan commented on pull request #28431: URL: https://github.com/apache/spark/pull/28431#issuecomment-623865659 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865497 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865497 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623865188 **[Test build #122301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122301/testReport)** for PR 28451 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #27710: [SPARK-30960][SQL] add back the legacy date/timestamp format support in CSV/JSON parser

2020-05-04 Thread GitBox
cloud-fan commented on a change in pull request #27710: URL: https://github.com/apache/spark/pull/27710#discussion_r419872705 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala ## @@ -239,7 +246,23 @@ class JacksonParser(

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419872224 ## File path: docs/sql-ref-literals.md ## @@ -71,128 +68,114 @@ SELECT 'it\'s $10.' AS col; +-+ |It's $10.| +-+ -{% endhighlight %}

[GitHub] [spark] cloud-fan commented on a change in pull request #28310: [SPARK-31527][SQL] date add/subtract interval only allow those day precision in ansi mode

2020-05-04 Thread GitBox
cloud-fan commented on a change in pull request #28310: URL: https://github.com/apache/spark/pull/28310#discussion_r419872162 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -618,6 +618,22 @@ object DateTimeUtils {

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419872224 ## File path: docs/sql-ref-literals.md ## @@ -71,128 +68,114 @@ SELECT 'it\'s $10.' AS col; +-+ |It's $10.| +-+ -{% endhighlight %}

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419871991 ## File path: docs/sql-ref-identifier.md ## @@ -27,54 +27,47 @@ An identifier is a string used to identify a database object such as a table, vi

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419871927 ## File path: docs/sql-ref-literals.md ## @@ -35,22 +35,19 @@ A string literal is used to specify a character string value. Syntax -{%

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419871848 ## File path: docs/sql-ref-ansi-compliance.md ## @@ -66,7 +66,7 @@ This means that in case an operation causes overflows, the result is the same wi

[GitHub] [spark] cloud-fan commented on pull request #28445: [SPARK-31212][SQL][2.4] Fix Failure of casting the '1000-02-29' string to the date type

2020-05-04 Thread GitBox
cloud-fan commented on pull request #28445: URL: https://github.com/apache/spark/pull/28445#issuecomment-623863062 @MaxGekk what's your opinion? I'm fine with this fix but I won't encourage people to spend much time fixing datetime related bugs in 2.4. The datetime part is completely

[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
igreenfield commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r419869698 ## File path: docs/configuration.md ## @@ -2670,6 +2670,9 @@ Spark uses [log4j](http://logging.apache.org/log4j/) for logging. You can config

[GitHub] [spark] cloud-fan commented on pull request #28383: [SPARK-31590][SQL] Metadata-only queries should not include subquery in partition filters

2020-05-04 Thread GitBox
cloud-fan commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-623861615 Shall we remove `OptimizeMetadataOnlyQuery`? IIRC it has a correcness issue and we disable it by default. cc @gengliangwang

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623857362 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
SparkQA removed a comment on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623778576 **[Test build #122295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122295/testReport)** for PR 28239 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623857362 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic allocation

2020-05-04 Thread GitBox
dongjoon-hyun commented on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623857249 Thank you all! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
SparkQA commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623856851 **[Test build #122295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122295/testReport)** for PR 28239 at commit

[GitHub] [spark] cloud-fan commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
cloud-fan commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623855672 LGTM except for the app id/name. I'm still not convinced that it's working, at least @Ngone51 reported he can't see app id/name by local testing. Can you clearly point

[GitHub] [spark] cloud-fan commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
cloud-fan commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r419860787 ## File path: docs/configuration.md ## @@ -2670,6 +2670,9 @@ Spark uses [log4j](http://logging.apache.org/log4j/) for logging. You can config

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419857329 ## File path: docs/sql-ref-functions-udf-aggregate.md ## @@ -113,26 +102,26 @@ OPTIONS ( ); SELECT * FROM employees; --- +---+--+ --- |

[GitHub] [spark] gatorsmile commented on a change in pull request #28224: [SPARK-31429][SQL][DOC] Automatically generates a SQL document for built-in functions

2020-05-04 Thread GitBox
gatorsmile commented on a change in pull request #28224: URL: https://github.com/apache/spark/pull/28224#discussion_r419857270 ## File path: docs/sql-ref-functions-builtin.md ## @@ -0,0 +1,77 @@ +--- +layout: global +title: Built-in Functions +displayTitle: Built-in Functions

[GitHub] [spark] SparkQA removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
SparkQA removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623844664 **[Test build #122299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122299/testReport)** for PR 28009 at commit

[GitHub] [spark] SparkQA commented on pull request #28164: [SPARK-31393][SQL] Show the correct alias in schema for expression

2020-05-04 Thread GitBox
SparkQA commented on pull request #28164: URL: https://github.com/apache/spark/pull/28164#issuecomment-623848821 **[Test build #122300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122300/testReport)** for PR 28164 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623848806 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
SparkQA commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623848721 **[Test build #122299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122299/testReport)** for PR 28009 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623848806 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28164: [SPARK-31393][SQL] Show the correct alias in schema for expression

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28164: URL: https://github.com/apache/spark/pull/28164#issuecomment-623847685 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28164: [SPARK-31393][SQL] Show the correct alias in schema for expression

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28164: URL: https://github.com/apache/spark/pull/28164#issuecomment-623847685 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
SparkQA removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623821922 **[Test build #122298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122298/testReport)** for PR 28009 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623846376 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623846376 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
SparkQA commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623846317 **[Test build #122298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122298/testReport)** for PR 28009 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623845229 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623845229 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
SparkQA commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623844664 **[Test build #122299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122299/testReport)** for PR 28009 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623824088 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623824088 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28009: [SPARK-31235][YARN] Separates different categories of applications

2020-05-04 Thread GitBox
SparkQA commented on pull request #28009: URL: https://github.com/apache/spark/pull/28009#issuecomment-623821922 **[Test build #122298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122298/testReport)** for PR 28009 at commit

[GitHub] [spark] beliefer commented on pull request #28430: [SPARK-31372][SQL][TEST][FOLLOW-UP] Improve ExpressionsSchemaSuite so that easy to track the diff.

2020-05-04 Thread GitBox
beliefer commented on pull request #28430: URL: https://github.com/apache/spark/pull/28430#issuecomment-623813026 @HyukjinKwon Thanks for your help! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623811893 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623811893 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
SparkQA commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623811305 **[Test build #122294 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122294/testReport)** for PR 26624 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
SparkQA removed a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623774430 **[Test build #122294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122294/testReport)** for PR 26624 at commit

[GitHub] [spark] maropu commented on a change in pull request #28383: [SPARK-31590][SQL] Metadata-only queries should not include subquery in partition filters

2020-05-04 Thread GitBox
maropu commented on a change in pull request #28383: URL: https://github.com/apache/spark/pull/28383#discussion_r419817895 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala ## @@ -117,7 +117,7 @@ case class

[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
igreenfield commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r419832871 ## File path: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ## @@ -17,21 +17,106 @@ package org.apache.spark.util +import

[GitHub] [spark] AmplabJenkins commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-623800734 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-623800734 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable

2020-05-04 Thread GitBox
imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r419830462 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInJoin.scala ## @@ -0,0 +1,109 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable

2020-05-04 Thread GitBox
SparkQA commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-623800494 **[Test build #122297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122297/testReport)** for PR 28123 at commit

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable

2020-05-04 Thread GitBox
imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r419830393 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInJoin.scala ## @@ -0,0 +1,109 @@ +/* + * Licensed to the

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for join if applicable

2020-05-04 Thread GitBox
imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r419830249 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala ## @@ -221,3 +223,22 @@ object

[GitHub] [spark] dilipbiswal commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table

2020-05-04 Thread GitBox
dilipbiswal commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-623799161 @maropu @srowen Can this get in now, if there are no other comments ? The reason i ask is @huaxingao has a big PR which is changing a lot of files. If this can get in first

[GitHub] [spark] dilipbiswal commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
dilipbiswal commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419827929 ## File path: docs/sql-ref-literals.md ## @@ -71,128 +68,114 @@ SELECT 'it\'s $10.' AS col; +-+ |It's $10.| +-+ -{% endhighlight

[GitHub] [spark] dilipbiswal commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
dilipbiswal commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419826842 ## File path: docs/sql-ref-literals.md ## @@ -35,22 +35,19 @@ A string literal is used to specify a character string value. Syntax -{%

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623793408 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623793408 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
SparkQA removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623791128 **[Test build #122296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122296/testReport)** for PR 28451 at commit

[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623793351 **[Test build #122296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122296/testReport)** for PR 28451 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623791394 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] huaxingao commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623791469 > Rather, we should remove indents in the other places for following the result format? It's better to remove indents. Will spend some time to find all the error

[GitHub] [spark] AmplabJenkins commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623791394 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
SparkQA commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-623791128 **[Test build #122296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122296/testReport)** for PR 28451 at commit

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419818939 ## File path: docs/_data/menu-sql.yaml ## @@ -156,22 +156,22 @@ url: sql-ref-syntax-qry-select-distribute-by.html -

[GitHub] [spark] maropu commented on pull request #28383: [SPARK-31590][SQL] Metadata-only queries should not include subquery in partition filters

2020-05-04 Thread GitBox
maropu commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-623791024 > Applying OptimizeMetadataOnlyQuery rule will generate scalar-subquery. Is this statement true? It seems the test query itself has a subquery. ``` // Analyzed plan

[GitHub] [spark] maropu commented on a change in pull request #28383: [SPARK-31590][SQL] Metadata-only queries should not include subquery in partition filters

2020-05-04 Thread GitBox
maropu commented on a change in pull request #28383: URL: https://github.com/apache/spark/pull/28383#discussion_r419817895 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala ## @@ -117,7 +117,7 @@ case class

[GitHub] [spark] huaxingao commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
huaxingao commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419817932 ## File path: docs/sql-ref-functions-udf-aggregate.md ## @@ -113,26 +102,26 @@ OPTIONS ( ); SELECT * FROM employees; --- +---+--+ --- |

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic alloca

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623789508 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic alloca

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623789506 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic allocation

2020-05-04 Thread GitBox
SparkQA removed a comment on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623755415 **[Test build #122293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122293/testReport)** for PR 28452 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic allocation

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623789506 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic allocation

2020-05-04 Thread GitBox
SparkQA commented on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623789260 **[Test build #122293 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122293/testReport)** for PR 28452 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #28336: [SPARK-31559][YARN] Re-obtain tokens at the startup of AM for yarn cluster mode if principal and keytab are available

2020-05-04 Thread GitBox
HeartSaVioR commented on pull request #28336: URL: https://github.com/apache/spark/pull/28336#issuecomment-623787519 friendly reminder to @vanzin @squito also cc. @jerryshao, @tgravescs to expand the availability of reviews

[GitHub] [spark] HyukjinKwon commented on pull request #28452: [SPARK-27963][FOLLOW-UP][DOCS][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic allocation

2020-05-04 Thread GitBox
HyukjinKwon commented on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623787379 Merged to master and branch-3.0 since the related linter tests were already passed. I don't believe this change affects other tests or build.

[GitHub] [spark] HyukjinKwon commented on pull request #28452: [MINOR][CORE] Remove `for testing` because CleanerListener is used ExecutorMonitor during dynamic allocation

2020-05-04 Thread GitBox
HyukjinKwon commented on pull request #28452: URL: https://github.com/apache/spark/pull/28452#issuecomment-623787185 Let me just turn this to a followup of SPARK-27963 This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on pull request #28430: [SPARK-31372][SQL][TEST][FOLLOW-UP] Improve ExpressionsSchemaSuite so that easy to track the diff.

2020-05-04 Thread GitBox
HyukjinKwon commented on pull request #28430: URL: https://github.com/apache/spark/pull/28430#issuecomment-623786752 Merged to master and branc-3.0. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
HyukjinKwon commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r419813534 ## File path: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ## @@ -17,21 +17,106 @@ package org.apache.spark.util +import

[GitHub] [spark] maropu commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
maropu commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419790030 ## File path: docs/_data/menu-sql.yaml ## @@ -156,22 +156,22 @@ url: sql-ref-syntax-qry-select-distribute-by.html -

[GitHub] [spark] dilipbiswal commented on a change in pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

2020-05-04 Thread GitBox
dilipbiswal commented on a change in pull request #28451: URL: https://github.com/apache/spark/pull/28451#discussion_r419810299 ## File path: docs/sql-ref-ansi-compliance.md ## @@ -66,7 +66,7 @@ This means that in case an operation causes overflows, the result is the same wi

[GitHub] [spark] viirya commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
viirya commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r419808850 ## File path: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ## @@ -17,21 +17,106 @@ package org.apache.spark.util +import java.util

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623778968 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623778968 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
SparkQA commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623778576 **[Test build #122295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122295/testReport)** for PR 28239 at commit

[GitHub] [spark] maropu commented on pull request #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException

2020-05-04 Thread GitBox
maropu commented on pull request #28239: URL: https://github.com/apache/spark/pull/28239#issuecomment-623775852 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
SparkQA commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623774430 **[Test build #122294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122294/testReport)** for PR 26624 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
AmplabJenkins removed a comment on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623772844 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

2020-05-04 Thread GitBox
AmplabJenkins commented on pull request #26624: URL: https://github.com/apache/spark/pull/26624#issuecomment-623772844 This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   3   4   5   6   7   >