[GitHub] [spark] maropu commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
maropu commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650502122 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #28880: [SPARK-29465][YARN][WEBUI] Adding Check to not to set UI port (spark.ui.port) property if mentioned explicitly

2020-06-27 Thread GitBox
SparkQA commented on pull request #28880: URL: https://github.com/apache/spark/pull/28880#issuecomment-650505296 **[Test build #124562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124562/testReport)** for PR 28880 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650512362 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650512362 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499675 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,22 @@ object

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499643 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,22 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650502968 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-650506382 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-650506382 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446500230 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala ## @@ -460,6 +460,40 @@ abstract class

[GitHub] [spark] SparkQA removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-650477379 **[Test build #124557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124557/testReport)** for PR 28708 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-650481575 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-27 Thread GitBox
SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-650506061 **[Test build #124557 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124557/testReport)** for PR 28708 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650511187 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650511499 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
maropu commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650511668 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650511499 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650511500 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-650511291 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650519318 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650519318 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28880: [SPARK-29465][YARN][WEBUI] Adding Check to not to set UI port (spark.ui.port) property if mentioned explicitly

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28880: URL: https://github.com/apache/spark/pull/28880#issuecomment-650505500 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28880: [SPARK-29465][YARN][WEBUI] Adding Check to not to set UI port (spark.ui.port) property if mentioned explicitly

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28880: URL: https://github.com/apache/spark/pull/28880#issuecomment-650505500 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
SparkQA commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650512191 **[Test build #124563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124563/testReport)** for PR 28919 at commit

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499394 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499537 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499537 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499494 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class

[GitHub] [spark] SparkQA commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650502780 **[Test build #124561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124561/testReport)** for PR 28676 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28880: [SPARK-29465][YARN][WEBUI] Adding Check to not to set UI port (spark.ui.port) property if mentioned explicitly

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28880: URL: https://github.com/apache/spark/pull/28880#issuecomment-650508322 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28880: [SPARK-29465][YARN][WEBUI] Adding Check to not to set UI port (spark.ui.port) property if mentioned explicitly

2020-06-27 Thread GitBox
SparkQA commented on pull request #28880: URL: https://github.com/apache/spark/pull/28880#issuecomment-650508289 **[Test build #124562 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124562/testReport)** for PR 28880 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28880: [SPARK-29465][YARN][WEBUI] Adding Check to not to set UI port (spark.ui.port) property if mentioned explicitly

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28880: URL: https://github.com/apache/spark/pull/28880#issuecomment-650508322 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28880: [SPARK-29465][YARN][WEBUI] Adding Check to not to set UI port (spark.ui.port) property if mentioned explicitly

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28880: URL: https://github.com/apache/spark/pull/28880#issuecomment-650505296 **[Test build #124562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124562/testReport)** for PR 28880 at commit

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650519071 **[Test build #124564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124564/testReport)** for PR 28841 at commit

[GitHub] [spark] cchighman commented on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
cchighman commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650518946 @HeartSaVioR The three files which had indentations without changes are now removed from this PR after corrections.

[GitHub] [spark] AmplabJenkins commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650502968 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650511184 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-650511288 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-650511288 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650511184 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-650488481 **[Test build #124560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124560/testReport)** for PR 27690 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650502780 **[Test build #124561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124561/testReport)** for PR 28676 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650475410 **[Test build #124554 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124554/testReport)** for PR 28919 at commit

[GitHub] [spark] SparkQA commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
SparkQA commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650511125 **[Test build #124554 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124554/testReport)** for PR 28919 at commit

[GitHub] [spark] SparkQA commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650511128 **[Test build #124561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124561/testReport)** for PR 28676 at commit

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

2020-06-27 Thread GitBox
SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-650511127 **[Test build #124560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124560/testReport)** for PR 27690 at commit

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499982 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala ## @@ -460,6 +460,40 @@ abstract class

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499910 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class

[GitHub] [spark] maropu commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
maropu commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650535975 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] maropu commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

2020-06-27 Thread GitBox
maropu commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650537977 hm..., I'm not sure about the root cause, but I think the simplest way to fix the issue is that we just remove the mockito part if its possible to test this PR without it.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650551433 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650551433 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650557976 **[Test build #124564 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124564/testReport)** for PR 28841 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650519071 **[Test build #124564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124564/testReport)** for PR 28841 at commit

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446508750 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,22 @@ object

[GitHub] [spark] dbaliafroozeh commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

2020-06-27 Thread GitBox
dbaliafroozeh commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r446514163 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -326,7 +327,8 @@ object QueryExecution { */

[GitHub] [spark] dbaliafroozeh commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

2020-06-27 Thread GitBox
dbaliafroozeh commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r446514163 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -326,7 +327,8 @@ object QueryExecution { */

[GitHub] [spark] wangyum opened a new pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

2020-06-27 Thread GitBox
wangyum opened a new pull request #28934: URL: https://github.com/apache/spark/pull/28934 ### What changes were proposed in this pull request? The data usually expand if joining event-based table(Chinese named 拉链表). This PR makes it avoid coalescing shuffle partitions if joining

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446502299 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,22 @@ object

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r446509317 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala ## @@ -488,6 +489,25 @@ class FileIndexSuite

[GitHub] [spark] AmplabJenkins commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536783 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536783 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536600 **[Test build #124565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124565/testReport)** for PR 28676 at commit

[GitHub] [spark] maropu commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node

2020-06-27 Thread GitBox
maropu commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-650536856 kindly ping @gengliangwang @cloud-fan This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] dbaliafroozeh commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

2020-06-27 Thread GitBox
dbaliafroozeh commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r446516852 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/reuse/Reuse.scala ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] moomindani commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

2020-06-27 Thread GitBox
moomindani commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r446516892 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -97,12 +99,38 @@ private[hive] trait

[GitHub] [spark] maropu commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-27 Thread GitBox
maropu commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-650552251 > it is more of a manual step and can be used only if the user knows the nature of data upfront.Like in my benchmark, where we expect the the all but few grouping keys to be

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650558195 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650558195 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650572743 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650572743 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-27 Thread GitBox
maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r446509229 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala ## @@ -126,4 +129,39 @@ class HiveMetadataCacheSuite extends

[GitHub] [spark] SparkQA commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
SparkQA commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650551206 **[Test build #124563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124563/testReport)** for PR 28919 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650512191 **[Test build #124563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124563/testReport)** for PR 28919 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536600 **[Test build #124565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124565/testReport)** for PR 28676 at commit

[GitHub] [spark] SparkQA commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

2020-06-27 Thread GitBox
SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650572247 **[Test build #124565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124565/testReport)** for PR 28676 at commit

[GitHub] [spark] viirya commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

2020-06-27 Thread GitBox
viirya commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r446540549 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -97,12 +99,38 @@ private[hive] trait SaveAsHiveFile

[GitHub] [spark] dongjoon-hyun commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

2020-06-27 Thread GitBox
dongjoon-hyun commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650586938 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589149 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589153 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589150 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589150 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588852 **[Test build #124571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124571/testReport)** for PR 28629 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-27 Thread GitBox
SparkQA removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650588854 **[Test build #124569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124569/testReport)** for PR 28863 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589148 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28541: [SPARK-31720][CORE] TaskMemoryManager allocate failed when new task coming

2020-06-27 Thread GitBox
dongjoon-hyun commented on a change in pull request #28541: URL: https://github.com/apache/spark/pull/28541#discussion_r446547861 ## File path: core/src/main/scala/org/apache/spark/memory/ExecutionMemoryPool.scala ## @@ -105,11 +106,12 @@ private[memory] class

[GitHub] [spark] SparkQA commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-27 Thread GitBox
SparkQA commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589145 **[Test build #124569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124569/testReport)** for PR 28863 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589003 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589003 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #28545: [WIP][SPARK-30090][SHELL] Adapt Spark REPL to Scala 2.13

2020-06-27 Thread GitBox
dongjoon-hyun commented on pull request #28545: URL: https://github.com/apache/spark/pull/28545#issuecomment-650589010 Hi, @karolchmist . Thank you for contribution. Apache Spark 3.0.0 is released. Can we resume this work for Apache Spark 3.1.0 (December 2020)?

[GitHub] [spark] AmplabJenkins commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588996 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650588999 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650588999 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588996 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589148 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

2020-06-27 Thread GitBox
SparkQA commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589144 **[Test build #124571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124571/testReport)** for PR 28629 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-650638287 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-650638287 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #24801: [SPARK-27950][DSTREAMS][Kinesis] dynamoDBEndpointUrl and cloudWatchMetricsLevel for Kinesis

2020-06-27 Thread GitBox
AmplabJenkins commented on pull request #24801: URL: https://github.com/apache/spark/pull/24801#issuecomment-650640959 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24801: [SPARK-27950][DSTREAMS][Kinesis] dynamoDBEndpointUrl and cloudWatchMetricsLevel for Kinesis

2020-06-27 Thread GitBox
AmplabJenkins removed a comment on pull request #24801: URL: https://github.com/apache/spark/pull/24801#issuecomment-599001276 Can one of the admins verify this patch? This is an automated message from the Apache Git

  1   2   3   4   >