[GitHub] [spark] cloud-fan closed pull request #29015: [SPARK-32215] Expose a (protected) /workers/kill endpoint on the MasterWebUI

cloud-fan closed pull request #29015: URL: https://github.com/apache/spark/pull/29015 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] Ngone51 commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

Ngone51 commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r456236145 ## File path: core/src/main/scala/org/apache/spark/scheduler/ExecutorDecommissionInfo.scala ## @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] SparkQA commented on pull request #29117: [WIP] Debug flaky pip installation test failure

SparkQA commented on pull request #29117: URL: https://github.com/apache/spark/pull/29117#issuecomment-659892894 **[Test build #126031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126031/testReport)** for PR 29117 at commit

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-659893200 **[Test build #126033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126033/testReport)** for PR 27690 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29117: [WIP] Debug flaky pip installation test failure

AmplabJenkins commented on pull request #29117: URL: https://github.com/apache/spark/pull/29117#issuecomment-659893058 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] gaborgsomogyi commented on pull request #29131: [SPARK-32321][SS] Remove KAFKA-7703 workaround

gaborgsomogyi commented on pull request #29131: URL: https://github.com/apache/spark/pull/29131#issuecomment-659918072 I think this is conservative enough since retry still happens and now we're speaking about potential issues which has been never reported but I see your point, closing

[GitHub] [spark] maropu opened a new pull request #29143: [SPARK-32344][SQL] Unevaluable expr is set to FIRST/LAST ignoreNullsExpr in distinct aggregates

maropu opened a new pull request #29143: URL: https://github.com/apache/spark/pull/29143 ### What changes were proposed in this pull request? This PR intends to fix a bug of distinct FIRST/LAST aggregates in v2.4.6/v3.0.0/master; ``` scala> sql("SELECT FIRST(DISTINCT

[GitHub] [spark] beliefer commented on pull request #28917: [SPARK-31847][CORE][TESTS] DAGSchedulerSuite: Rewrite the test framework to support apply specified spark configurations.

beliefer commented on pull request #28917: URL: https://github.com/apache/spark/pull/28917#issuecomment-659917743 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] gaborgsomogyi closed pull request #29131: [SPARK-32321][SS] Remove KAFKA-7703 workaround

gaborgsomogyi closed pull request #29131: URL: https://github.com/apache/spark/pull/29131 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] maropu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

maropu commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r456293505 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +131,55 @@ trait

[GitHub] [spark] cloud-fan commented on a change in pull request #29135: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

cloud-fan commented on a change in pull request #29135: URL: https://github.com/apache/spark/pull/29135#discussion_r456311045 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDistinctAggregates.scala ## @@ -226,7 +226,10 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29117: [WIP] Debug flaky pip installation test failure

AmplabJenkins removed a comment on pull request #29117: URL: https://github.com/apache/spark/pull/29117#issuecomment-659970626 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29142: [SPARK-29343][SQL][FOLLOW-UP] Add more aggregate function to support eliminate sorts.

AmplabJenkins removed a comment on pull request #29142: URL: https://github.com/apache/spark/pull/29142#issuecomment-659877788 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29104: [SPARK-32290][SQL] NotInSubquery SingleColumn Optimize

AmplabJenkins commented on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-659895637 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29142: [SPARK-29343][SQL][FOLLOW-UP] Add more aggregate function to support eliminate sorts.

SparkQA commented on pull request #29142: URL: https://github.com/apache/spark/pull/29142#issuecomment-659895843 **[Test build #126034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126034/testReport)** for PR 29142 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29104: [SPARK-32290][SQL] NotInSubquery SingleColumn Optimize

AmplabJenkins removed a comment on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-659895637 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29104: [SPARK-32290][SQL] NotInSubquery SingleColumn Optimize

AmplabJenkins removed a comment on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-659895646 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29133: [SPARK-32253][INFRA] Show errors only for the sbt tests of github actions

dongjoon-hyun commented on a change in pull request #29133: URL: https://github.com/apache/spark/pull/29133#discussion_r456254309 ## File path: project/SparkBuild.scala ## @@ -1027,6 +1027,11 @@ object TestSettings { }.getOrElse(Nil): _*), // Show full stack trace

[GitHub] [spark] huaxingao commented on pull request #29139: [SPARK-32339][ML][DOC] Improve MLlib BLAS native acceleration docs

huaxingao commented on pull request #29139: URL: https://github.com/apache/spark/pull/29139#issuecomment-659905157 cc @srowen @viirya @zhengruifeng I scanned through. I think the doc is well written and the information is useful. Here are the screen capture: before

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29133: [SPARK-32253][INFRA] Show errors only for the sbt tests of github actions

dongjoon-hyun commented on a change in pull request #29133: URL: https://github.com/apache/spark/pull/29133#discussion_r456254721 ## File path: project/SparkBuild.scala ## @@ -1027,6 +1027,11 @@ object TestSettings { }.getOrElse(Nil): _*), // Show full stack trace

[GitHub] [spark] viirya commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

viirya commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r456254750 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -97,7 +99,42 @@ private[hive] trait SaveAsHiveFile

[GitHub] [spark] SparkQA commented on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

SparkQA commented on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659911810 **[Test build #126037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126037/testReport)** for PR 29141 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

AmplabJenkins removed a comment on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-659910694 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] maropu commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

maropu commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r456276640 ## File path: docs/sql-ref-syntax-qry-select-lateral-view.md ## @@ -0,0 +1,130 @@ +--- +layout: global +title: LATERAL VIEW Clause +displayTitle: LATERAL

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-659956760 **[Test build #126043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126043/testReport)** for PR 27690 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29135: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

cloud-fan commented on a change in pull request #29135: URL: https://github.com/apache/spark/pull/29135#discussion_r456313787 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggUtils.scala ## @@ -142,19 +142,25 @@ object AggUtils { val

[GitHub] [spark] AmplabJenkins commented on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

AmplabJenkins commented on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659912621 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

AmplabJenkins removed a comment on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659912621 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-659923949 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

maropu commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r456273005 ## File path: docs/sql-ref-syntax-ddl-create-table-hiveformat.md ## @@ -114,9 +162,45 @@ CREATE TABLE student (id INT, name STRING) PARTITIONED BY

[GitHub] [spark] JkSelf commented on a change in pull request #29021: [SPARK-32201][SQL] More general skew join pattern matching

JkSelf commented on a change in pull request #29021: URL: https://github.com/apache/spark/pull/29021#discussion_r456272490 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala ## @@ -340,3 +340,28 @@ case class

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r456301933 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +131,55 @@ trait

[GitHub] [spark] AmplabJenkins commented on pull request #29144: [SPARK-15118][SQL] Spark SQLConf should support loading hive properties in hive-site.xml

AmplabJenkins commented on pull request #29144: URL: https://github.com/apache/spark/pull/29144#issuecomment-659961545 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29144: [SPARK-15118][SQL] Spark SQLConf should support loading hive properties in hive-site.xml

AmplabJenkins removed a comment on pull request #29144: URL: https://github.com/apache/spark/pull/29144#issuecomment-659960920 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] LantaoJin commented on a change in pull request #29021: [SPARK-32201][SQL] More general skew join pattern matching

LantaoJin commented on a change in pull request #29021: URL: https://github.com/apache/spark/pull/29021#discussion_r456311324 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeLocalShuffleReader.scala ## @@ -142,6 +142,14 @@ object

[GitHub] [spark] LantaoJin commented on a change in pull request #29021: [SPARK-32201][SQL] More general skew join pattern matching

LantaoJin commented on a change in pull request #29021: URL: https://github.com/apache/spark/pull/29021#discussion_r456311324 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeLocalShuffleReader.scala ## @@ -142,6 +142,14 @@ object

[GitHub] [spark] LantaoJin commented on a change in pull request #29021: [SPARK-32201][SQL] More general skew join pattern matching

LantaoJin commented on a change in pull request #29021: URL: https://github.com/apache/spark/pull/29021#discussion_r456311324 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeLocalShuffleReader.scala ## @@ -142,6 +142,14 @@ object

[GitHub] [spark] holdenk commented on a change in pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

holdenk commented on a change in pull request #28911: URL: https://github.com/apache/spark/pull/28911#discussion_r456238717 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/BlockStoreClient.java ## @@ -61,4 +63,17 @@ public MetricSet

[GitHub] [spark] SparkQA commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

SparkQA commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-659908203 **[Test build #126035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126035/testReport)** for PR 29000 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29126: [SPARK-32324][SQL]Fix error messages during using PIVOT and lateral view

SparkQA removed a comment on pull request #29126: URL: https://github.com/apache/spark/pull/29126#issuecomment-659771033 **[Test build #126008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126008/testReport)** for PR 29126 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29138: [SPARK-32338] [SQL] Overload slice to accept Column for start and length

AmplabJenkins removed a comment on pull request #29138: URL: https://github.com/apache/spark/pull/29138#issuecomment-659907989 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29126: [SPARK-32324][SQL]Fix error messages during using PIVOT and lateral view

SparkQA commented on pull request #29126: URL: https://github.com/apache/spark/pull/29126#issuecomment-659907943 **[Test build #126008 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126008/testReport)** for PR 29126 at commit

[GitHub] [spark] maropu commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

maropu commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r456267828 ## File path: docs/sql-ref-syntax-qry-select-groupby.md ## @@ -38,6 +38,8 @@ GROUP BY GROUPING SETS (grouping_set [ , ...]) While aggregate functions are

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29143: [SPARK-32344][SQL] Unevaluable expr is set to FIRST/LAST ignoreNullsExpr in distinct aggregates

AmplabJenkins removed a comment on pull request #29143: URL: https://github.com/apache/spark/pull/29143#issuecomment-659920038 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

maropu commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r456274641 ## File path: docs/sql-ref-syntax-qry-select.md ## @@ -83,8 +85,14 @@ SELECT [ hints , ... ] [ ALL | DISTINCT ] { named_expression [ , ... ] } *

[GitHub] [spark] maropu commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

maropu commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r456283416 ## File path: docs/sql-ref-syntax-qry-select-pivot.md ## @@ -0,0 +1,100 @@ +--- +layout: global +title: PIVOT Clause +displayTitle: PIVOT Clause +license:

[GitHub] [spark] HyukjinKwon commented on pull request #29117: [WIP] Debug flaky pip installation test failure

HyukjinKwon commented on pull request #29117: URL: https://github.com/apache/spark/pull/29117#issuecomment-659965918 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] maropu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

maropu commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r456305487 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SparkScriptTransformationSuite.scala ## @@ -0,0 +1,121 @@ +/* + * Licensed to

[GitHub] [spark] LantaoJin commented on a change in pull request #29021: [SPARK-32201][SQL] More general skew join pattern matching

LantaoJin commented on a change in pull request #29021: URL: https://github.com/apache/spark/pull/29021#discussion_r456318955 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala ## @@ -340,3 +340,28 @@ case class

[GitHub] [spark] LantaoJin commented on a change in pull request #29021: [SPARK-32201][SQL] More general skew join pattern matching

LantaoJin commented on a change in pull request #29021: URL: https://github.com/apache/spark/pull/29021#discussion_r456318955 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala ## @@ -340,3 +340,28 @@ case class

[GitHub] [spark] cloud-fan commented on pull request #29015: [SPARK-32215] Expose a (protected) /workers/kill endpoint on the MasterWebUI

cloud-fan commented on pull request #29015: URL: https://github.com/apache/spark/pull/29015#issuecomment-659879374 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #29104: [SPARK-32290][SQL] NotInSubquery SingleColumn Optimize

SparkQA commented on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-659894625 **[Test build #126010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126010/testReport)** for PR 29104 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29104: [SPARK-32290][SQL] NotInSubquery SingleColumn Optimize

SparkQA removed a comment on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-659773807 **[Test build #126010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126010/testReport)** for PR 29104 at commit

[GitHub] [spark] SparkQA commented on pull request #29138: [SPARK-32338] [SQL] Overload slice to accept Column for start and length

SparkQA commented on pull request #29138: URL: https://github.com/apache/spark/pull/29138#issuecomment-659909550 **[Test build #126019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126019/testReport)** for PR 29138 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29101: [SPARK-32302][SQL] Partially push down disjunctive predicates through Join/Partitions

AmplabJenkins commented on pull request #29101: URL: https://github.com/apache/spark/pull/29101#issuecomment-659909926 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-659909543 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-659909546 **[Test build #126033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126033/testReport)** for PR 27690 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-659909907 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

SparkQA removed a comment on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659862760 **[Test build #126030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126030/testReport)** for PR 29141 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-659909907 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

SparkQA removed a comment on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-659771058 **[Test build #126009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126009/testReport)** for PR 27019 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-659824734 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

AmplabJenkins removed a comment on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659909840 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-659909670 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28840: [SPARK-31999][SQL] Add REFRESH FUNCTION command

AmplabJenkins removed a comment on pull request #28840: URL: https://github.com/apache/spark/pull/28840#issuecomment-659909802 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29101: [SPARK-32302][SQL] Partially push down disjunctive predicates through Join/Partitions

AmplabJenkins removed a comment on pull request #29101: URL: https://github.com/apache/spark/pull/29101#issuecomment-659909926 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

SparkQA removed a comment on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-659833193 **[Test build #126024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126024/testReport)** for PR 29020 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29142: [SPARK-29343][SQL][FOLLOW-UP] Add more aggregate function to support eliminate sorts.

AmplabJenkins removed a comment on pull request #29142: URL: https://github.com/apache/spark/pull/29142#issuecomment-659909624 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

SparkQA commented on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659909545 **[Test build #126030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126030/testReport)** for PR 29141 at commit

[GitHub] [spark] SparkQA commented on pull request #29079: [SPARK-32286][SQL] Coalesce bucketed table for shuffled hash join if applicable

SparkQA commented on pull request #29079: URL: https://github.com/apache/spark/pull/29079#issuecomment-659909552 **[Test build #126032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126032/testReport)** for PR 29079 at commit

[GitHub] [spark] SparkQA commented on pull request #29101: [SPARK-32302][SQL] Partially push down disjunctive predicates through Join/Partitions

SparkQA commented on pull request #29101: URL: https://github.com/apache/spark/pull/29101#issuecomment-659909548 **[Test build #126020 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126020/testReport)** for PR 29101 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-659909670 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29079: [SPARK-32286][SQL] Coalesce bucketed table for shuffled hash join if applicable

AmplabJenkins commented on pull request #29079: URL: https://github.com/apache/spark/pull/29079#issuecomment-659909720 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-659909812 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

SparkQA commented on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-659909560 **[Test build #126027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126027/testReport)** for PR 28852 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

AmplabJenkins commented on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659909840 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29138: [SPARK-32338] [SQL] Overload slice to accept Column for start and length

SparkQA removed a comment on pull request #29138: URL: https://github.com/apache/spark/pull/29138#issuecomment-659813043 **[Test build #126019 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126019/testReport)** for PR 29138 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28840: [SPARK-31999][SQL] Add REFRESH FUNCTION command

SparkQA removed a comment on pull request #28840: URL: https://github.com/apache/spark/pull/28840#issuecomment-659840659 **[Test build #126026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126026/testReport)** for PR 28840 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-659893200 **[Test build #126033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126033/testReport)** for PR 27690 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

AmplabJenkins commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-659910296 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #29141: [SPARK-32018][SQL][2.4] UnsafeRow.setDecimal should set null with overflowed value

dongjoon-hyun commented on pull request #29141: URL: https://github.com/apache/spark/pull/29141#issuecomment-659910194 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA removed a comment on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

SparkQA removed a comment on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-659856381 **[Test build #126029 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126029/testReport)** for PR 29032 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29079: [SPARK-32286][SQL] Coalesce bucketed table for shuffled hash join if applicable

SparkQA removed a comment on pull request #29079: URL: https://github.com/apache/spark/pull/29079#issuecomment-659874701 **[Test build #126032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126032/testReport)** for PR 29079 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28917: [SPARK-31847][CORE][TESTS] DAGSchedulerSuite: Rewrite the test framework to support apply specified spark configurations.

SparkQA removed a comment on pull request #28917: URL: https://github.com/apache/spark/pull/28917#issuecomment-659821944 **[Test build #126022 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126022/testReport)** for PR 28917 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

SparkQA removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-659844682 **[Test build #126027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126027/testReport)** for PR 28852 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28917: [SPARK-31847][CORE][TESTS] DAGSchedulerSuite: Rewrite the test framework to support apply specified spark configurations.

AmplabJenkins commented on pull request #28917: URL: https://github.com/apache/spark/pull/28917#issuecomment-659910207 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

AmplabJenkins commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-659910451 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29101: [SPARK-32302][SQL] Partially push down disjunctive predicates through Join/Partitions

SparkQA removed a comment on pull request #29101: URL: https://github.com/apache/spark/pull/29101#issuecomment-659816179 **[Test build #126020 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126020/testReport)** for PR 29101 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29142: [SPARK-29343][SQL][FOLLOW-UP] Add more aggregate function to support eliminate sorts.

SparkQA removed a comment on pull request #29142: URL: https://github.com/apache/spark/pull/29142#issuecomment-659895843 **[Test build #126034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126034/testReport)** for PR 29142 at commit

[GitHub] [spark] SparkQA commented on pull request #28840: [SPARK-31999][SQL] Add REFRESH FUNCTION command

SparkQA commented on pull request #28840: URL: https://github.com/apache/spark/pull/28840#issuecomment-659909556 **[Test build #126026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126026/testReport)** for PR 28840 at commit

[GitHub] [spark] SparkQA commented on pull request #29142: [SPARK-29343][SQL][FOLLOW-UP] Add more aggregate function to support eliminate sorts.

SparkQA commented on pull request #29142: URL: https://github.com/apache/spark/pull/29142#issuecomment-659909553 **[Test build #126034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126034/testReport)** for PR 29142 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-659909973 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

SparkQA commented on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-659909555 **[Test build #126024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126024/testReport)** for PR 29020 at commit

[GitHub] [spark] SparkQA commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

SparkQA commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-659909557 **[Test build #126029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126029/testReport)** for PR 29032 at commit

[GitHub] [spark] SparkQA commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

SparkQA commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-659909562 **[Test build #126009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126009/testReport)** for PR 27019 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

AmplabJenkins removed a comment on pull request #28852: URL: https://github.com/apache/spark/pull/28852#issuecomment-659923949 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #29056: [SPARK-31753][SQL][DOCS] Add missing keywords in the SQL docs

maropu commented on a change in pull request #29056: URL: https://github.com/apache/spark/pull/29056#discussion_r456270855 ## File path: docs/sql-ref-syntax-ddl-create-table-hiveformat.md ## @@ -114,9 +162,45 @@ CREATE TABLE student (id INT, name STRING) PARTITIONED BY

[GitHub] [spark] moomindani commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

moomindani commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r456294954 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -97,7 +99,42 @@ private[hive] trait

[GitHub] [spark] maropu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

maropu commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r456293505 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +131,55 @@ trait

[GitHub] [spark] cloud-fan commented on a change in pull request #29135: [SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

cloud-fan commented on a change in pull request #29135: URL: https://github.com/apache/spark/pull/29135#discussion_r456308513 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ProjectFilterInAggregates.scala ## @@ -0,0 +1,193 @@ +/* + *

  1   2   3   4   5   6   7   8   >