[GitHub] [spark] HyukjinKwon opened a new pull request #29057: [SPARK-32245][INFRA] Run Spark tests in Github Actions

HyukjinKwon opened a new pull request #29057: URL: https://github.com/apache/spark/pull/29057 ### What changes were proposed in this pull request? This PR aims to run the Spark tests in Github Actions. To briefly explain the main idea: - Reuse `dev/run-tests.py` with

[GitHub] [spark] maropu commented on a change in pull request #29055: [SPARK-32251][SQL][DOC] fix SQL keyword document

maropu commented on a change in pull request #29055: URL: https://github.com/apache/spark/pull/29055#discussion_r452243955 ## File path: docs/sql-ref-ansi-compliance.md ## @@ -193,7 +193,7 @@ Below is a list of all the keywords in Spark SQL.

[GitHub] [spark] HyukjinKwon commented on pull request #29057: [SPARK-32245][INFRA] Run Spark tests in Github Actions

HyukjinKwon commented on pull request #29057: URL: https://github.com/apache/spark/pull/29057#issuecomment-656149858 I already individually tested, and I am pretty sure it will show a green light. See also https://github.com/HyukjinKwon/spark/runs/854113657

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

AmplabJenkins removed a comment on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656148940 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #29055: [SPARK-32251][SQL][DOC] fix SQL keyword document

maropu commented on a change in pull request #29055: URL: https://github.com/apache/spark/pull/29055#discussion_r452243955 ## File path: docs/sql-ref-ansi-compliance.md ## @@ -193,7 +193,7 @@ Below is a list of all the keywords in Spark SQL.

[GitHub] [spark] tgravescs commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

tgravescs commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-656148922 yeah 10x definitely seems safe as most of the number are more at the 8x number for zstd. I'm fine with leaving the current logic for the small files, we can always follow up

[GitHub] [spark] AmplabJenkins commented on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

AmplabJenkins commented on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656148940 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Ngone51 commented on pull request #28850: [SPARK-32015][Core]Remote inheritable thread local variables after spark context is stopped

Ngone51 commented on pull request #28850: URL: https://github.com/apache/spark/pull/28850#issuecomment-656148667 @srowen Let me help explain a bit more: `activeThreadSession` is a `InheritableThreadLocal`. Therefore, any child thread of the Spark main thread could keep a reference

[GitHub] [spark] SparkQA removed a comment on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

SparkQA removed a comment on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656111930 **[Test build #125471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125471/testReport)** for PR 29036 at commit

[GitHub] [spark] SparkQA commented on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

SparkQA commented on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656147462 **[Test build #125471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125471/testReport)** for PR 29036 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29055: [SPARK-32251][SQL][DOC] fix SQL keyword document

AmplabJenkins removed a comment on pull request #29055: URL: https://github.com/apache/spark/pull/29055#issuecomment-656145312 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29055: [SPARK-32251][SQL][DOC] fix SQL keyword document

SparkQA commented on pull request #29055: URL: https://github.com/apache/spark/pull/29055#issuecomment-656145164 **[Test build #125480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125480/testReport)** for PR 29055 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29055: [SPARK-32251][SQL][DOC] fix SQL keyword document

AmplabJenkins commented on pull request #29055: URL: https://github.com/apache/spark/pull/29055#issuecomment-656145312 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29056: [SPARK-31753][SQL][DOCS][WIP]Add missing keywords

AmplabJenkins removed a comment on pull request #29056: URL: https://github.com/apache/spark/pull/29056#issuecomment-656144321 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29056: [SPARK-31753][SQL][DOCS][WIP]Add missing keywords

AmplabJenkins commented on pull request #29056: URL: https://github.com/apache/spark/pull/29056#issuecomment-656144859 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29056: [SPARK-31753][SQL][DOCS][WIP]Add missing keywords

AmplabJenkins commented on pull request #29056: URL: https://github.com/apache/spark/pull/29056#issuecomment-656144321 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] GuoPhilipse opened a new pull request #29056: [SPARK-31753][SQL][DOCS]Add missing keywords

GuoPhilipse opened a new pull request #29056: URL: https://github.com/apache/spark/pull/29056 ### What changes were proposed in this pull request? update sql-ref docs ### Why are the changes needed? let more users know the sql key words usage ### Does this PR

[GitHub] [spark] cloud-fan commented on pull request #29055: [SPARK-32251][SQL][DOC] fix SQL keyword document

cloud-fan commented on pull request #29055: URL: https://github.com/apache/spark/pull/29055#issuecomment-656141772 cc @maropu @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

AmplabJenkins removed a comment on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-656141200 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan opened a new pull request #29055: [SPARK-32251][SQL][DOC] fix SQL keyword document

cloud-fan opened a new pull request #29055: URL: https://github.com/apache/spark/pull/29055 ### What changes were proposed in this pull request? This PR improves the test to make sure all the SQL keywords are documented correctly. It fixes several issues: 1. some keywords

[GitHub] [spark] AmplabJenkins commented on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

AmplabJenkins commented on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-656141200 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

SparkQA commented on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-656140374 **[Test build #125479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125479/testReport)** for PR 28971 at commit

[GitHub] [spark] srowen commented on pull request #28971: [WIP][SPARK-29292][SPARK-30010][CORE] Let core compile for Scala 2.13

srowen commented on pull request #28971: URL: https://github.com/apache/spark/pull/28971#issuecomment-656137859 Jenkins retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-656136676 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-656136676 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-656136201 **[Test build #125478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125478/testReport)** for PR 27690 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting

AmplabJenkins removed a comment on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-656132649 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

maropu commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-656132974 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

AmplabJenkins commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-656132649 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

SparkQA commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-656131960 **[Test build #125477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125477/testReport)** for PR 28874 at commit

[GitHub] [spark] srowen commented on pull request #28850: [SPARK-32015][Core]Remote inheritable thread local variables after spark context is stopped

srowen commented on pull request #28850: URL: https://github.com/apache/spark/pull/28850#issuecomment-656131143 Does this help actually clear the reference though? This is an automated message from the Apache Git Service. To

[GitHub] [spark] srowen commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

srowen commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-656129769 Jenkins test this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r452217417 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLog.scala ## @@ -97,18 +97,13 @@ class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28904: [SPARK-30462][SS] Streamline the logic on file stream source and sink metadata log to avoid memory issue

HeartSaVioR commented on a change in pull request #28904: URL: https://github.com/apache/spark/pull/28904#discussion_r452217417 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLog.scala ## @@ -97,18 +97,13 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [SPARK-31831][SQL] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656124585 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

AmplabJenkins removed a comment on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-656124662 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [SPARK-31831][SQL] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656124585 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

AmplabJenkins commented on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-656124662 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

SparkQA commented on pull request #28979: URL: https://github.com/apache/spark/pull/28979#issuecomment-656123915 **[Test build #125476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125476/testReport)** for PR 28979 at commit

[GitHub] [spark] SparkQA commented on pull request #29039: [SPARK-31831][SQL] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656123851 **[Test build #125475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125475/testReport)** for PR 29039 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

Ngone51 commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452210183 ## File path: sql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala ## @@ -504,23 +511,92 @@ class UDFSuite extends QueryTest with

[GitHub] [spark] Ngone51 commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

Ngone51 commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452208986 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -4521,9 +4522,10 @@ object functions { * @since 1.3.0 */ def

[GitHub] [spark] AmplabJenkins commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

AmplabJenkins commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-656120178 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

AmplabJenkins removed a comment on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-656120178 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

SparkQA commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-656119749 **[Test build #125474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125474/testReport)** for PR 29054 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #29039: [SPARK-31831][SQL] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

HeartSaVioR commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656119403 cc. @dongjoon-hyun @alismess-db This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AngersZhuuuu commented on pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

AngersZh commented on pull request #29054: URL: https://github.com/apache/spark/pull/29054#issuecomment-656117807 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Ngone51 commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

Ngone51 commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452188555 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -4521,9 +4522,10 @@ object functions { * @since 1.3.0 */ def

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [SPARK-31831] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656116457 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu opened a new pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

AngersZh opened a new pull request #29054: URL: https://github.com/apache/spark/pull/29054 ### What changes were proposed in this pull request? When we create a UDAF function use class extended `UserDefinedAggregeteFunction`,  when we call the function,  in support hive mode, in

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29053: [SPARK-32241][SQL] Remove empty children of union

AmplabJenkins removed a comment on pull request #29053: URL: https://github.com/apache/spark/pull/29053#issuecomment-656112505 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [SPARK-31831] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656116457 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29053: [SPARK-32241][SQL] Remove empty children of union

SparkQA commented on pull request #29053: URL: https://github.com/apache/spark/pull/29053#issuecomment-656115749 **[Test build #125472 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125472/testReport)** for PR 29053 at commit

[GitHub] [spark] SparkQA commented on pull request #29039: [SPARK-31831] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656115713 **[Test build #125473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125473/testReport)** for PR 29039 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

AmplabJenkins removed a comment on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656112514 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] ulysses-you commented on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

ulysses-you commented on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-656113225 It's ok to add an attribute `isCached` but it is not enough. 2 reasons: 1. A database can contains hundreds tables, `spark.sql("show tables").where("isCached=

[GitHub] [spark] HeartSaVioR commented on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

HeartSaVioR commented on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656112823 cc. @cloud-fan @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

AmplabJenkins removed a comment on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656112507 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

AmplabJenkins commented on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656112507 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29053: [SPARK-32241][SQL] Remove empty children of union

AmplabJenkins commented on pull request #29053: URL: https://github.com/apache/spark/pull/29053#issuecomment-656112505 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29036: [SPARK-32242][SQL] CliSuite flakiness fix via differentiating cli driver bootup timeout and query execution timeout

SparkQA commented on pull request #29036: URL: https://github.com/apache/spark/pull/29036#issuecomment-656111930 **[Test build #125471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125471/testReport)** for PR 29036 at commit

[GitHub] [spark] peter-toth opened a new pull request #29053: [SPARK-32241][SQL] Remove empty children of union

peter-toth opened a new pull request #29053: URL: https://github.com/apache/spark/pull/29053 ### What changes were proposed in this pull request? This PR removes the empty child relations of a `Union`. E.g. the query `SELECT c FROM t UNION ALL SELECT c FROM t WHERE FALSE` has the

[GitHub] [spark] maropu commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

maropu commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452192794 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -4521,9 +4522,10 @@ object functions { * @since 1.3.0 */ def

[GitHub] [spark] erikerlandson commented on a change in pull request #28983: [SPARK-32159][SQL] Fix integration between Aggregator[Array[_], _, _] and UnresolvedMapObjects

erikerlandson commented on a change in pull request #28983: URL: https://github.com/apache/spark/pull/28983#discussion_r452189220 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/udaf.scala ## @@ -517,3 +519,18 @@ case class ScalaAggregator[IN,

[GitHub] [spark] Ngone51 commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

Ngone51 commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452188555 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -4521,9 +4522,10 @@ object functions { * @since 1.3.0 */ def

[GitHub] [spark] HeartSaVioR commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

HeartSaVioR commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-656102306 Another summary for next set of builds > 125405 (https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125405/testReport/) All suites passed

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656098962 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #27988: [SPARK-31226][CORE][TEST] SizeBasedCoalesce logic will lose partition

cloud-fan commented on a change in pull request #27988: URL: https://github.com/apache/spark/pull/27988#discussion_r452181258 ## File path: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala ## @@ -1298,18 +1298,18 @@ class SizeBasedCoalescer(val maxSize: Int) extends

[GitHub] [spark] AmplabJenkins commented on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656098962 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] wankunde commented on pull request #28850: [SPARK-32015][Core]Remote inheritable thread local variables after spark context is stopped

wankunde commented on pull request #28850: URL: https://github.com/apache/spark/pull/28850#issuecomment-656098500 @Ngone51 Agree with you, and updated the code. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

SparkQA commented on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656098472 **[Test build #125469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125469/testReport)** for PR 29016 at commit

[GitHub] [spark] SparkQA commented on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

SparkQA commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656098521 **[Test build #125470 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125470/testReport)** for PR 28957 at commit

[GitHub] [spark] HyukjinKwon removed a comment on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

HyukjinKwon removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656003020 retest this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656096044 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28977: [WIP] Add all hive.execution suite in the parallel test group

AmplabJenkins removed a comment on pull request #28977: URL: https://github.com/apache/spark/pull/28977#issuecomment-656095735 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656096037 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

AmplabJenkins commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656096037 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

AmplabJenkins removed a comment on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656095656 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28977: [WIP] Add all hive.execution suite in the parallel test group

AmplabJenkins commented on pull request #28977: URL: https://github.com/apache/spark/pull/28977#issuecomment-656095735 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

SparkQA removed a comment on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656004596 **[Test build #125457 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125457/testReport)** for PR 28957 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

AmplabJenkins commented on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656095656 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] sarutak commented on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

sarutak commented on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656094949 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #28977: [WIP] Add all hive.execution suite in the parallel test group

SparkQA commented on pull request #28977: URL: https://github.com/apache/spark/pull/28977#issuecomment-656095044 **[Test build #125468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125468/testReport)** for PR 28977 at commit

[GitHub] [spark] SparkQA commented on pull request #28957: [WIP][SPARK-32138] Drop Python 2.7, 3.4 and 3.5

SparkQA commented on pull request #28957: URL: https://github.com/apache/spark/pull/28957#issuecomment-656095215 **[Test build #125457 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125457/testReport)** for PR 28957 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

AmplabJenkins removed a comment on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656093310 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

AmplabJenkins removed a comment on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656093300 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

AmplabJenkins commented on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656093300 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #28977: [WIP] Add all hive.execution suite in the parallel test group

HyukjinKwon commented on pull request #28977: URL: https://github.com/apache/spark/pull/28977#issuecomment-656092718 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

SparkQA commented on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656092721 **[Test build #125456 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125456/testReport)** for PR 29016 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29016: [SPARK-32200][WEBUI] Redirect to the history page when accessed to /history on the HistoryServer without appliation id

SparkQA removed a comment on pull request #29016: URL: https://github.com/apache/spark/pull/29016#issuecomment-656004530 **[Test build #125456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125456/testReport)** for PR 29016 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

cloud-fan commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452171846 ## File path: sql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala ## @@ -504,23 +511,92 @@ class UDFSuite extends QueryTest with

[GitHub] [spark] cloud-fan commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

cloud-fan commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452171570 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -4521,9 +4522,10 @@ object functions { * @since 1.3.0 */

[GitHub] [spark] cloud-fan commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

cloud-fan commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452171203 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala ## @@ -102,6 +105,25 @@ case class ScalaUDF( }

[GitHub] [spark] cloud-fan commented on a change in pull request #28979: [SPARK-32154][SQL] Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

cloud-fan commented on a change in pull request #28979: URL: https://github.com/apache/spark/pull/28979#discussion_r452171081 ## File path: sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala ## @@ -173,15 +174,11 @@ class UDFRegistration private[sql]

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29051: [WIP][SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

AmplabJenkins removed a comment on pull request #29051: URL: https://github.com/apache/spark/pull/29051#issuecomment-656085904 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29051: [WIP][SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

SparkQA removed a comment on pull request #29051: URL: https://github.com/apache/spark/pull/29051#issuecomment-655956165 **[Test build #125446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125446/testReport)** for PR 29051 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29051: [WIP][SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

AmplabJenkins removed a comment on pull request #29051: URL: https://github.com/apache/spark/pull/29051#issuecomment-656085897 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29051: [WIP][SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

AmplabJenkins commented on pull request #29051: URL: https://github.com/apache/spark/pull/29051#issuecomment-656085897 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29051: [WIP][SPARK-30276][SQL] Support Filter expression allows simultaneous use of DISTINCT

SparkQA commented on pull request #29051: URL: https://github.com/apache/spark/pull/29051#issuecomment-656085567 **[Test build #125446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125446/testReport)** for PR 29051 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28924: [SPARK-32091][CORE] Ignore timeout error when remove blocks on the lost executor

cloud-fan commented on a change in pull request #28924: URL: https://github.com/apache/spark/pull/28924#discussion_r452164478 ## File path: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala ## @@ -177,6 +181,95 @@ class BlockManagerSuite extends

<    4   5   6   7   8   9   10   11   12   13   >