[GitHub] [spark] HyukjinKwon commented on pull request #28762: [SPARK-31849][PYTHON][SQL][FOLLOW-UP] More correct error message in Python UDF exception message

HyukjinKwon commented on pull request #28762: URL: https://github.com/apache/spark/pull/28762#issuecomment-640972735 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] srowen commented on pull request #28763: [SPARK-28199][SS][FOLLOWUP] Mention the change of SPARK-28199 into the SS migration guide

srowen commented on pull request #28763: URL: https://github.com/apache/spark/pull/28763#issuecomment-641293765 That seems fine, but we're not going to roll another RC for this anyway This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

AmplabJenkins commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641067215 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28742: [SPARK-31921][CORE] Fix the wrong warning: "App app-xxx requires more resource than any of Workers could have"

SparkQA removed a comment on pull request #28742: URL: https://github.com/apache/spark/pull/28742#issuecomment-640763282 **[Test build #123643 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123643/testReport)** for PR 28742 at commit

[GitHub] [spark] asfgit closed pull request #28759: [SPARK-31934][BUILD] Remove set -x from docker image tool

asfgit closed pull request #28759: URL: https://github.com/apache/spark/pull/28759 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] huaxingao commented on a change in pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

huaxingao commented on a change in pull request #28710: URL: https://github.com/apache/spark/pull/28710#discussion_r436790826 ## File path: project/MimaExcludes.scala ## @@ -49,7 +49,34 @@ object MimaExcludes { //[SPARK-31840] Add instance weight support in

[GitHub] [spark] stczwd commented on pull request #28617: [SPARK-31694][SQL][WIP] Add SupportsPartitions Catalog APIs on DataSourceV2

stczwd commented on pull request #28617: URL: https://github.com/apache/spark/pull/28617#issuecomment-640961429 cc @cloud-fan @rdblue @xuanyuanking @dongjoon-hyun This is an automated message from the Apache Git Service. To

[GitHub] [spark] bart-samwel commented on pull request #28703: SPARK-29897 Add implicit cast for SubtractTimestamps

bart-samwel commented on pull request #28703: URL: https://github.com/apache/spark/pull/28703#issuecomment-641112800 That seems right. In theory we could allow `"string" - timestamp`, but that seems uncommon so I don't think it's worth investing time in. So let's close this.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

AmplabJenkins removed a comment on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641067170 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] Fokko opened a new pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

Fokko opened a new pull request #28754: URL: https://github.com/apache/spark/pull/28754 This allows us to make an average out of DateType. Under the hood, we take an average of the days since epoch, and convert it to date again. This requires the date object to be cast to a double

[GitHub] [spark] zsxwing closed pull request #28758: [SPARK-31923][Core]Ignore internal accumulators that use unrecognized types rather than crashing (branch-2.4)

zsxwing closed pull request #28758: URL: https://github.com/apache/spark/pull/28758 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #28763: [SPARK-28199][SS][FOLLOWUP] Mention the change of SPARK-28199 into the SS migration guide

SparkQA commented on pull request #28763: URL: https://github.com/apache/spark/pull/28763#issuecomment-641003136 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] baohe-zhang commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641024447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #27983: [SPARK-31936][SQL]Implement ScriptTransformation in sql/core

AngersZh edited a comment on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-641321553 > why do we need to keep `HiveScriptTransformationExec`? What's the behavior difference between the native and hive script transformation? Implement

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r436796185 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1197,6 +1213,71 @@ private[history] class

[GitHub] [spark] juliuszsompolski commented on a change in pull request #28751: [SPARK-31926][SQL][test-hive1.2] Fix concurrency issue for ThriftCLIService to getPortNumber

juliuszsompolski commented on a change in pull request #28751: URL: https://github.com/apache/spark/pull/28751#discussion_r437262570 ## File path: sql/hive-thriftserver/v1.2/src/main/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java ## @@ -45,13 +46,13 @@

[GitHub] [spark] gengliangwang commented on pull request #28760: [PARK-31935][SQL]Data source options should be propagated in method `checkAndGlobPathIfNecessary`

gengliangwang commented on pull request #28760: URL: https://github.com/apache/spark/pull/28760#issuecomment-640937897 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28751: [SPARK-31926][SQL] Fix concurrency issue for ThriftCLIService to getPortNumber

SparkQA commented on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-640557074 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] gengliangwang opened a new pull request #28760: [PARK-31935][SQL]Data source options should be propagated in method `checkAndGlobPathIfNecessary`

gengliangwang opened a new pull request #28760: URL: https://github.com/apache/spark/pull/28760 ### What changes were proposed in this pull request? Data source options should be propagated into the Hadoop configuration of method `checkAndGlobPathIfNecessary` From

[GitHub] [spark] HyukjinKwon closed pull request #28762: [SPARK-31849][PYTHON][SQL][FOLLOW-UP] More correct error message in Python UDF exception message

HyukjinKwon closed pull request #28762: URL: https://github.com/apache/spark/pull/28762 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #27986: [SPARK-31220][SQL] repartition obeys initialPartitionNum when adaptiveExecutionEnabled

SparkQA commented on pull request #27986: URL: https://github.com/apache/spark/pull/27986#issuecomment-640592066 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28752: [SPARK-31642][FOLLOWUP] Fix Sorting for duration column and make Status column sortable

AmplabJenkins removed a comment on pull request #28752: URL: https://github.com/apache/spark/pull/28752#issuecomment-640590456 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-640955046 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Ngone51 commented on a change in pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

Ngone51 commented on a change in pull request #28746: URL: https://github.com/apache/spark/pull/28746#discussion_r437100175 ## File path: core/src/main/scala/org/apache/spark/deploy/LocalSparkCluster.scala ## @@ -74,6 +74,10 @@ class LocalSparkCluster( def stop(): Unit =

[GitHub] [spark] SparkQA commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

SparkQA commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-640924005 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] viirya commented on a change in pull request #28750: [SPARK-31916][SQL] StringConcat can lead to StringIndexOutOfBoundsException

viirya commented on a change in pull request #28750: URL: https://github.com/apache/spark/pull/28750#discussion_r437158187 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -29,6 +29,7 @@ import

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r437129696 ## File path: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java ## @@ -47,55 +47,48 @@ private int

[GitHub] [spark] Ngone51 commented on a change in pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

Ngone51 commented on a change in pull request #28645: URL: https://github.com/apache/spark/pull/28645#discussion_r437108708 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -2847,6 +2848,36 @@ class Analyzer( } }

[GitHub] [spark] maropu commented on a change in pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

maropu commented on a change in pull request #28761: URL: https://github.com/apache/spark/pull/28761#discussion_r437279326 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFiltersBase.scala ## @@ -37,12 +40,44 @@ trait OrcFiltersBase {

[GitHub] [spark] maropu commented on a change in pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

maropu commented on a change in pull request #28764: URL: https://github.com/apache/spark/pull/28764#discussion_r437169927 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/numeric.sql.out ## @@ -4423,6 +4423,176 @@ struct<> +-- !query +SELECT

[GitHub] [spark] SaurabhChawla100 opened a new pull request #28768: [SPARK-31941][CORE]Add the code change to handle the SparkException

SaurabhChawla100 opened a new pull request #28768: URL: https://github.com/apache/spark/pull/28768 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change?

[GitHub] [spark] maryannxue commented on pull request #28669: [SPARK-31864][SQL] Adjust AQE skew join trigger condition

maryannxue commented on pull request #28669: URL: https://github.com/apache/spark/pull/28669#issuecomment-640703977 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] viirya opened a new pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

viirya opened a new pull request #28761: URL: https://github.com/apache/spark/pull/28761 ### What changes were proposed in this pull request? We added nested column predicate pushdown for Parquet in #27728. This patch extends the feature support to ORC. ### Why

[GitHub] [spark] cloud-fan commented on pull request #28763: [SPARK-28199][SS][FOLLOWUP] Mention the change of SPARK-28199 into the SS migration guide

cloud-fan commented on pull request #28763: URL: https://github.com/apache/spark/pull/28763#issuecomment-641026958 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] github-actions[bot] closed pull request #27683: [SPARK-30917][SQL]: The behaviour of UnaryMinus should not depend on SQLConf.get

github-actions[bot] closed pull request #27683: URL: https://github.com/apache/spark/pull/27683 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28765: [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map should be case insensitive.

AmplabJenkins removed a comment on pull request #28765: URL: https://github.com/apache/spark/pull/28765#issuecomment-641092368 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

SparkQA commented on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-640584004 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] tgravescs commented on a change in pull request #27598: [SPARK-30845] Do not upload local pyspark archives for spark-submit on Yarn

tgravescs commented on a change in pull request #27598: URL: https://github.com/apache/spark/pull/27598#discussion_r437438931 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ## @@ -635,7 +635,12 @@ private[spark] class Client(

[GitHub] [spark] dilipbiswal commented on a change in pull request #28750: [SPARK-31916][SQL] StringConcat can overflow , leads to StringIndexOutOfBoundsException

dilipbiswal commented on a change in pull request #28750: URL: https://github.com/apache/spark/pull/28750#discussion_r436808519 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringUtils.scala ## @@ -122,8 +122,8 @@ object StringUtils extends

[GitHub] [spark] HeartSaVioR commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641028299 @baohe-zhang Just FYI, probably you may want to mention me - looks like you've been mentioned other folk.

[GitHub] [spark] AmplabJenkins commented on pull request #28766: [SPARK-31939][SQL] Fix Parsing day of year when year field pattern is missing

AmplabJenkins commented on pull request #28766: URL: https://github.com/apache/spark/pull/28766#issuecomment-641146441 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28767: [SPARK-31940][SQL][DOCS] Document the default JVM time zone in to/fromJavaDate and legacy date formatters

SparkQA commented on pull request #28767: URL: https://github.com/apache/spark/pull/28767#issuecomment-641161869 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA removed a comment on pull request #28763: [SPARK-28199][SS][FOLLOWUP] Mention the change of SPARK-28199 into the SS migration guide

SparkQA removed a comment on pull request #28763: URL: https://github.com/apache/spark/pull/28763#issuecomment-641003136 **[Test build #123665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123665/testReport)** for PR 28763 at commit

[GitHub] [spark] cloud-fan commented on pull request #28669: [SPARK-31864][SQL] Adjust AQE skew join trigger condition

cloud-fan commented on pull request #28669: URL: https://github.com/apache/spark/pull/28669#issuecomment-640574818 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] SparkQA removed a comment on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

SparkQA removed a comment on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641206321 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28767: [SPARK-31940][SQL][DOCS] Document the default JVM time zone in to/fromJavaDate and legacy date formatters

SparkQA removed a comment on pull request #28767: URL: https://github.com/apache/spark/pull/28767#issuecomment-641161869 **[Test build #123684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123684/testReport)** for PR 28767 at commit

[GitHub] [spark] maropu commented on pull request #28703: SPARK-29897 Add implicit cast for SubtractTimestamps

maropu commented on pull request #28703: URL: https://github.com/apache/spark/pull/28703#issuecomment-640951706 (Probably, not `TimeSub` but `TimeAdd` ;) https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L281)

[GitHub] [spark] AngersZhuuuu commented on pull request #27983: [SPARK-15694][SQL]Implement ScriptTransformation in sql/core

AngersZh commented on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-640977413 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

SparkQA removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-640924005 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #28726: [SPARK-31906][SQL][DOCS] Enhance comments in NamedExpression.qualifier

cloud-fan commented on pull request #28726: URL: https://github.com/apache/spark/pull/28726#issuecomment-641316304 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors

AmplabJenkins removed a comment on pull request #28619: URL: https://github.com/apache/spark/pull/28619#issuecomment-641023126 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

HyukjinKwon commented on a change in pull request #28754: URL: https://github.com/apache/spark/pull/28754#discussion_r437128608 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Average.scala ## @@ -40,10 +40,17 @@ case class

[GitHub] [spark] SparkQA removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-640763276 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28751: [SPARK-31926][SQL] Fix concurrency issue for ThriftCLIService to getPortNumber

AmplabJenkins commented on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-640557597 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

SparkQA commented on pull request #28258: URL: https://github.com/apache/spark/pull/28258#issuecomment-640632086 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #28760: [PARK-31935][SQL]Data source options should be propagated in method `checkAndGlobPathIfNecessary`

AmplabJenkins commented on pull request #28760: URL: https://github.com/apache/spark/pull/28760#issuecomment-640938193 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #28751: [SPARK-31926][SQL][test-hive1.2] Fix concurrency issue for ThriftCLIService to getPortNumber

maropu commented on a change in pull request #28751: URL: https://github.com/apache/spark/pull/28751#discussion_r437361561 ## File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SharedThriftServer.scala ## @@ -53,11 +55,21 @@ trait

[GitHub] [spark] SparkQA commented on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

SparkQA commented on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641066479 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

SparkQA commented on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641206321 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA removed a comment on pull request #28757: [SPARK-31932][SQL][TESTS] Add date/timestamp benchmarks for `HiveResult.hiveResultString()`

SparkQA removed a comment on pull request #28757: URL: https://github.com/apache/spark/pull/28757#issuecomment-640842850 **[Test build #123646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123646/testReport)** for PR 28757 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-640797313 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] srowen commented on a change in pull request #28750: [SPARK-31916][SQL] StringConcat can overflow , leads to StringIndexOutOfBoundsException

srowen commented on a change in pull request #28750: URL: https://github.com/apache/spark/pull/28750#discussion_r436668197 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/StringUtils.scala ## @@ -122,8 +122,8 @@ object StringUtils extends Logging

[GitHub] [spark] fornaix opened a new pull request #28755: [SPARK-29394][SQL]Support ISO 8601 format for intervals

fornaix opened a new pull request #28755: URL: https://github.com/apache/spark/pull/28755 ### What changes were proposed in this pull request? This pr is to support ISO 8601 format for intervals. - P[n]Y[n]M[n]DT[n]H[n]M[n]S - P[n]W - P[]-[MM]-[DD]T[hh]:[mm]:[ss]

[GitHub] [spark] zhli1142015 opened a new pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 opened a new pull request #28769: URL: https://github.com/apache/spark/pull/28769 ### What changes were proposed in this pull request? Close LevelDBIterator when LevelDB.close() is called. ### Why are the changes needed? This pull request would

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28391: [SPARK-31593][SS] Remove unnecessary streaming query progress update

HeartSaVioR commented on a change in pull request #28391: URL: https://github.com/apache/spark/pull/28391#discussion_r437051581 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingDeduplicationSuite.scala ## @@ -270,7 +271,12 @@ class

[GitHub] [spark] attilapiros edited a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

attilapiros edited a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641294356 Hi @holdenk! I have opened a new PR for your branch: https://github.com/holdenk/spark/pull/6 I know this seems to be a lot of change but it is really

[GitHub] [spark] viirya commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

viirya commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-640955747 @srowen @huaxingao any more comments for this change that I need to address? This is an automated message from

[GitHub] [spark] SparkQA removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-640524296 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28758: [SPARK-31923][Core]Ignore internal accumulators that use unrecognized types rather than crashing (branch-2.4)

SparkQA commented on pull request #28758: URL: https://github.com/apache/spark/pull/28758#issuecomment-640875286 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA removed a comment on pull request #28756: [SPARK-8981][CORE][FOLLOW-UP] Clean up MDC properties after running a task

SparkQA removed a comment on pull request #28756: URL: https://github.com/apache/spark/pull/28756#issuecomment-640700418 **[Test build #123639 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123639/testReport)** for PR 28756 at commit

[GitHub] [spark] SparkQA commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-640955046 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] tgravescs commented on pull request #27598: [SPARK-30845] Do not upload local pyspark archives for spark-submit on Yarn

tgravescs commented on pull request #27598: URL: https://github.com/apache/spark/pull/27598#issuecomment-640882029 thanks @shanyu merged to master This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HeartSaVioR commented on pull request #24996: [SPARK-28199][SS] Move Trigger implementations to Triggers.scala and avoid exposing these to the end users

HeartSaVioR commented on pull request #24996: URL: https://github.com/apache/spark/pull/24996#issuecomment-640921998 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] sarutak edited a comment on pull request #28585: [SPARK-31765][WEBUI][test-maven] Upgrade HtmlUnit >= 2.37.0

sarutak edited a comment on pull request #28585: URL: https://github.com/apache/spark/pull/28585#issuecomment-640153448 The errors were reproduced in PR builder and they are fixed by excluding the old `bytebuddy` so I believe it's safe to merge. What do you think, @dongjoon-hyun,

[GitHub] [spark] SparkQA removed a comment on pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors

SparkQA removed a comment on pull request #28619: URL: https://github.com/apache/spark/pull/28619#issuecomment-641022846 **[Test build #123668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123668/testReport)** for PR 28619 at commit

[GitHub] [spark] gengliangwang commented on a change in pull request #28760: [SPARK-31935][SQL]Data source options should be propagated in method `checkAndGlobPathIfNecessary`

gengliangwang commented on a change in pull request #28760: URL: https://github.com/apache/spark/pull/28760#discussion_r437060140 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ## @@ -570,7 +570,7 @@ case class DataSource(

[GitHub] [spark] sarutak commented on a change in pull request #28748: [WIP][SPARK-30119][WEBUI]Support pagination for streaming tab

sarutak commented on a change in pull request #28748: URL: https://github.com/apache/spark/pull/28748#discussion_r436694872 ## File path: streaming/src/test/scala/org/apache/spark/streaming/UISeleniumSuite.scala ## @@ -125,24 +125,37 @@ class UISeleniumSuite //

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-640556879 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] therealJacobWu commented on a change in pull request #28731: [SPARK-31909][SQL] Add SPARK_SUBMIT_OPTS to Beeline Script

therealJacobWu commented on a change in pull request #28731: URL: https://github.com/apache/spark/pull/28731#discussion_r436981238 ## File path: bin/beeline ## @@ -28,5 +28,7 @@ if [ -z "${SPARK_HOME}" ]; then source "$(dirname "$0")"/find-spark-home fi +.

[GitHub] [spark] maropu commented on pull request #28755: [SPARK-29394][SQL]Support ISO 8601 format for intervals

maropu commented on pull request #28755: URL: https://github.com/apache/spark/pull/28755#issuecomment-640949585 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] HeartSaVioR commented on pull request #28391: [SPARK-31593][SS] Remove unnecessary streaming query progress update

HeartSaVioR commented on pull request #28391: URL: https://github.com/apache/spark/pull/28391#issuecomment-640935752 cc. @tdas @zsxwing @jose-torres This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA removed a comment on pull request #28752: [SPARK-31642][FOLLOWUP] Fix Sorting for duration column and make Status column sortable

SparkQA removed a comment on pull request #28752: URL: https://github.com/apache/spark/pull/28752#issuecomment-640462659 **[Test build #123624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123624/testReport)** for PR 28752 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

AmplabJenkins commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-641313089 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] holdenk commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-640736229 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] viirya commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

viirya commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-641073844 cc @dbtsai @dongjoon-hyun @cloud-fan @maropu This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

SparkQA removed a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641066478 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon opened a new pull request #28762: [SPARK-31849][PYTHON][SQL][FOLLOW-UP] More correct error message in Python UDF exception message

HyukjinKwon opened a new pull request #28762: URL: https://github.com/apache/spark/pull/28762 ### What changes were proposed in this pull request? This PR proposes to fix wordings in the Python UDF exception error message from: From: An exception was thrown from Python

[GitHub] [spark] AmplabJenkins commented on pull request #28767: [SPARK-31940][SQL][DOCS] Document the default JVM time zone in to/fromJavaDate and legacy date formatters

AmplabJenkins commented on pull request #28767: URL: https://github.com/apache/spark/pull/28767#issuecomment-641163291 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27986: [SPARK-31220][SQL] repartition obeys initialPartitionNum when adaptiveExecutionEnabled

AmplabJenkins removed a comment on pull request #27986: URL: https://github.com/apache/spark/pull/27986#issuecomment-640592619 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28737: [SPARK-31913][SQL]: Fix StackOverflowError in FileScanRDD

AmplabJenkins removed a comment on pull request #28737: URL: https://github.com/apache/spark/pull/28737#issuecomment-640689295 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

maropu commented on pull request #18323: URL: https://github.com/apache/spark/pull/18323#issuecomment-640955951 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

AmplabJenkins removed a comment on pull request #27518: URL: https://github.com/apache/spark/pull/27518#issuecomment-640643335 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-640763276 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] yaooqinn commented on a change in pull request #28751: [SPARK-31926][SQL] Fix concurrency issue for ThriftCLIService to getPortNumber

yaooqinn commented on a change in pull request #28751: URL: https://github.com/apache/spark/pull/28751#discussion_r436649612 ## File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SharedThriftServer.scala ## @@ -74,18 +86,31 @@ trait

[GitHub] [spark] cloud-fan commented on a change in pull request #27986: [SPARK-31220][SQL] repartition obeys initialPartitionNum when adaptiveExecutionEnabled

cloud-fan commented on a change in pull request #27986: URL: https://github.com/apache/spark/pull/27986#discussion_r436656608 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -1021,4 +1021,20 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28751: [SPARK-31926][SQL] Fix concurrency issue for ThriftCLIService to getPortNumber

AmplabJenkins removed a comment on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-640557597 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28752: [SPARK-31642][FOLLOWUP] Fix Sorting for duration column and make Status column sortable

AmplabJenkins commented on pull request #28752: URL: https://github.com/apache/spark/pull/28752#issuecomment-640590456 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-640955464 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

maropu commented on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641202844 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

SparkQA removed a comment on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-640584004 This is an automated message from the Apache Git Service. To respond to the message, please log on to

<    1   2   3   4   5   6   7   >