[GitHub] [spark] AmplabJenkins commented on pull request #28770: [SPARK-31942] Revert "[SPARK-31864][SQL] Adjust AQE skew join trigger condition

AmplabJenkins commented on pull request #28770: URL: https://github.com/apache/spark/pull/28770#issuecomment-641242421 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] jkpl opened a new pull request #28753: [SPARK-31927][K8S] Bump Fabric8 to 4.10.2 to support k8s 1.18

jkpl opened a new pull request #28753: URL: https://github.com/apache/spark/pull/28753 ### What changes were proposed in this pull request? Update the Fabric8 Kubernetes client to the latest release to support Kubernetes version 1.18 (and 1.17) in Spark 2.4.x. ### Why are the

[GitHub] [spark] cloud-fan commented on pull request #24996: [SPARK-28199][SS] Move Trigger implementations to Triggers.scala and avoid exposing these to the end users

cloud-fan commented on pull request #24996: URL: https://github.com/apache/spark/pull/24996#issuecomment-640756200 shall we have an item in the migration guide for it? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA removed a comment on pull request #28766: [SPARK-31939][SQL] Fix Parsing day of year when year field pattern is missing

SparkQA removed a comment on pull request #28766: URL: https://github.com/apache/spark/pull/28766#issuecomment-641145651 **[Test build #123683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123683/testReport)** for PR 28766 at commit [`2d797e0`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #28758: [SPARK-31923][Core]Ignore internal accumulators that use unrecognized types rather than crashing (branch-2.4)

SparkQA removed a comment on pull request #28758: URL: https://github.com/apache/spark/pull/28758#issuecomment-640875286 **[Test build #123648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123648/testReport)** for PR 28758 at commit [`2e64849`](https://gi

[GitHub] [spark] jkpl commented on pull request #28753: [SPARK-31927][K8S] Bump Fabric8 to 4.10.2 to support k8s 1.18

jkpl commented on pull request #28753: URL: https://github.com/apache/spark/pull/28753#issuecomment-640545204 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] AmplabJenkins commented on pull request #28616: [SPARK-31798][SHUFFLE][API] Shuffle Writer API changes to return custom map output metadata

AmplabJenkins commented on pull request #28616: URL: https://github.com/apache/spark/pull/28616#issuecomment-640957576 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28755: [SPARK-29394][SQL]Support ISO 8601 format for intervals

AmplabJenkins commented on pull request #28755: URL: https://github.com/apache/spark/pull/28755#issuecomment-640563320 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] mccheah commented on pull request #28616: [SPARK-31798][SHUFFLE][API] Shuffle Writer API changes to return custom map output metadata

mccheah commented on pull request #28616: URL: https://github.com/apache/spark/pull/28616#issuecomment-640955912 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #28765: [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map should be case insensitive.

AmplabJenkins commented on pull request #28765: URL: https://github.com/apache/spark/pull/28765#issuecomment-641092368 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28750: [SPARK-31916][SQL] StringConcat can overflow , leads to StringIndexOutOfBoundsException

AmplabJenkins removed a comment on pull request #28750: URL: https://github.com/apache/spark/pull/28750#issuecomment-640583011 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] lipzhu commented on a change in pull request #28726: [SPARK-31906][SQL][DOCS] Enhance comments in NamedExpression.qualifier

lipzhu commented on a change in pull request #28726: URL: https://github.com/apache/spark/pull/28726#discussion_r437111507 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala ## @@ -85,6 +85,7 @@ trait NamedExpression exte

[GitHub] [spark] maropu commented on a change in pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

maropu commented on a change in pull request #28768: URL: https://github.com/apache/spark/pull/28768#discussion_r437311656 ## File path: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ## @@ -86,6 +86,7 @@ private[spark] class SparkUI private ( .orElse(store.env

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28726: [SPARK-31906][SQL][DOCS] Enhance comments in NamedExpression.qualifier

HyukjinKwon commented on a change in pull request #28726: URL: https://github.com/apache/spark/pull/28726#discussion_r437124960 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala ## @@ -85,6 +85,7 @@ trait NamedExpression

[GitHub] [spark] SparkQA commented on pull request #28760: [PARK-31935][SQL]Data source options should be propagated in method `checkAndGlobPathIfNecessary`

SparkQA commented on pull request #28760: URL: https://github.com/apache/spark/pull/28760#issuecomment-640937812 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] gaborgsomogyi commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

gaborgsomogyi commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-640541944 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28737: [SPARK-31913][SQL]: Fix StackOverflowError in FileScanRDD

AmplabJenkins commented on pull request #28737: URL: https://github.com/apache/spark/pull/28737#issuecomment-640689295 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28737: [SPARK-31913][SQL]: Fix StackOverflowError in FileScanRDD

SparkQA commented on pull request #28737: URL: https://github.com/apache/spark/pull/28737#issuecomment-640688102 **[Test build #123630 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123630/testReport)** for PR 28737 at commit [`1e17fd0`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28036: [SPARK-26341][CORE]Expose executor memory metrics at the stage level, in the Stages tab

AmplabJenkins commented on pull request #28036: URL: https://github.com/apache/spark/pull/28036#issuecomment-641101187 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] cloud-fan commented on pull request #28757: [SPARK-31932][SQL][TESTS] Add date/timestamp benchmarks for `HiveResult.hiveResultString()`

cloud-fan commented on pull request #28757: URL: https://github.com/apache/spark/pull/28757#issuecomment-641028398 I'm merging it to 3.0 as well, so that it's easier to compare the perf between 3.0 and 3.1 in the future. thanks, merging to master/3.0! --

[GitHub] [spark] SparkQA commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

SparkQA commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-641312310 **[Test build #123699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123699/testReport)** for PR 27246 at commit [`e48a936`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

viirya commented on a change in pull request #28704: URL: https://github.com/apache/spark/pull/28704#discussion_r437150737 ## File path: mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala ## @@ -248,6 +248,19 @@ object MLUtils extends Logging { }.toArray }

[GitHub] [spark] SparkQA commented on pull request #28036: [SPARK-26341][CORE]Expose executor memory metrics at the stage level, in the Stages tab

SparkQA commented on pull request #28036: URL: https://github.com/apache/spark/pull/28036#issuecomment-641100165 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] liancheng commented on pull request #28760: [SPARK-31935][SQL] Hadoop file system config should be effective in data source options

liancheng commented on pull request #28760: URL: https://github.com/apache/spark/pull/28760#issuecomment-640995849 LGTM, thanks! This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] zsxwing commented on a change in pull request #28760: [SPARK-31935][SQL]Data source options should be propagated in method `checkAndGlobPathIfNecessary`

zsxwing commented on a change in pull request #28760: URL: https://github.com/apache/spark/pull/28760#discussion_r437057671 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala ## @@ -570,7 +570,7 @@ case class DataSource( ch

[GitHub] [spark] SparkQA commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

SparkQA commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641066478 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #28757: [SPARK-31932][SQL][TESTS] Add date/timestamp benchmarks for `toHiveString`

SparkQA commented on pull request #28757: URL: https://github.com/apache/spark/pull/28757#issuecomment-640842850 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] turboFei edited a comment on pull request #28765: [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map parsed from partition path should be case insensitive

turboFei edited a comment on pull request #28765: URL: https://github.com/apache/spark/pull/28765#issuecomment-641093704 cc @viirya @cloud-fan This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [spark] yaooqinn commented on pull request #28755: [SPARK-29394][SQL]Support ISO 8601 format for intervals

yaooqinn commented on pull request #28755: URL: https://github.com/apache/spark/pull/28755#issuecomment-641041893 We reverted ISO interval output before. I don't know whether it's a good idea to support the ISO or ANSI SQL Standard style since 3.1 or not. IMHO, intervals in the form of `va

[GitHub] [spark] SparkQA commented on pull request #28766: [SPARK-31939][SQL] Fix Parsing day of year when year field pattern is missing

SparkQA commented on pull request #28766: URL: https://github.com/apache/spark/pull/28766#issuecomment-641145651 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AngersZh commented on a change in pull request #28733: URL: https://github.com/apache/spark/pull/28733#discussion_r437277043 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala ## @@ -198,6 +199,88 @@ trait PredicateHelper {

[GitHub] [spark] maropu commented on a change in pull request #28726: [SPARK-31906][SQL][DOCS] Enhance comments in NamedExpression.qualifier

maropu commented on a change in pull request #28726: URL: https://github.com/apache/spark/pull/28726#discussion_r437178386 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala ## @@ -85,6 +85,7 @@ trait NamedExpression exte

[GitHub] [spark] AmplabJenkins commented on pull request #28745: [SPARK-31915][SQL][PYTHON] Remove projection that adds grouping keys in grouped and cogrouped pandas UDFs

AmplabJenkins commented on pull request #28745: URL: https://github.com/apache/spark/pull/28745#issuecomment-641290863 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HeartSaVioR commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

HeartSaVioR commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-641097576 I agree the new addition of the similar option feels tricky. Maybe you've already indicated there're some cases `maxFileAge` has to be ignored which means Spark is ne

[GitHub] [spark] SparkQA commented on pull request #28759: [SPARK-31934][BUILD] Remove set -x from docker image tool

SparkQA commented on pull request #28759: URL: https://github.com/apache/spark/pull/28759#issuecomment-640930710 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28766: [SPARK-31939][SQL] Fix Parsing day of year when year field pattern is missing

AmplabJenkins removed a comment on pull request #28766: URL: https://github.com/apache/spark/pull/28766#issuecomment-641146441 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] zhengruifeng commented on a change in pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

zhengruifeng commented on a change in pull request #28704: URL: https://github.com/apache/spark/pull/28704#discussion_r437107792 ## File path: mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala ## @@ -248,6 +248,19 @@ object MLUtils extends Logging { }.toArray

[GitHub] [spark] AmplabJenkins commented on pull request #28762: [SPARK-31849][PYTHON][SQL][FOLLOW-UP] More correct error message in Python UDF exception message

AmplabJenkins commented on pull request #28762: URL: https://github.com/apache/spark/pull/28762#issuecomment-640965214 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28755: [SPARK-29394][SQL]Support ISO 8601 format for intervals

SparkQA removed a comment on pull request #28755: URL: https://github.com/apache/spark/pull/28755#issuecomment-640950793 **[Test build #123658 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123658/testReport)** for PR 28755 at commit [`4bfac69`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28757: [SPARK-31932][SQL][TESTS] Add date/timestamp benchmarks for `HiveResult.hiveResultString()`

AmplabJenkins commented on pull request #28757: URL: https://github.com/apache/spark/pull/28757#issuecomment-640844245 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

AmplabJenkins commented on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641067170 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] cloud-fan commented on pull request #28734: [SPARK-31912][SQL][TESTS] Normalize all binary comparison expressions

cloud-fan commented on pull request #28734: URL: https://github.com/apache/spark/pull/28734#issuecomment-641319075 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437043628 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,205 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] baohe-zhang edited a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang edited a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641024447 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] sathyaprakashg commented on pull request #28703: SPARK-29897 Add implicit cast for SubtractTimestamps

sathyaprakashg commented on pull request #28703: URL: https://github.com/apache/spark/pull/28703#issuecomment-641075311 @bart-samwel @cloud-fan @maropu In case of subtract use case, If left expression is string and right expression is timestamp, then there is no ambiguity. Timestamp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

AmplabJenkins removed a comment on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-640584830 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on pull request #17953: [SPARK-20680][SQL] Spark-sql do not support for void column datatype …

maropu commented on pull request #17953: URL: https://github.com/apache/spark/pull/17953#issuecomment-641072690 @amit-hitachi I think we don't have any plan for this work. But, you could revive this discussion in the corresponding jira side. ---

[GitHub] [spark] cloud-fan commented on pull request #28765: [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map parsed from partition path should be case insensitive

cloud-fan commented on pull request #28765: URL: https://github.com/apache/spark/pull/28765#issuecomment-641156270 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors

AmplabJenkins commented on pull request #28619: URL: https://github.com/apache/spark/pull/28619#issuecomment-641023126 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28759: [SPARK-31934][BUILD] Remove set -x from docker image tool

SparkQA removed a comment on pull request #28759: URL: https://github.com/apache/spark/pull/28759#issuecomment-640930710 **[Test build #123651 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123651/testReport)** for PR 28759 at commit [`e300be5`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28758: [SPARK-31923][Core]Ignore internal accumulators that use unrecognized types rather than crashing (branch-2.4)

AmplabJenkins commented on pull request #28758: URL: https://github.com/apache/spark/pull/28758#issuecomment-640875960 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28756: [SPARK-8981][CORE][FOLLOW-UP] Clean up MDC properties after running a task

AmplabJenkins commented on pull request #28756: URL: https://github.com/apache/spark/pull/28756#issuecomment-640701041 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

AmplabJenkins commented on pull request #27518: URL: https://github.com/apache/spark/pull/27518#issuecomment-640643335 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] zsxwing commented on a change in pull request #28758: [SPARK-31923][Core]Ignore internal accumulators that use unrecognized types rather than crashing (branch-2.4)

zsxwing commented on a change in pull request #28758: URL: https://github.com/apache/spark/pull/28758#discussion_r436988149 ## File path: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala ## @@ -485,29 +483,6 @@ class JsonProtocolSuite extends SparkFunSuite {

[GitHub] [spark] maropu commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

maropu commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-641077009 retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] MaxGekk opened a new pull request #28757: [SPARK-31932][SQL][TESTS] Add date/timestamp benchmarks for `toHiveString`

MaxGekk opened a new pull request #28757: URL: https://github.com/apache/spark/pull/28757 ### What changes were proposed in this pull request? Add benchmarks for `HiveResult.hiveResultString()/toHiveString()` to measure throughput of `toHiveString` for the date/timestamp types: - java

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AmplabJenkins removed a comment on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-640550996 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] asfgit closed pull request #27598: [SPARK-30845] Do not upload local pyspark archives for spark-submit on Yarn

asfgit closed pull request #27598: URL: https://github.com/apache/spark/pull/27598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] HeartSaVioR commented on pull request #28763: [SPARK-28199][SS][FOLLOWUP] Mention the change of SPARK-28199 into the SS migration guide

HeartSaVioR commented on pull request #28763: URL: https://github.com/apache/spark/pull/28763#issuecomment-641002505 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-640955464 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

SparkQA removed a comment on pull request #27518: URL: https://github.com/apache/spark/pull/27518#issuecomment-640641799 **[Test build #123638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123638/testReport)** for PR 27518 at commit [`af55a08`](https://gi

[GitHub] [spark] cloud-fan commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

cloud-fan commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r437231685 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -56,6 +54,26 @@ trait TimeZoneAware

[GitHub] [spark] skambha closed pull request #27629: [SPARK-28067][SQL]Fix incorrect results during aggregate sum for decimal overflow by throwing exception

skambha closed pull request #27629: URL: https://github.com/apache/spark/pull/27629 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] iRakson commented on pull request #28748: [WIP][SPARK-30119][WEBUI]Support pagination for streaming tab

iRakson commented on pull request #28748: URL: https://github.com/apache/spark/pull/28748#issuecomment-640578773 cc @srowen I forgot to cc you This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] AmplabJenkins commented on pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

AmplabJenkins commented on pull request #28258: URL: https://github.com/apache/spark/pull/28258#issuecomment-640633493 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

SparkQA removed a comment on pull request #28258: URL: https://github.com/apache/spark/pull/28258#issuecomment-640632086 **[Test build #123637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123637/testReport)** for PR 28258 at commit [`0e152f4`](https://gi

[GitHub] [spark] turboFei opened a new pull request #28765: [SPARK-29295][FOLLOWUP] Dynamic partition map should be case insensitive.

turboFei opened a new pull request #28765: URL: https://github.com/apache/spark/pull/28765 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How wa

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-640697045 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SaurabhChawla100 commented on pull request #27636: [SPARK-30873][CORE][YARN]Handling Node Decommissioning for Yarn cluster manger in Spark

SaurabhChawla100 commented on pull request #27636: URL: https://github.com/apache/spark/pull/27636#issuecomment-641018544 > @tgravescs @SaurabhChawla100 .. I am wondering if you guys are still pushing for this PR to get into Spark master ? Are you also thinking of targeting to Spark 3.0 br

[GitHub] [spark] zsxwing commented on pull request #28744: [SPARK-31923][Core]Ignore internal accumulators that use unrecognized types rather than crashing

zsxwing commented on pull request #28744: URL: https://github.com/apache/spark/pull/28744#issuecomment-640687413 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-640797313 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #27983: [SPARK-31936][SQL]Implement ScriptTransformation in sql/core

maropu commented on a change in pull request #27983: URL: https://github.com/apache/spark/pull/27983#discussion_r437476009 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/script/ScriptTransformationExec.scala ## @@ -0,0 +1,239 @@ +/* + * Licensed to the Ap

[GitHub] [spark] SparkQA removed a comment on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

SparkQA removed a comment on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641073304 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] turboFei commented on pull request #28765: [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map should be case insensitive.

turboFei commented on pull request #28765: URL: https://github.com/apache/spark/pull/28765#issuecomment-641093704 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA removed a comment on pull request #28762: [SPARK-31849][PYTHON][SQL][FOLLOW-UP] More correct error message in Python UDF exception message

SparkQA removed a comment on pull request #28762: URL: https://github.com/apache/spark/pull/28762#issuecomment-640964944 **[Test build #123663 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123663/testReport)** for PR 28762 at commit [`9eb69c0`](https://gi

[GitHub] [spark] cloud-fan commented on pull request #25715: [SPARK-28969][PYTHON][ML] OneVsRestParams parity between scala and python

cloud-fan commented on pull request #25715: URL: https://github.com/apache/spark/pull/25715#issuecomment-640741243 I think breaking changes like this should add an item in the migration guide. This is an automated message fro

[GitHub] [spark] prakharjain09 commented on a change in pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors

prakharjain09 commented on a change in pull request #28619: URL: https://github.com/apache/spark/pull/28619#discussion_r437133100 ## File path: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ## @@ -1892,6 +1892,112 @@ class TaskSetManagerSuite te

[GitHub] [spark] maropu commented on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

maropu commented on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641202844 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

SparkQA removed a comment on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-640584004 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] SparkQA commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

SparkQA commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-640538971 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-640955464 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28752: [SPARK-31642][FOLLOWUP] Fix Sorting for duration column and make Status column sortable

AmplabJenkins commented on pull request #28752: URL: https://github.com/apache/spark/pull/28752#issuecomment-640590456 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28751: [SPARK-31926][SQL] Fix concurrency issue for ThriftCLIService to getPortNumber

AmplabJenkins removed a comment on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-640557597 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #27986: [SPARK-31220][SQL] repartition obeys initialPartitionNum when adaptiveExecutionEnabled

cloud-fan commented on a change in pull request #27986: URL: https://github.com/apache/spark/pull/27986#discussion_r436656608 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -1021,4 +1021,20 @@ class AdaptiveQuer

[GitHub] [spark] yaooqinn commented on a change in pull request #28751: [SPARK-31926][SQL] Fix concurrency issue for ThriftCLIService to getPortNumber

yaooqinn commented on a change in pull request #28751: URL: https://github.com/apache/spark/pull/28751#discussion_r436649612 ## File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/SharedThriftServer.scala ## @@ -74,18 +86,31 @@ trait SharedTh

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-640763276 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28737: [SPARK-31913][SQL]: Fix StackOverflowError in FileScanRDD

AmplabJenkins removed a comment on pull request #28737: URL: https://github.com/apache/spark/pull/28737#issuecomment-640689295 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27986: [SPARK-31220][SQL] repartition obeys initialPartitionNum when adaptiveExecutionEnabled

AmplabJenkins removed a comment on pull request #27986: URL: https://github.com/apache/spark/pull/27986#issuecomment-640592619 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on pull request #18323: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

maropu commented on pull request #18323: URL: https://github.com/apache/spark/pull/18323#issuecomment-640955951 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27518: [SPARK-30768][SQL] Constraints inferred from inequality attributes

AmplabJenkins removed a comment on pull request #27518: URL: https://github.com/apache/spark/pull/27518#issuecomment-640643335 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28767: [SPARK-31940][SQL][DOCS] Document the default JVM time zone in to/fromJavaDate and legacy date formatters

AmplabJenkins commented on pull request #28767: URL: https://github.com/apache/spark/pull/28767#issuecomment-641163291 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] viirya commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

viirya commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-641073844 cc @dbtsai @dongjoon-hyun @cloud-fan @maropu This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] HyukjinKwon opened a new pull request #28762: [SPARK-31849][PYTHON][SQL][FOLLOW-UP] More correct error message in Python UDF exception message

HyukjinKwon opened a new pull request #28762: URL: https://github.com/apache/spark/pull/28762 ### What changes were proposed in this pull request? This PR proposes to fix wordings in the Python UDF exception error message from: From: An exception was thrown from Python

[GitHub] [spark] SparkQA removed a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

SparkQA removed a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641066478 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] HeartSaVioR commented on pull request #28391: [SPARK-31593][SS] Remove unnecessary streaming query progress update

HeartSaVioR commented on pull request #28391: URL: https://github.com/apache/spark/pull/28391#issuecomment-640935752 cc. @tdas @zsxwing @jose-torres This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] maropu commented on pull request #28755: [SPARK-29394][SQL]Support ISO 8601 format for intervals

maropu commented on pull request #28755: URL: https://github.com/apache/spark/pull/28755#issuecomment-640949585 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA removed a comment on pull request #28752: [SPARK-31642][FOLLOWUP] Fix Sorting for duration column and make Status column sortable

SparkQA removed a comment on pull request #28752: URL: https://github.com/apache/spark/pull/28752#issuecomment-640462659 **[Test build #123624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123624/testReport)** for PR 28752 at commit [`acd9d54`](https://gi

[GitHub] [spark] sarutak commented on a change in pull request #28748: [WIP][SPARK-30119][WEBUI]Support pagination for streaming tab

sarutak commented on a change in pull request #28748: URL: https://github.com/apache/spark/pull/28748#discussion_r436694872 ## File path: streaming/src/test/scala/org/apache/spark/streaming/UISeleniumSuite.scala ## @@ -125,24 +125,37 @@ class UISeleniumSuite // Chec

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-640556879 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

AmplabJenkins commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-641313089 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

  1   2   3   4   5   6   7   8   >