[GitHub] [spark] shaneknapp closed pull request #28666: [BUILD][INFRA] bump the timeout to match the jenkins PRB

shaneknapp closed pull request #28666: URL: https://github.com/apache/spark/pull/28666 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28666: [BUILD][INFRA] bump the timeout to match the jenkins PRB

AmplabJenkins removed a comment on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635616372 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28666: [BUILD][INFRA] bump the timeout to match the jenkins PRB

AmplabJenkins commented on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635616372 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28666: [BUILD][INFRA] bump the timeout to match the jenkins PRB

SparkQA removed a comment on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635522914 **[Test build #123243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123243/testReport)** for PR 28666 at commit

[GitHub] [spark] SparkQA commented on pull request #28666: [BUILD][INFRA] bump the timeout to match the jenkins PRB

SparkQA commented on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635615385 **[Test build #123243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123243/testReport)** for PR 28666 at commit

[GitHub] [spark] SparkQA commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

SparkQA commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-635615292 **[Test build #123246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123246/testReport)** for PR 27066 at commit

[GitHub] [spark] dilip-k-m edited a comment on pull request #4039: [SPARK-5236] Fix ClassCastException in SpecificMutableRow

dilip-k-m edited a comment on pull request #4039: URL: https://github.com/apache/spark/pull/4039#issuecomment-635613051 I've got the same issue in production with Spark 1.6. I was able to replicate in our performance test environment. So, concluded that, with the same cluster

[GitHub] [spark] dilip-k-m commented on pull request #4039: [SPARK-5236] Fix ClassCastException in SpecificMutableRow

dilip-k-m commented on pull request #4039: URL: https://github.com/apache/spark/pull/4039#issuecomment-635613051 I've got the same issue in production. I was able to replicate in our performance test environment. So, concluded that, with the same cluster configuration, if a spark job is

[GitHub] [spark] fqaiser94 commented on a change in pull request #27066: [SPARK-31317][SQL] Add withField method to Column

fqaiser94 commented on a change in pull request #27066: URL: https://github.com/apache/spark/pull/27066#discussion_r432128545 ## File path: sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala ## @@ -923,4 +923,452 @@ class ColumnExpressionSuite extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

AmplabJenkins removed a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635610362 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

AmplabJenkins commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635610362 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

SparkQA removed a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635459663 **[Test build #123239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123239/testReport)** for PR 28650 at commit

[GitHub] [spark] SparkQA commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

SparkQA commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635609182 **[Test build #123239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123239/testReport)** for PR 28650 at commit

[GitHub] [spark] shaneknapp commented on pull request #28666: [BUILD][INFRA] bump the timeout to match the jenkins PRB

shaneknapp commented on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635607691 i'm going to go ahead and merge this now, as it's blocking maven PRs. @srowen @HyukjinKwon @sarutak

[GitHub] [spark] AmplabJenkins commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635608008 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins removed a comment on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635608008 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

SparkQA removed a comment on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635515643 **[Test build #123242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123242/testReport)** for PR 28665 at commit

[GitHub] [spark] SparkQA commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

SparkQA commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635607100 **[Test build #123242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123242/testReport)** for PR 28665 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28667: [WIP][SPARK-31860][BUILD] only push release tags on success

AmplabJenkins removed a comment on pull request #28667: URL: https://github.com/apache/spark/pull/28667#issuecomment-635605706 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28667: [WIP][SPARK-31860][BUILD] only push release tags on success

AmplabJenkins commented on pull request #28667: URL: https://github.com/apache/spark/pull/28667#issuecomment-635605706 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28667: [WIP][SPARK-31860][BUILD] only push release tags on success

SparkQA commented on pull request #28667: URL: https://github.com/apache/spark/pull/28667#issuecomment-635605135 **[Test build #123245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123245/testReport)** for PR 28667 at commit

[GitHub] [spark] HeartSaVioR edited a comment on pull request #27694: [SPARK-30946][SS] Serde entry with UnsafeRow on FileStream(Source/Sink)Log with LZ4 compression

HeartSaVioR edited a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-635603504 > One lesson I learned from the past is UnsafeRow is not designed to be persisted across Spark versions. This sounds like a blocker for SS, as we leverage it

[GitHub] [spark] holdenk commented on pull request #28667: [WIP][SPARK-31860][BUILD] only push release tags on success

holdenk commented on pull request #28667: URL: https://github.com/apache/spark/pull/28667#issuecomment-635603600 cc @dongjoon-hyun ? This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] holdenk opened a new pull request #28667: [WIP][SPARK-31860][BUILD] only push release tags on success

holdenk opened a new pull request #28667: URL: https://github.com/apache/spark/pull/28667 ### What changes were proposed in this pull request? Only push the release tag after the build has finished. ### Why are the changes needed? If the build fails we don't need a

[GitHub] [spark] HeartSaVioR edited a comment on pull request #27694: [SPARK-30946][SS] Serde entry with UnsafeRow on FileStream(Source/Sink)Log with LZ4 compression

HeartSaVioR edited a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-635603504 > One lesson I learned from the past is UnsafeRow is not designed to be persisted across Spark versions. This sounds like a blocker for SS, as we leverage it

[GitHub] [spark] HeartSaVioR commented on pull request #27694: [SPARK-30946][SS] Serde entry with UnsafeRow on FileStream(Source/Sink)Log with LZ4 compression

HeartSaVioR commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-635603504 > One lesson I learned from the past is UnsafeRow is not designed to be persisted across Spark versions. This sounds like a blocker for SS, as we leverage it to store

[GitHub] [spark] holdenk commented on a change in pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

holdenk commented on a change in pull request #28331: URL: https://github.com/apache/spark/pull/28331#discussion_r432110087 ## File path: core/src/test/scala/org/apache/spark/storage/BlockManagerDecommissionSuite.scala ## @@ -69,36 +84,64 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635591610 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635591610 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] holdenk commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

holdenk commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635591113 Oh wait we already do that in the K8s integration test. So I think that's tested but I'll see what I can do to improve the unit tests as well.

[GitHub] [spark] SparkQA commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635590890 **[Test build #123244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123244/testReport)** for PR 28331 at commit

[GitHub] [spark] holdenk commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

holdenk commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635583163 Probably the easist thing to do is delete the executor in the K8s test after it's had a chance to migrate the blocks.

[GitHub] [spark] holdenk commented on a change in pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

holdenk commented on a change in pull request #28331: URL: https://github.com/apache/spark/pull/28331#discussion_r432097155 ## File path: core/src/test/scala/org/apache/spark/storage/BlockManagerDecommissionSuite.scala ## @@ -69,36 +84,64 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

AmplabJenkins removed a comment on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635549762 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

AmplabJenkins commented on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635549762 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

SparkQA removed a comment on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635463295 **[Test build #123240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123240/testReport)** for PR 28658 at commit

[GitHub] [spark] SparkQA commented on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

SparkQA commented on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635548841 **[Test build #123240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123240/testReport)** for PR 28658 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28654: [SPARK-31834][SQL]Improve error message for incompatible data types

AmplabJenkins removed a comment on pull request #28654: URL: https://github.com/apache/spark/pull/28654#issuecomment-635541684 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28654: [SPARK-31834][SQL]Improve error message for incompatible data types

AmplabJenkins commented on pull request #28654: URL: https://github.com/apache/spark/pull/28654#issuecomment-635541684 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28654: [SPARK-31834][SQL]Improve error message for incompatible data types

SparkQA removed a comment on pull request #28654: URL: https://github.com/apache/spark/pull/28654#issuecomment-635384182 **[Test build #123236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123236/testReport)** for PR 28654 at commit

[GitHub] [spark] SparkQA commented on pull request #28654: [SPARK-31834][SQL]Improve error message for incompatible data types

SparkQA commented on pull request #28654: URL: https://github.com/apache/spark/pull/28654#issuecomment-635540632 **[Test build #123236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123236/testReport)** for PR 28654 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28626: [SPARK-28481][SQL] More expressions should extend NullIntolerant

AmplabJenkins removed a comment on pull request #28626: URL: https://github.com/apache/spark/pull/28626#issuecomment-635535474 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28626: [SPARK-28481][SQL] More expressions should extend NullIntolerant

AmplabJenkins commented on pull request #28626: URL: https://github.com/apache/spark/pull/28626#issuecomment-635535474 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28626: [SPARK-28481][SQL] More expressions should extend NullIntolerant

SparkQA removed a comment on pull request #28626: URL: https://github.com/apache/spark/pull/28626#issuecomment-635384195 **[Test build #123237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123237/testReport)** for PR 28626 at commit

[GitHub] [spark] SparkQA commented on pull request #28626: [SPARK-28481][SQL] More expressions should extend NullIntolerant

SparkQA commented on pull request #28626: URL: https://github.com/apache/spark/pull/28626#issuecomment-635534477 **[Test build #123237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123237/testReport)** for PR 28626 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28666: bump the timeout to match the jenkins PRB

AmplabJenkins removed a comment on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635523623 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28666: bump the timeout to match the jenkins PRB

AmplabJenkins commented on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635523623 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28666: bump the timeout to match the jenkins PRB

SparkQA commented on pull request #28666: URL: https://github.com/apache/spark/pull/28666#issuecomment-635522914 **[Test build #123243 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123243/testReport)** for PR 28666 at commit

[GitHub] [spark] shaneknapp opened a new pull request #28666: bump the timeout to match the jenkins PRB

shaneknapp opened a new pull request #28666: URL: https://github.com/apache/spark/pull/28666 ### What changes were proposed in this pull request? bump the timeout to match what's set in jenkins ### Why are the changes needed? tests be timing out! ### Does this PR

[GitHub] [spark] skonto edited a comment on pull request #28561: [SPARK-31740][K8S][TESTS] Use github URL instead of a broken link

skonto edited a comment on pull request #28561: URL: https://github.com/apache/spark/pull/28561#issuecomment-635519286 +1 LGTM. AFAIK this url is very old check

[GitHub] [spark] skonto commented on pull request #28561: [SPARK-31740][K8S][TESTS] Use github URL instead of a broken link

skonto commented on pull request #28561: URL: https://github.com/apache/spark/pull/28561#issuecomment-635519286 +1 LGTM. AFAIK this url is very old check

[GitHub] [spark] skonto edited a comment on pull request #28561: [SPARK-31740][K8S][TESTS] Use github URL instead of a broken link

skonto edited a comment on pull request #28561: URL: https://github.com/apache/spark/pull/28561#issuecomment-635519286 +1 LGTM. AFAIK this url is very old check

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-635513495 **[Test build #123235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123235/testReport)** for PR 27690 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635516319 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins removed a comment on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635516319 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

SparkQA commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635515643 **[Test build #123242 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123242/testReport)** for PR 28665 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-635514705 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-635514705 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins removed a comment on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635513453 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins removed a comment on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635513441 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

SparkQA removed a comment on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635511876 **[Test build #123241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123241/testReport)** for PR 28665 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-635355138 **[Test build #123235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123235/testReport)** for PR 27690 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635513441 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

SparkQA commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635513427 **[Test build #123241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123241/testReport)** for PR 28665 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins removed a comment on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635512463 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

AmplabJenkins commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635512463 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28665: [SPARK-31858][BUILD][test-hadoop3.2] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

SparkQA commented on pull request #28665: URL: https://github.com/apache/spark/pull/28665#issuecomment-635511876 **[Test build #123241 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123241/testReport)** for PR 28665 at commit

[GitHub] [spark] dongjoon-hyun opened a new pull request #28665: [SPARK-31858][BUILD] Upgrade commons-io to 2.5 in Hadoop 3.2 profile

dongjoon-hyun opened a new pull request #28665: URL: https://github.com/apache/spark/pull/28665 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ###

[GitHub] [spark] BryanCutler commented on pull request #28659: [SPARK-25351][PYTHON][TEST][FOLLOWUP] Fix test assertions to be consistent

BryanCutler commented on pull request #28659: URL: https://github.com/apache/spark/pull/28659#issuecomment-635506194 Thanks @HyukjinKwon ! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] EnricoMi commented on pull request #28663: [SPARK-31853][DOCS] Mention removal of params mixins setter in migration guide

EnricoMi commented on pull request #28663: URL: https://github.com/apache/spark/pull/28663#issuecomment-635498007 Code that used to call into the setters, e.g. `HasOutputCols.setOutputCols(value)` will not compile anymore.

[GitHub] [spark] attilapiros commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown

attilapiros commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-635492967 So it is not recalculation but the timing: the decommissioned executor is still giving back those shuffle data files: You can check it by adding a temporary log:

[GitHub] [spark] karuppayya commented on pull request #28662: [SPARK-31850] Prevent DetermineTableStats from computing stats multiple times for same table

karuppayya commented on pull request #28662: URL: https://github.com/apache/spark/pull/28662#issuecomment-635484418 @dongjoon-hyun @cloud-fan @gatorsmile Can anyone please help reviewing this change. Thanks This is an

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28661: [SPARK-31849][PYTHON][SQL] Make PySpark exceptions more Pythonic

AmplabJenkins removed a comment on pull request #28661: URL: https://github.com/apache/spark/pull/28661#issuecomment-635479265 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28661: [SPARK-31849][PYTHON][SQL] Make PySpark exceptions more Pythonic

AmplabJenkins removed a comment on pull request #28661: URL: https://github.com/apache/spark/pull/28661#issuecomment-635479255 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28661: [SPARK-31849][PYTHON][SQL] Make PySpark exceptions more Pythonic

AmplabJenkins commented on pull request #28661: URL: https://github.com/apache/spark/pull/28661#issuecomment-635479255 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28661: [SPARK-31849][PYTHON][SQL] Make PySpark exceptions more Pythonic

SparkQA removed a comment on pull request #28661: URL: https://github.com/apache/spark/pull/28661#issuecomment-635338654 **[Test build #123234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123234/testReport)** for PR 28661 at commit

[GitHub] [spark] SparkQA commented on pull request #28661: [SPARK-31849][PYTHON][SQL] Make PySpark exceptions more Pythonic

SparkQA commented on pull request #28661: URL: https://github.com/apache/spark/pull/28661#issuecomment-635478585 **[Test build #123234 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123234/testReport)** for PR 28661 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

AmplabJenkins removed a comment on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635463919 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] yaooqinn commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r431977318 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql ## @@ -160,3 +156,83 @@ select from_json('{"time":"26/October/2015"}', 'time

[GitHub] [spark] AmplabJenkins commented on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

AmplabJenkins commented on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635463919 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

SparkQA commented on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635463295 **[Test build #123240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123240/testReport)** for PR 28658 at commit

[GitHub] [spark] yaooqinn commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r431977318 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql ## @@ -160,3 +156,83 @@ select from_json('{"time":"26/October/2015"}', 'time

[GitHub] [spark] jiangxb1987 commented on pull request #28658: [SPARK-31730][CORE][TEST][3.0] Fix flaky tests in BarrierTaskContextSuite

jiangxb1987 commented on pull request #28658: URL: https://github.com/apache/spark/pull/28658#issuecomment-635461529 Retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

SparkQA commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635459663 **[Test build #123239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123239/testReport)** for PR 28650 at commit

[GitHub] [spark] yaooqinn edited a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn edited a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635448724 No the pattern `yyy-MM-dd` is valid for both version of formatters, but calling the `format()` throws an exception in the new one but silently suppressed in

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

AmplabJenkins removed a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635456809 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

AmplabJenkins commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635456809 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] yaooqinn commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635449894 But maybe we should apply `SparkUpgradeException ` to `format()` as `parse()` for better error msg for end-users

[GitHub] [spark] yaooqinn commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635448724 No the pattern `yyy-MM-dd` is valid for both version of formatters, but calling the `format()` throws an exception in the new one but silently suppressed in

[GitHub] [spark] cloud-fan commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

cloud-fan commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635441490 So we didn't fail when we construct the formatter with `yyy-MM-dd`? Then I think this PR doesn't help either.

[GitHub] [spark] yaooqinn commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r431946963 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1053,93 +1018,34 @@ case class

[GitHub] [spark] yaooqinn edited a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn edited a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635436944 > ``` > spark-sql> select from_unixtime(1, 'yyy-MM-dd'); > NULL > ``` > > Why we don't throw `SparkUpgradeException` in this case? that

[GitHub] [spark] yaooqinn commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635436944 > ``` > spark-sql> select from_unixtime(1, 'yyy-MM-dd'); > NULL > ``` > > Why we don't throw `SparkUpgradeException` in this case? that logic

[GitHub] [spark] yaooqinn commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r431945499 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -56,6 +56,28 @@ trait

[GitHub] [spark] cloud-fan commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

cloud-fan commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-635435177 ``` spark-sql> select from_unixtime(1, 'yyy-MM-dd'); NULL ``` Why we don't throw `SparkUpgradeException` in this case?

[GitHub] [spark] cloud-fan commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

cloud-fan commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r431943785 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1053,93 +1018,34 @@ case class

[GitHub] [spark] yaooqinn commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

yaooqinn commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r431941232 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1053,93 +1018,34 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

AmplabJenkins removed a comment on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-635430847 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

AmplabJenkins commented on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-635430847 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

SparkQA removed a comment on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-635384233 **[Test build #123238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123238/testReport)** for PR 28439 at commit

<    1   2   3   4   5   6   >