[GitHub] [spark] AmplabJenkins commented on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

AmplabJenkins commented on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632142075 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

SparkQA removed a comment on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632058436 **[Test build #122931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122931/testReport)** for PR 28525 at commit [`c59fcd6`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

SparkQA commented on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632140854 **[Test build #122931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122931/testReport)** for PR 28525 at commit [`c59fcd6`](https://github.co

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428704776 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.resp

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428699825 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.res

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428699825 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.res

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428693738 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.resp

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428693738 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.resp

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428691523 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.res

[GitHub] [spark] SparkQA commented on pull request #25965: [SPARK-26425][SS] Add more constraint checks in file streaming source to avoid checkpoint corruption

SparkQA commented on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-632108254 **[Test build #122934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122934/testReport)** for PR 25965 at commit [`1744179`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

AmplabJenkins removed a comment on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632104886 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

AmplabJenkins commented on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632104886 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

SparkQA commented on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632104207 **[Test build #122933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122933/testReport)** for PR 27664 at commit [`83451c1`](https://github.com

[GitHub] [spark] HeartSaVioR commented on a change in pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

HeartSaVioR commented on a change in pull request #27664: URL: https://github.com/apache/spark/pull/27664#discussion_r428668027 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -182,19 +182,34 @@ class HDFSMetadataLog[T

[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428667113 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter( ex

[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428667113 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter( ex

[GitHub] [spark] maropu commented on a change in pull request #28560: [SPARK-27217][SQL] Nested column aliasing for more operators which can prune nested column

maropu commented on a change in pull request #28560: URL: https://github.com/apache/spark/pull/28560#discussion_r428666366 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -341,6 +341,78 @@ class NestedColu

[GitHub] [spark] maropu commented on a change in pull request #28560: [SPARK-27217][SQL] Nested column aliasing for more operators which can prune nested column

maropu commented on a change in pull request #28560: URL: https://github.com/apache/spark/pull/28560#discussion_r428662926 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -48,7 +53,10 @@ object NestedColumnAlia

[GitHub] [spark] xuanyuanking commented on a change in pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

xuanyuanking commented on a change in pull request #27664: URL: https://github.com/apache/spark/pull/27664#discussion_r428661867 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -182,19 +182,34 @@ class HDFSMetadataLog[

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428661567 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.resp

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428657865 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.res

[GitHub] [spark] cloud-fan commented on pull request #27066: [SPARK-31317][SQL] Add withFields method to Column

cloud-fan commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-632087480 I went through the discussion in the JIRA, there is one interesting use case ``` data.withColumn("newArray", transform('array, structElem => structElem.withField("d",

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

AmplabJenkins removed a comment on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632082073 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] GuoPhilipse commented on a change in pull request #28593: [SPARK-31710][SQL] Add two compatibility flag to cast long to timestamp

GuoPhilipse commented on a change in pull request #28593: URL: https://github.com/apache/spark/pull/28593#discussion_r428646486 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2586,6 +2586,22 @@ object SQLConf { .checkValue(_

[GitHub] [spark] AmplabJenkins commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

AmplabJenkins commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632082073 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428645814 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if (!Utils.resp

[GitHub] [spark] SparkQA removed a comment on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

SparkQA removed a comment on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-631965191 **[Test build #122922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122922/testReport)** for PR 28556 at commit [`db601df`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

AmplabJenkins removed a comment on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632081247 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

SparkQA commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632081113 **[Test build #122922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122922/testReport)** for PR 28556 at commit [`db601df`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

AmplabJenkins commented on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632081247 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

SparkQA removed a comment on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632016717 **[Test build #122927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122927/testReport)** for PR 28583 at commit [`53ebb55`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28583: [SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier

SparkQA commented on pull request #28583: URL: https://github.com/apache/spark/pull/28583#issuecomment-632080202 **[Test build #122927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122927/testReport)** for PR 28583 at commit [`53ebb55`](https://github.co

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428643339 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -123,16 +134,24 @@ private class ClientEndpoint( }(forwardMessageExecu

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r428643084 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -123,16 +134,24 @@ private class ClientEndpoint( }(forwardMessageExecu

[GitHub] [spark] maropu commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

maropu commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632075943 I left some minor comments though, it looks okay. cc: @dongjoon-hyun @dbtsai This is an automated message from t

[GitHub] [spark] cloud-fan commented on pull request #27066: [SPARK-31317][SQL] Add withFields method to Column

cloud-fan commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-632075377 I haven't looked at the patch yet, but looking at the proposed API ``` val result = data.withColumn("a", 'a.withFields( ("b", $"a.b".withFields( ("a", $"a.b.a"

[GitHub] [spark] maropu commented on a change in pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

maropu commented on a change in pull request #28556: URL: https://github.com/apache/spark/pull/28556#discussion_r428635965 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -144,7 +144,6 @@ class NestedColum

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins removed a comment on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632069072 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632069072 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

SparkQA commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632068610 **[Test build #122932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122932/testReport)** for PR 28576 at commit [`161223b`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065812 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] SparkQA removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632032488 **[Test build #122928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122928/testReport)** for PR 28592 at commit [`1144c03`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065807 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065738 **[Test build #122928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122928/testReport)** for PR 28592 at commit [`1144c03`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632065807 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632059081 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

AmplabJenkins removed a comment on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632059000 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632059071 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

SparkQA removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-631955288 **[Test build #122921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122921/testReport)** for PR 28534 at commit [`c1257f2`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632059071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28525: [SPARK-27562][Shuffle] Complete the verification mechanism for shuffle transmitted data

AmplabJenkins commented on pull request #28525: URL: https://github.com/apache/spark/pull/28525#issuecomment-632059000 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

SparkQA commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632058682 **[Test build #122921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122921/testReport)** for PR 28534 at commit [`c1257f2`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

cloud-fan commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428615629 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter( e

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632052714 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] cloud-fan commented on a change in pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

cloud-fan commented on a change in pull request #28598: URL: https://github.com/apache/spark/pull/28598#discussion_r428614900 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetTest.scala ## @@ -162,4 +162,36 @@ private[sql] trait P

[GitHub] [spark] AmplabJenkins commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632052702 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632052702 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

SparkQA removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-631930528 **[Test build #122918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122918/testReport)** for PR 28534 at commit [`e4db9a5`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

SparkQA commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632052069 **[Test build #122918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122918/testReport)** for PR 28534 at commit [`e4db9a5`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins removed a comment on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632051500 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins removed a comment on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632051493 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

SparkQA removed a comment on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-631989880 **[Test build #122923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122923/testReport)** for PR 28576 at commit [`8de3bda`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

AmplabJenkins commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632051493 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28576: [SPARK-31755][SQL] allow missing year/hour when parsing date/timestamp string

SparkQA commented on pull request #28576: URL: https://github.com/apache/spark/pull/28576#issuecomment-632051200 **[Test build #122923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122923/testReport)** for PR 28576 at commit [`8de3bda`](https://github.co

[GitHub] [spark] cloud-fan commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

cloud-fan commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632049705 Let's also add some UT in `DateExpressionsSuite`, which can test both codegen and interpreted code path. This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

AmplabJenkins removed a comment on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632046933 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

AmplabJenkins commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632046933 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

AmplabJenkins removed a comment on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632046310 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

cloud-fan commented on a change in pull request #28534: URL: https://github.com/apache/spark/pull/28534#discussion_r428607554 ## File path: sql/core/src/test/resources/sql-tests/results/ansi/datetime.sql.out ## @@ -1,5 +1,65 @@ -- Automatically generated by SQLQueryTestSuite -

[GitHub] [spark] SparkQA removed a comment on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

SparkQA removed a comment on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-631923918 **[Test build #122917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122917/testReport)** for PR 28556 at commit [`b77a1ba`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

AmplabJenkins commented on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632046310 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28556: [SPARK-31736][SQL] Nested column aliasing for RepartitionByExpression/Join

SparkQA commented on pull request #28556: URL: https://github.com/apache/spark/pull/28556#issuecomment-632046065 **[Test build #122917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122917/testReport)** for PR 28556 at commit [`b77a1ba`](https://github.co

[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428607225 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime-corrected.sql ## @@ -0,0 +1,2 @@ +--SET spark.sql.legacy.timeParserPolicy=CORRECTED

[GitHub] [spark] cloud-fan commented on a change in pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

cloud-fan commented on a change in pull request #28534: URL: https://github.com/apache/spark/pull/28534#discussion_r428606868 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -401,6 +401,83 @@ case class DayOfY

[GitHub] [spark] SparkQA commented on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

SparkQA commented on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632045706 **[Test build #122930 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122930/testReport)** for PR 28598 at commit [`1c8b0e9`](https://github.com

[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428606030 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -40,7 +43,23 @@ class Iso8601DateFormatter( ex

[GitHub] [spark] cloud-fan commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

cloud-fan commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428605898 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime-corrected.sql ## @@ -0,0 +1,2 @@ +--SET spark.sql.legacy.timeParserPolicy=CORRECTED

[GitHub] [spark] MaxGekk commented on pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

MaxGekk commented on pull request #28598: URL: https://github.com/apache/spark/pull/28598#issuecomment-632043745 @cloud-fan @mswit-databricks @adrian-ionescu Please, review this PR. This is an automated message from the Apach

[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428604890 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -77,14 +96,14 @@ trait LegacyDateFormatter extend

[GitHub] [spark] MaxGekk opened a new pull request #28598: [SPARK-31785][SQL][TESTS] Add a helper function to test all parquet readers

MaxGekk opened a new pull request #28598: URL: https://github.com/apache/spark/pull/28598 ### What changes were proposed in this pull request? Add `withAllParquetReaders` to `ParquetTest`. The function allow to run a block of code for all available Parquet readers. ### Why are the

[GitHub] [spark] cloud-fan commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

cloud-fan commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428604480 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TimestampFormatter.scala ## @@ -61,8 +64,23 @@ class Iso8601TimestampForma

[GitHub] [spark] cloud-fan commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

cloud-fan commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r428603874 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala ## @@ -77,14 +96,14 @@ trait LegacyDateFormatter exten

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

AmplabJenkins removed a comment on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632039530 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

AmplabJenkins commented on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632039530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

SparkQA commented on pull request #28128: URL: https://github.com/apache/spark/pull/28128#issuecomment-632038999 **[Test build #122929 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122929/testReport)** for PR 28128 at commit [`cfa1462`](https://github.com

[GitHub] [spark] vinooganesh commented on a change in pull request #28128: [SPARK-31354] SparkContext only register one SparkSession ApplicationEnd listener

vinooganesh commented on a change in pull request #28128: URL: https://github.com/apache/spark/pull/28128#discussion_r428599620 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala ## @@ -1064,6 +1055,20 @@ object SparkSession extends Logging { // Pr

[GitHub] [spark] cloud-fan edited a comment on pull request #28523: [SPARK-31706][SQL] add back the support of streaming update mode

cloud-fan edited a comment on pull request #28523: URL: https://github.com/apache/spark/pull/28523#issuecomment-631869094 I thought you have canceled your veto in https://github.com/apache/spark/pull/28523#issuecomment-628164147 , as what you agreed on is exactly what this PR is doing. Hop

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632033075 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632033075 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632032488 **[Test build #122928 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122928/testReport)** for PR 28592 at commit [`1144c03`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"

AmplabJenkins removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632027308 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"

AmplabJenkins commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632027308 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"

SparkQA removed a comment on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-631955215 **[Test build #122920 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122920/testReport)** for PR 28596 at commit [`2be2ca7`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28596: [SPARK-31784][CORE][TEST] Fix test BarrierTaskContextSuite."share messages with allGather() call"

SparkQA commented on pull request #28596: URL: https://github.com/apache/spark/pull/28596#issuecomment-632026359 **[Test build #122920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122920/testReport)** for PR 28596 at commit [`2be2ca7`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632021186 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632021179 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632013240 **[Test build #122926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122926/testReport)** for PR 28592 at commit [`1d31ed2`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632021131 **[Test build #122926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122926/testReport)** for PR 28592 at commit [`1d31ed2`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632021179 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] sarutak commented on a change in pull request #28585: [SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0

sarutak commented on a change in pull request #28585: URL: https://github.com/apache/spark/pull/28585#discussion_r428578312 ## File path: core/src/test/scala/org/apache/spark/ui/UISeleniumSuite.scala ## @@ -784,6 +784,7 @@ class UISeleniumSuite extends SparkFunSuite with WebBro

<    1   2   3   4   5   6   >