[GitHub] [spark] AmplabJenkins removed a comment on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632631225 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632636609 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632636609 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28605: [SPARK-31790][DOCS] cast(long as timestamp) show different result between Hive and Spark

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632445014 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28605: [SPARK-31790][DOCS] cast(long as timestamp) show different result between Hive and Spark

2020-05-22 Thread GitBox
HyukjinKwon commented on a change in pull request #28605: URL: https://github.com/apache/spark/pull/28605#discussion_r429186020 ## File path: docs/sql-migration-guide.md ## @@ -961,3 +961,4 @@ Below are the scenarios in which Hive and Spark generate different results: *

[GitHub] [spark] HyukjinKwon commented on pull request #28605: [SPARK-31790][DOCS] cast(long as timestamp) show different result between Hive and Spark

2020-05-22 Thread GitBox
HyukjinKwon commented on pull request #28605: URL: https://github.com/apache/spark/pull/28605#issuecomment-632636610 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] akshatb1 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

2020-05-22 Thread GitBox
akshatb1 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r429196227 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +202,22 @@ private class ClientEndpoint( } else if

[GitHub] [spark] SparkQA removed a comment on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632543111 **[Test build #122978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122978/testReport)** for PR 28604 at commit

[GitHub] [spark] SparkQA commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input

2020-05-22 Thread GitBox
SparkQA commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632654184 **[Test build #122978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122978/testReport)** for PR 28604 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28611: [DO-NOT-MERGE][SPARK-XXXXX][SQL] Count only R/W (getter/setter) properties as columns for Java Beans

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28611: URL: https://github.com/apache/spark/pull/28611#issuecomment-632655297 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28611: [DO-NOT-MERGE][SPARK-XXXXX][SQL] Count only R/W (getter/setter) properties as columns for Java Beans

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28611: URL: https://github.com/apache/spark/pull/28611#issuecomment-632655297 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on pull request #28604: [SPARK-29854][SQL][TESTS] Add tests to check lpad/rpad throw an exception for invalid length input

2020-05-22 Thread GitBox
HyukjinKwon commented on pull request #28604: URL: https://github.com/apache/spark/pull/28604#issuecomment-632660576 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632663148 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632663148 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #28612: [SPARK-31755][SQL][3.0] allow missing year/hour when parsing date/timestamp string

2020-05-22 Thread GitBox
cloud-fan commented on pull request #28612: URL: https://github.com/apache/spark/pull/28612#issuecomment-632672601 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #28612: [SPARK-31755][SQL][3.0] allow missing year/hour when parsing date/timestamp string

2020-05-22 Thread GitBox
SparkQA commented on pull request #28612: URL: https://github.com/apache/spark/pull/28612#issuecomment-632672925 **[Test build #123002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123002/testReport)** for PR 28612 at commit

[GitHub] [spark] cloud-fan commented on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator

2020-05-22 Thread GitBox
cloud-fan commented on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632687576 > from the test case shall I remove checkEvaluation for byte,short and int ? Yes please This is an

[GitHub] [spark] prakharjain09 commented on pull request #28619: [WIP][SPARK-21040][CORE] Speculate tasks which are running on decommission executors

2020-05-22 Thread GitBox
prakharjain09 commented on pull request #28619: URL: https://github.com/apache/spark/pull/28619#issuecomment-632985151 I am adding more UTs for this change. So added WIP in title. @holdenk Please give some feedback on the actual logic.

[GitHub] [spark] maropu commented on a change in pull request #28610: [SPARK-31793][SQL] Reduce the memory usage in file scan location metadata

2020-05-22 Thread GitBox
maropu commented on a change in pull request #28610: URL: https://github.com/apache/spark/pull/28610#discussion_r429242653 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -2904,6 +2904,24 @@ private[spark] object Utils extends Logging {

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2

2020-05-22 Thread GitBox
dongjoon-hyun edited a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632694613 Thank you for review, @maver1ck and @ScrapCodes . In this PR, I tried 4.10.1 and 4.10.0 first. Please see the first commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632700711 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632700603 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632700711 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Ngone51 commented on pull request #26339: [SPARK-27194][SPARK-29302][SQL] For dynamic partition overwrite operation, fix speculation task conflict issue and FileAlreadyExistsException

2020-05-22 Thread GitBox
Ngone51 commented on pull request #26339: URL: https://github.com/apache/spark/pull/26339#issuecomment-632722257 The vote thread now has more eys on than this PR and as you know this PR is somehow overlooked for a while.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28612: [SPARK-31755][SQL][3.0] allow missing year/hour when parsing date/timestamp string

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28612: URL: https://github.com/apache/spark/pull/28612#issuecomment-632730356 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] zsxwing commented on a change in pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

2020-05-22 Thread GitBox
zsxwing commented on a change in pull request #27664: URL: https://github.com/apache/spark/pull/27664#discussion_r429291412 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -182,19 +182,34 @@ class HDFSMetadataLog[T <:

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Add two compatibility flag to cast long to timestamp

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-632749045 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] srowen commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

2020-05-22 Thread GitBox
srowen commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-632762784 Jenkins retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632767499 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632636242 **[Test build #122994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122994/testReport)** for PR 28592 at commit

[GitHub] [spark] SparkQA commented on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-22 Thread GitBox
SparkQA commented on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632766977 **[Test build #122994 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122994/testReport)** for PR 28592 at commit

[GitHub] [spark] srowen commented on pull request #28585: [SPARK-31765][WEBUI] Upgrade HtmlUnit >= 2.37.0

2020-05-22 Thread GitBox
srowen commented on pull request #28585: URL: https://github.com/apache/spark/pull/28585#issuecomment-632689314 I merged https://github.com/apache/spark/pull/28578 ; this may need a rebase This is an automated message from

[GitHub] [spark] srowen closed pull request #28578: [SPARK-31756][WEBUI] Add real headless browser support for UI test

2020-05-22 Thread GitBox
srowen closed pull request #28578: URL: https://github.com/apache/spark/pull/28578 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] srowen commented on pull request #28578: [SPARK-31756][WEBUI] Add real headless browser support for UI test

2020-05-22 Thread GitBox
srowen commented on pull request #28578: URL: https://github.com/apache/spark/pull/28578#issuecomment-632688904 Merged to master This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] srowen commented on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
srowen commented on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632699431 Jenkins test this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632704305 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28611: [DO-NOT-MERGE][SPARK-XXXXX][SQL] Count only R/W (getter/setter) properties as columns for Java Beans

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #28611: URL: https://github.com/apache/spark/pull/28611#issuecomment-632654875 **[Test build #122997 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122997/testReport)** for PR 28611 at commit

[GitHub] [spark] Ngone51 commented on pull request #26339: [SPARK-27194][SPARK-29302][SQL] For dynamic partition overwrite operation, fix speculation task conflict issue and FileAlreadyExistsException

2020-05-22 Thread GitBox
Ngone51 commented on pull request #26339: URL: https://github.com/apache/spark/pull/26339#issuecomment-63271 @koertkuipers you could send your concern to the [vote of Spark 3.0 release](http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Apache-Spark-3-0-RC2-td29419.html)

[GitHub] [spark] SparkQA commented on pull request #28611: [DO-NOT-MERGE][SPARK-XXXXX][SQL] Count only R/W (getter/setter) properties as columns for Java Beans

2020-05-22 Thread GitBox
SparkQA commented on pull request #28611: URL: https://github.com/apache/spark/pull/28611#issuecomment-632717664 **[Test build #122997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122997/testReport)** for PR 28611 at commit

[GitHub] [spark] Ngone51 edited a comment on pull request #26339: [SPARK-27194][SPARK-29302][SQL] For dynamic partition overwrite operation, fix speculation task conflict issue and FileAlreadyExistsEx

2020-05-22 Thread GitBox
Ngone51 edited a comment on pull request #26339: URL: https://github.com/apache/spark/pull/26339#issuecomment-63271 @koertkuipers you could send your concern to the [vote of Spark 3.0

[GitHub] [spark] TJX2014 commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
TJX2014 commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-63272 > Hi @TJX2014 , the original JIRA doesn't fit this PR very well. Can you create a new JIRA ticket and update the PR title? Thanks! @cloud-fan ok, I will create the jira.

[GitHub] [spark] TJX2014 commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
TJX2014 commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632722927 Thanks very much. Got a lot. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28612: [SPARK-31755][SQL][3.0] allow missing year/hour when parsing date/timestamp string

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28612: URL: https://github.com/apache/spark/pull/28612#issuecomment-632730343 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28612: [SPARK-31755][SQL][3.0] allow missing year/hour when parsing date/timestamp string

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #28612: URL: https://github.com/apache/spark/pull/28612#issuecomment-632672925 **[Test build #123002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123002/testReport)** for PR 28612 at commit

[GitHub] [spark] SparkQA commented on pull request #28612: [SPARK-31755][SQL][3.0] allow missing year/hour when parsing date/timestamp string

2020-05-22 Thread GitBox
SparkQA commented on pull request #28612: URL: https://github.com/apache/spark/pull/28612#issuecomment-632730125 **[Test build #123002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123002/testReport)** for PR 28612 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28612: [SPARK-31755][SQL][3.0] allow missing year/hour when parsing date/timestamp string

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28612: URL: https://github.com/apache/spark/pull/28612#issuecomment-632730343 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632736293 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
SparkQA commented on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632744896 **[Test build #123006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123006/testReport)** for PR 28608 at commit

[GitHub] [spark] huaxingao commented on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
huaxingao commented on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632745066 @srowen This is for 3.0. Thanks! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] holdenk commented on pull request #26440: [SPARK-20628][CORE][K8S] Start to improve Spark decommissioning & preemption support

2020-05-22 Thread GitBox
holdenk commented on pull request #26440: URL: https://github.com/apache/spark/pull/26440#issuecomment-632761584 Some other committers reviewed it early on and I brought it to the dev@ list and left it open after stating intent to merge Incase any committer who had been involved with

[GitHub] [spark] dongjoon-hyun commented on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2

2020-05-22 Thread GitBox
dongjoon-hyun commented on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632694613 Thank you for review, @maver1ck and @ScrapCodes . In this PR, I tried 4.10.1 and 4.10.0 first. However, the dependency change was too huge. And, `kubernetes-model` was

[GitHub] [spark] SparkQA commented on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
SparkQA commented on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632700218 **[Test build #123004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123004/testReport)** for PR 28439 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632700603 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] GuoPhilipse commented on a change in pull request #28593: [SPARK-31710][SQL] Add two compatibility flag to cast long to timestamp

2020-05-22 Thread GitBox
GuoPhilipse commented on a change in pull request #28593: URL: https://github.com/apache/spark/pull/28593#discussion_r429272954 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2586,6 +2586,22 @@ object SQLConf {

[GitHub] [spark] cloud-fan commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
cloud-fan commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632715986 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
cloud-fan closed pull request #28534: URL: https://github.com/apache/spark/pull/28534 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28611: [DO-NOT-MERGE][SPARK-XXXXX][SQL] Count only R/W (getter/setter) properties as columns for Java Beans

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28611: URL: https://github.com/apache/spark/pull/28611#issuecomment-632717904 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] koertkuipers commented on pull request #26339: [SPARK-27194][SPARK-29302][SQL] For dynamic partition overwrite operation, fix speculation task conflict issue and FileAlreadyExistsExce

2020-05-22 Thread GitBox
koertkuipers commented on pull request #26339: URL: https://github.com/apache/spark/pull/26339#issuecomment-632720826 @Ngone51 yeah i thought about doing that, but i dont want to slowdown the spark 3 release even more (and this is not a regression i guess?). now i am just hoping someone

[GitHub] [spark] cloud-fan commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
cloud-fan commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632720666 Hi @TJX2014 , the original JIRA doesn't fit this PR very well. Can you create a new JIRA ticket and update the PR title? Thanks!

[GitHub] [spark] SparkQA removed a comment on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632601177 **[Test build #122991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122991/testReport)** for PR 27006 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632736293 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on a change in pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

2020-05-22 Thread GitBox
HeartSaVioR commented on a change in pull request #27664: URL: https://github.com/apache/spark/pull/27664#discussion_r429313023 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/FileStreamSinkLogSuite.scala ## @@ -267,4 +308,38 @@ class

[GitHub] [spark] SparkQA commented on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
SparkQA commented on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632757331 **[Test build #123007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123007/testReport)** for PR 27006 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632767507 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28592: URL: https://github.com/apache/spark/pull/28592#issuecomment-632767499 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] sandeep-katta commented on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator

2020-05-22 Thread GitBox
sandeep-katta commented on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632692345 > ``` > org.scalatest.exceptions.TestFailedException: "struct<([3 div 2]):bigint>" did not equal "struct<([CAST(3 AS BIGINT) div CAST(2 AS BIGINT)]):bigint>" Schema

[GitHub] [spark] SparkQA commented on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator

2020-05-22 Thread GitBox
SparkQA commented on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632703701 **[Test build #123005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123005/testReport)** for PR 28600 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632704305 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] koertkuipers commented on pull request #26339: [SPARK-27194][SPARK-29302][SQL] For dynamic partition overwrite operation, fix speculation task conflict issue and FileAlreadyExistsExce

2020-05-22 Thread GitBox
koertkuipers commented on pull request #26339: URL: https://github.com/apache/spark/pull/26339#issuecomment-632707720 i am getting worried now this wont make it into spark 3.0.0 this is a fault tolerance bug in spark. not as serious as a correctness issue, but pretty high up there i

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632746622 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632746622 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
cloud-fan commented on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632746709 sorry it conflicts, can you fix it? This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Add two compatibility flag to cast long to timestamp

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-632749045 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Add two compatibility flag to cast long to timestamp

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-632749787 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632754175 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632754175 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632765866 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-632765921 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on pull request #28600: [SPARK-31761][SQL] cast integer to Long to avoid IntegerOverflow for IntegralDivide operator

2020-05-22 Thread GitBox
cloud-fan commented on pull request #28600: URL: https://github.com/apache/spark/pull/28600#issuecomment-632690724 ``` org.scalatest.exceptions.TestFailedException: "struct<([3 div 2]):bigint>" did not equal "struct<([CAST(3 AS BIGINT) div CAST(2 AS BIGINT)]):bigint>" Schema did not

[GitHub] [spark] maropu commented on a change in pull request #28610: [SPARK-31793][SQL] Reduce the memory usage in file scan location metadata

2020-05-22 Thread GitBox
maropu commented on a change in pull request #28610: URL: https://github.com/apache/spark/pull/28610#discussion_r429249205 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -55,10 +55,12 @@ trait DataSourceScanExec extends

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28601: [SPARK-31786][K8S][BUILD] Upgrade kubernetes-client to 4.9.2

2020-05-22 Thread GitBox
dongjoon-hyun edited a comment on pull request #28601: URL: https://github.com/apache/spark/pull/28601#issuecomment-632694613 Thank you for review, @maver1ck and @ScrapCodes . In this PR, I tried 4.10.1 and 4.10.0 first. Please see the first commit

[GitHub] [spark] SparkQA commented on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
SparkQA commented on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632699516 **[Test build #122987 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122987/testReport)** for PR 28534 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28534: [SPARK-31710][SQL] Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #28534: URL: https://github.com/apache/spark/pull/28534#issuecomment-632576718 **[Test build #122987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122987/testReport)** for PR 28534 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28611: [DO-NOT-MERGE][SPARK-XXXXX][SQL] Count only R/W (getter/setter) properties as columns for Java Beans

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28611: URL: https://github.com/apache/spark/pull/28611#issuecomment-632717893 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28611: [DO-NOT-MERGE][SPARK-XXXXX][SQL] Count only R/W (getter/setter) properties as columns for Java Beans

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28611: URL: https://github.com/apache/spark/pull/28611#issuecomment-632717893 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] gatorsmile commented on pull request #26440: [SPARK-20628][CORE][K8S] Start to improve Spark decommissioning & preemption support

2020-05-22 Thread GitBox
gatorsmile commented on pull request #26440: URL: https://github.com/apache/spark/pull/26440#issuecomment-632723714 @holdenk Did we get any LGTM from the other committer before merging this PR? This is an automated message

[GitHub] [spark] SparkQA commented on pull request #27006: [SPARK-30352][SQL] DataSourceV2: Add CURRENT_CATALOG function

2020-05-22 Thread GitBox
SparkQA commented on pull request #27006: URL: https://github.com/apache/spark/pull/27006#issuecomment-632734387 **[Test build #122991 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122991/testReport)** for PR 27006 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632745496 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632745496 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
SparkQA commented on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632746005 **[Test build #123004 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123004/testReport)** for PR 28439 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28593: [SPARK-31710][SQL] Add two compatibility flag to cast long to timestamp

2020-05-22 Thread GitBox
cloud-fan commented on a change in pull request #28593: URL: https://github.com/apache/spark/pull/28593#discussion_r429305774 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2586,6 +2586,22 @@ object SQLConf { .checkValue(_

[GitHub] [spark] SparkQA removed a comment on pull request #28439: [SPARK-30119][WEBUI] Add Pagination Support to Streaming Page

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #28439: URL: https://github.com/apache/spark/pull/28439#issuecomment-632700218 **[Test build #123004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123004/testReport)** for PR 28439 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632752231 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
SparkQA commented on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632752009 **[Test build #123006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123006/testReport)** for PR 28608 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
SparkQA removed a comment on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632744896 **[Test build #123006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123006/testReport)** for PR 28608 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28608: [SPARK-31612][SQL][DOCS][FOLLOW-UP] Fix a few issues in SQL ref

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28608: URL: https://github.com/apache/spark/pull/28608#issuecomment-632752231 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

2020-05-22 Thread GitBox
AmplabJenkins commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-632765921 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

2020-05-22 Thread GitBox
AmplabJenkins removed a comment on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632765866 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #27664: [SPARK-30915][SS] CompactibleFileStreamLog: Avoid reading the metadata log file when finding the latest batch ID

2020-05-22 Thread GitBox
SparkQA commented on pull request #27664: URL: https://github.com/apache/spark/pull/27664#issuecomment-632765181 **[Test build #123009 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123009/testReport)** for PR 27664 at commit

[GitHub] [spark] SparkQA commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

2020-05-22 Thread GitBox
SparkQA commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-632765189 **[Test build #123008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123008/testReport)** for PR 28485 at commit

<    1   2   3   4   5   6   7   >