[GitHub] [spark] SparkQA removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641471490 **[Test build #123701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123701/testReport)** for PR 28593 at commit [`1d5c750`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

2020-06-09 Thread GitBox
SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641611980 **[Test build #123701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123701/testReport)** for PR 28593 at commit [`1d5c750`](https://github.co

[GitHub] [spark] gengliangwang commented on a change in pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

2020-06-09 Thread GitBox
gengliangwang commented on a change in pull request #28733: URL: https://github.com/apache/spark/pull/28733#discussion_r437753240 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala ## @@ -198,6 +199,88 @@ trait PredicateHelper

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641610426 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641610426 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

2020-06-09 Thread GitBox
SparkQA commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641609954 **[Test build #123706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123706/testReport)** for PR 28772 at commit [`6bd326f`](https://github.com

[GitHub] [spark] huaxingao opened a new pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

2020-06-09 Thread GitBox
huaxingao opened a new pull request #28772: URL: https://github.com/apache/spark/pull/28772 ### What changes were proposed in this pull request? Add instance weight support in LinearRegressionSummary ### Why are the changes needed? LinearRegression and RegressionMetrics

[GitHub] [spark] jzhuge commented on pull request #28147: [SPARK-31357][SQL][WIP] Catalog API for view metadata

2020-06-09 Thread GitBox
jzhuge commented on pull request #28147: URL: https://github.com/apache/spark/pull/28147#issuecomment-641603964 @cloud-fan @dongjoon-hyun @brkyvz Could you comment on this WIP PR? Go or no-go? Need more clarifications? Even though it is labeled as WIP, the code is pretty stable and in prod

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641573252 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641573252 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-09 Thread GitBox
SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641573224 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28328/ ---

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-09 Thread GitBox
SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641558773 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28328/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641550578 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641550578 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

2020-06-09 Thread GitBox
SparkQA commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641549973 **[Test build #123705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123705/testReport)** for PR 28733 at commit [`a0c7110`](https://github.com

[GitHub] [spark] gengliangwang commented on a change in pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

2020-06-09 Thread GitBox
gengliangwang commented on a change in pull request #28733: URL: https://github.com/apache/spark/pull/28733#discussion_r437693276 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -545,6 +545,19 @@ object SQLConf { .booleanConf

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641545663 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641545654 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641475152 **[Test build #123702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123702/testReport)** for PR 28412 at commit [`3494307`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641545654 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
SparkQA commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641545201 **[Test build #123702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123702/testReport)** for PR 28412 at commit [`3494307`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641533468 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641533468 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
SparkQA commented on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641532792 **[Test build #123703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123703/testReport)** for PR 28771 at commit [`6984453`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

2020-06-09 Thread GitBox
SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641532764 **[Test build #123704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123704/testReport)** for PR 28708 at commit [`8f7d4eb`](https://github.com

[GitHub] [spark] gengliangwang commented on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
gengliangwang commented on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641531625 cc @liancheng This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] gengliangwang commented on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
gengliangwang commented on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641531353 This PR is to backport #28760 to branch-2.4 This is an automated message from the Apache Git Service. To r

[GitHub] [spark] gengliangwang opened a new pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
gengliangwang opened a new pull request #28771: URL: https://github.com/apache/spark/pull/28771 ### What changes were proposed in this pull request? Mkae Hadoop file system config effective in data source options. From `org.apache.hadoop.fs.FileSystem.java`: ```

[GitHub] [spark] gengliangwang closed pull request #28760: [SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
gengliangwang closed pull request #28760: URL: https://github.com/apache/spark/pull/28760 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] gengliangwang commented on pull request #28760: [SPARK-31935][SQL] Hadoop file system config should be effective in data source options

2020-06-09 Thread GitBox
gengliangwang commented on pull request #28760: URL: https://github.com/apache/spark/pull/28760#issuecomment-641516116 Merging to master. I will backport to branch-2.4 and branch-3.0 later This is an automated message from

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641512767 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641512767 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641406226 **[Test build #123700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123700/testReport)** for PR 28768 at commit [`e8769fb`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

2020-06-09 Thread GitBox
SparkQA commented on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641511786 **[Test build #123700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123700/testReport)** for PR 28768 at commit [`e8769fb`](https://github.co

[GitHub] [spark] siknezevic commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

2020-06-09 Thread GitBox
siknezevic commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-641508339 I am aware of test failures. I will push new update that will fix test issues. Due to program logic change these tests have to be adjusted. Just need to change limits when sp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28745: [SPARK-31915][SQL][PYTHON] Remove projection that adds grouping keys in grouped and cogrouped pandas UDFs

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28745: URL: https://github.com/apache/spark/pull/28745#issuecomment-641500668 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28745: [SPARK-31915][SQL][PYTHON] Remove projection that adds grouping keys in grouped and cogrouped pandas UDFs

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28745: URL: https://github.com/apache/spark/pull/28745#issuecomment-641500668 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28745: [SPARK-31915][SQL][PYTHON] Remove projection that adds grouping keys in grouped and cogrouped pandas UDFs

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #28745: URL: https://github.com/apache/spark/pull/28745#issuecomment-641289193 **[Test build #123698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123698/testReport)** for PR 28745 at commit [`2bf09c0`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28745: [SPARK-31915][SQL][PYTHON] Remove projection that adds grouping keys in grouped and cogrouped pandas UDFs

2020-06-09 Thread GitBox
SparkQA commented on pull request #28745: URL: https://github.com/apache/spark/pull/28745#issuecomment-641499511 **[Test build #123698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123698/testReport)** for PR 28745 at commit [`2bf09c0`](https://github.co

[GitHub] [spark] sathyaprakashg commented on pull request #28703: SPARK-29897 Add implicit cast for SubtractTimestamps

2020-06-09 Thread GitBox
sathyaprakashg commented on pull request #28703: URL: https://github.com/apache/spark/pull/28703#issuecomment-641483924 Thanks @bart-samwel I closed the PR. Could you please help me close this ticket. I don't have access to close it . https://issues.apache.org/jira/browse/SPARK-29897 ---

[GitHub] [spark] sathyaprakashg closed pull request #28703: SPARK-29897 Add implicit cast for SubtractTimestamps

2020-06-09 Thread GitBox
sathyaprakashg closed pull request #28703: URL: https://github.com/apache/spark/pull/28703 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641475758 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641475758 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
SparkQA commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-641475152 **[Test build #123702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123702/testReport)** for PR 28412 at commit [`3494307`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641472156 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641472156 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

2020-06-09 Thread GitBox
SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641471490 **[Test build #123701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123701/testReport)** for PR 28593 at commit [`1d5c750`](https://github.com

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437608213 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437602431 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437600535 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1167,6 +1172,17 @@ private[history] class FsHistoryP

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437598233 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1197,6 +1213,78 @@ private[history] class FsHistoryP

[GitHub] [spark] gerashegalov commented on a change in pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

2020-06-09 Thread GitBox
gerashegalov commented on a change in pull request #28746: URL: https://github.com/apache/spark/pull/28746#discussion_r437592878 ## File path: core/src/main/scala/org/apache/spark/deploy/LocalSparkCluster.scala ## @@ -74,6 +74,10 @@ class LocalSparkCluster( def stop(): Uni

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641454235 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641454235 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641245723 **[Test build #123694 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123694/testReport)** for PR 28764 at commit [`668f419`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

2020-06-09 Thread GitBox
SparkQA commented on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641453142 **[Test build #123694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123694/testReport)** for PR 28764 at commit [`668f419`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-641450746 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-641450746 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-641239491 **[Test build #123693 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123693/testReport)** for PR 28645 at commit [`2527c69`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28645: [SPARK-31826][SQL] Support composed type of case class for typed Scala UDF

2020-06-09 Thread GitBox
SparkQA commented on pull request #28645: URL: https://github.com/apache/spark/pull/28645#issuecomment-641449469 **[Test build #123693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123693/testReport)** for PR 28645 at commit [`2527c69`](https://github.co

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437581083 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HistoryServerMemoryManager.scala ## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437580626 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437580889 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437580420 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] cloud-fan closed pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

2020-06-09 Thread GitBox
cloud-fan closed pull request #28650: URL: https://github.com/apache/spark/pull/28650 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

2020-06-09 Thread GitBox
cloud-fan commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641443909 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437578109 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437578179 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

2020-06-09 Thread GitBox
baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437577822 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] cloud-fan commented on pull request #28751: [SPARK-31926][SQL][test-hive1.2] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-09 Thread GitBox
cloud-fan commented on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-641435280 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] cloud-fan closed pull request #28751: [SPARK-31926][SQL][test-hive1.2] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-09 Thread GitBox
cloud-fan closed pull request #28751: URL: https://github.com/apache/spark/pull/28751 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] SaurabhChawla100 commented on a change in pull request #28768: [SPARK-31941][CORE]Add the code change to handle the SparkException

2020-06-09 Thread GitBox
SaurabhChawla100 commented on a change in pull request #28768: URL: https://github.com/apache/spark/pull/28768#discussion_r437290253 ## File path: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ## @@ -86,6 +86,7 @@ private[spark] class SparkUI private ( .orElse

[GitHub] [spark] cloud-fan commented on pull request #28745: [SPARK-31915][SQL][PYTHON] Remove projection that adds grouping keys in grouped and cogrouped pandas UDFs

2020-06-09 Thread GitBox
cloud-fan commented on pull request #28745: URL: https://github.com/apache/spark/pull/28745#issuecomment-641432882 will this cause perf regression? e.g. if grouping expr is expensive, with the Project we only need to evaluate it once. --

[GitHub] [spark] cloud-fan closed pull request #27986: [SPARK-31220][SQL] repartition obeys initialPartitionNum when adaptiveExecutionEnabled

2020-06-09 Thread GitBox
cloud-fan closed pull request #27986: URL: https://github.com/apache/spark/pull/27986 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan closed pull request #28765: [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map parsed from partition path should be case insensitive

2020-06-09 Thread GitBox
cloud-fan closed pull request #28765: URL: https://github.com/apache/spark/pull/28765 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #27986: [SPARK-31220][SQL] repartition obeys initialPartitionNum when adaptiveExecutionEnabled

2020-06-09 Thread GitBox
cloud-fan commented on pull request #27986: URL: https://github.com/apache/spark/pull/27986#issuecomment-641404167 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] SparkQA commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

2020-06-09 Thread GitBox
SparkQA commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-641405101 **[Test build #123699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123699/testReport)** for PR 27246 at commit [`e48a936`](https://github.co

[GitHub] [spark] sarutak commented on a change in pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

2020-06-09 Thread GitBox
sarutak commented on a change in pull request #28768: URL: https://github.com/apache/spark/pull/28768#discussion_r437537787 ## File path: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ## @@ -86,7 +86,7 @@ private[spark] class SparkUI private ( .orElse(store.en

[GitHub] [spark] AmplabJenkins commented on pull request #28751: [SPARK-31926][SQL][test-hive1.2] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-641405232 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28751: [SPARK-31926][SQL][test-hive1.2] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-641405232 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #27983: [SPARK-31936][SQL]Implement ScriptTransformation in sql/core

2020-06-09 Thread GitBox
AngersZh commented on a change in pull request #27983: URL: https://github.com/apache/spark/pull/27983#discussion_r437558157 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/script/ScriptTransformationExec.scala ## @@ -0,0 +1,239 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA commented on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

2020-06-09 Thread GitBox
SparkQA commented on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641406226 **[Test build #123700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123700/testReport)** for PR 28768 at commit [`e8769fb`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-641405643 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun closed pull request #28742: [SPARK-31921][CORE] Fix the wrong warning: "App app-xxx requires more resource than any of Workers could have"

2020-06-09 Thread GitBox
dongjoon-hyun closed pull request #28742: URL: https://github.com/apache/spark/pull/28742 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641414274 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] cloud-fan commented on pull request #28765: [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map parsed from partition path should be case insensitive

2020-06-09 Thread GitBox
cloud-fan commented on pull request #28765: URL: https://github.com/apache/spark/pull/28765#issuecomment-641398316 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] SparkQA removed a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641202467 **[Test build #123689 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123689/testReport)** for PR 28650 at commit [`8139dfc`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641414274 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

2020-06-09 Thread GitBox
SparkQA removed a comment on pull request #27246: URL: https://github.com/apache/spark/pull/27246#issuecomment-641312310 **[Test build #123699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123699/testReport)** for PR 27246 at commit [`e48a936`](https://gi

[GitHub] [spark] yaooqinn edited a comment on pull request #28751: [SPARK-31926][SQL] Fix concurrency issue for ThriftCLIService to getPortNumber

2020-06-09 Thread GitBox
yaooqinn edited a comment on pull request #28751: URL: https://github.com/apache/spark/pull/28751#issuecomment-640457288 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28616: [SPARK-31798][SHUFFLE][API] Shuffle Writer API changes to return custom map output metadata

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28616: URL: https://github.com/apache/spark/pull/28616#issuecomment-640957576 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] yaooqinn commented on a change in pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

2020-06-09 Thread GitBox
yaooqinn commented on a change in pull request #28650: URL: https://github.com/apache/spark/pull/28650#discussion_r437232828 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -56,6 +54,26 @@ trait TimeZoneAwareE

[GitHub] [spark] SparkQA commented on pull request #28650: [SPARK-31830][SQL] Consistent error handling for datetime formatting and parsing functions

2020-06-09 Thread GitBox
SparkQA commented on pull request #28650: URL: https://github.com/apache/spark/pull/28650#issuecomment-641412878 **[Test build #123689 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123689/testReport)** for PR 28650 at commit [`8139dfc`](https://github.co

[GitHub] [spark] jkpl closed pull request #28753: [SPARK-31927][K8S] Bump Fabric8 to 4.10.2 to support k8s 1.18

2020-06-09 Thread GitBox
jkpl closed pull request #28753: URL: https://github.com/apache/spark/pull/28753 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] srowen commented on pull request #27805: [SPARK-31056][SQL] Add CalendarIntervals division

2020-06-09 Thread GitBox
srowen commented on pull request #27805: URL: https://github.com/apache/spark/pull/27805#issuecomment-640611674 Right, "24 hours" is too simplistic an example. I get that "1 year 2 months" is just months. I think the arguments still stand: non-standard, limited scope, and achievable with e

[GitHub] [spark] wangyum commented on pull request #28734: [SPARK-31912][SQL][TESTS] Normalize all binary comparison expressions

2020-06-09 Thread GitBox
wangyum commented on pull request #28734: URL: https://github.com/apache/spark/pull/28734#issuecomment-641355129 I encountered this issue when inferring from inequality attributes before, Its order may change: https://github.com/apache/spark/blob/bfa60393789673b1cd77fbda038cdc8c17c1a866

[GitHub] [spark] tgravescs commented on pull request #27636: [SPARK-30873][CORE][YARN]Handling Node Decommissioning for Yarn cluster manger in Spark

2020-06-09 Thread GitBox
tgravescs commented on pull request #27636: URL: https://github.com/apache/spark/pull/27636#issuecomment-640883318 sorry I haven't had time to look at this more. It will not go into spark 3.0 as that is very close to shipping. --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28743: [SPARK-31920][PYTHON][WIP] Fix pandas conversion using Arrow with __arrow_array__ columns

2020-06-09 Thread GitBox
AmplabJenkins removed a comment on pull request #28743: URL: https://github.com/apache/spark/pull/28743#issuecomment-640721541 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan closed pull request #28763: [SPARK-28199][SS][FOLLOWUP] Mention the change of SPARK-28199 into the SS migration guide

2020-06-09 Thread GitBox
cloud-fan closed pull request #28763: URL: https://github.com/apache/spark/pull/28763 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] AmplabJenkins commented on pull request #28759: [SPARK-31934][BUILD] Remove set -x from docker image tool

2020-06-09 Thread GitBox
AmplabJenkins commented on pull request #28759: URL: https://github.com/apache/spark/pull/28759#issuecomment-640931144 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

<    1   2   3   4   5   6   7   8   >