[GitHub] [spark] AmplabJenkins commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641633573 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins removed a comment on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641633573 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437789267 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1167,6 +1172,17 @@ private[history] class

[GitHub] [spark] github-actions[bot] commented on pull request #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

github-actions[bot] commented on pull request #16478: URL: https://github.com/apache/spark/pull/16478#issuecomment-641647774 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] SparkQA commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

SparkQA commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641614935 **[Test build #123709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123709/testReport)** for PR 28772 at commit

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641614950 **[Test build #123708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123708/testReport)** for PR 28708 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641532764 **[Test build #123704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123704/testReport)** for PR 28708 at commit

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641623473 **[Test build #123704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123704/testReport)** for PR 28708 at commit

[GitHub] [spark] SparkQA commented on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

SparkQA commented on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641625470 **[Test build #123703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123703/testReport)** for PR 28771 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

SparkQA removed a comment on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641532792 **[Test build #123703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123703/testReport)** for PR 28771 at commit

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641627813 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28333/

[GitHub] [spark] SparkQA commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

SparkQA commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641635256 **[Test build #123709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123709/testReport)** for PR 28772 at commit

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437784544 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1197,6 +1213,78 @@ private[history] class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437784544 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1197,6 +1213,78 @@ private[history] class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641650249 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] srowen commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

srowen commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641650646 We can implement AutoCloseable instead (superinterface of Closeable) but I don't think that by itself helps if there's some code path that leaks. It's a bit of a band-aid but

[GitHub] [spark] agrawaldevesh commented on a change in pull request #27636: [SPARK-30873][CORE][YARN]Handling Node Decommissioning for Yarn cluster manger in Spark

agrawaldevesh commented on a change in pull request #27636: URL: https://github.com/apache/spark/pull/27636#discussion_r437795234 ## File path: core/src/main/scala/org/apache/spark/scheduler/DecommissionTracker.scala ## @@ -0,0 +1,405 @@ +/* + * Licensed to the Apache

[GitHub] [spark] maropu opened a new pull request #28773: [SPARK-26905][SQL] Add `TYPE` in the ANSI non-reserved list

maropu opened a new pull request #28773: URL: https://github.com/apache/spark/pull/28773 ### What changes were proposed in this pull request? This PR intends to add `TYPE` in the ANSI non-reserved list because it is not reserved in the standard. See SPARK-26905 for a full

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641650246 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] HeartSaVioR commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641655790 In many spots in AppStatusStore, they call `view` but never explicitly call `close`. The implementation is focused on being concise, not concerning about the possible

[GitHub] [spark] AmplabJenkins commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641662756 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

SparkQA commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641662475 **[Test build #123713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123713/testReport)** for PR 28704 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AmplabJenkins commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641612877 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641612468 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AmplabJenkins removed a comment on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641612877 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641624125 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641624125 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28773: [SPARK-26905][SQL] Add `TYPE` in the ANSI non-reserved list

SparkQA commented on pull request #28773: URL: https://github.com/apache/spark/pull/28773#issuecomment-641651098 **[Test build #123711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123711/testReport)** for PR 28773 at commit

[GitHub] [spark] jzhuge commented on pull request #28147: [SPARK-31357][SQL][WIP] Catalog API for view metadata

jzhuge commented on pull request #28147: URL: https://github.com/apache/spark/pull/28147#issuecomment-641603964 @cloud-fan @dongjoon-hyun @brkyvz Could you comment on this WIP PR? Go or no-go? Need more clarifications? Even though it is labeled as WIP, the code is pretty stable and in

[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641611980 **[Test build #123701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123701/testReport)** for PR 28593 at commit

[GitHub] [spark] SparkQA commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

SparkQA commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641622318 **[Test build #123710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123710/testReport)** for PR 28733 at commit

[GitHub] [spark] zhli1142015 commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641626275 @squito , @srowen , @cloud-fan , could you please help review? This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

AmplabJenkins removed a comment on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641625917 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28771: [2.4][SPARK-31935][SQL] Hadoop file system config should be effective in data source options

AmplabJenkins commented on pull request #28771: URL: https://github.com/apache/spark/pull/28771#issuecomment-641625917 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

SparkQA removed a comment on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641609954 **[Test build #123706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123706/testReport)** for PR 28772 at commit

[GitHub] [spark] SparkQA commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

SparkQA commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641633244 **[Test build #123706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123706/testReport)** for PR 28772 at commit

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437784544 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1197,6 +1213,78 @@ private[history] class

[GitHub] [spark] zhli1142015 commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641658022 > In many spots in AppStatusStore, they call `view` but never explicitly call `close`. The implementation is focused on being concise, not concerning about the possible

[GitHub] [spark] gengliangwang commented on a change in pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

gengliangwang commented on a change in pull request #28733: URL: https://github.com/apache/spark/pull/28733#discussion_r437753240 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala ## @@ -198,6 +199,88 @@ trait PredicateHelper

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins removed a comment on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641610426 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641615414 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins removed a comment on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641615414 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AmplabJenkins commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641620349 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AmplabJenkins removed a comment on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641620349 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641634288 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641634267 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28333/

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641634288 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] zhli1142015 commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641652164 > I think this 'works', though I do wonder how many iterators are open at once. It can't be like thousands, right? > > I guess we don't otherwise have a guess about

[GitHub] [spark] SparkQA removed a comment on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

SparkQA removed a comment on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641656471 **[Test build #123712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123712/testReport)** for PR 28704 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins removed a comment on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641656994 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641656994 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

SparkQA commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641656985 **[Test build #123712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123712/testReport)** for PR 28704 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins removed a comment on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641662756 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

SparkQA commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641663419 **[Test build #123705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123705/testReport)** for PR 28733 at commit

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437811279 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1197,6 +1213,78 @@ private[history] class

[GitHub] [spark] huaxingao opened a new pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

huaxingao opened a new pull request #28772: URL: https://github.com/apache/spark/pull/28772 ### What changes were proposed in this pull request? Add instance weight support in LinearRegressionSummary ### Why are the changes needed? LinearRegression and RegressionMetrics

[GitHub] [spark] SparkQA commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

SparkQA commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641612499 **[Test build #123707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123707/testReport)** for PR 28733 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641612459 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641612459 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-641471490 **[Test build #123701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123701/testReport)** for PR 28593 at commit

[GitHub] [spark] redsanket commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

redsanket commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437780958 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] maropu commented on pull request #28764: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH_BUCKET

maropu commented on pull request #28764: URL: https://github.com/apache/spark/pull/28764#issuecomment-641636444 Could you review this? @HyukjinKwon @viirya @wangyum @yaooqinn This is an automated message from the Apache Git

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437786292 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28773: [SPARK-26905][SQL] Add `TYPE` in the ANSI non-reserved list

AmplabJenkins removed a comment on pull request #28773: URL: https://github.com/apache/spark/pull/28773#issuecomment-641651407 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28773: [SPARK-26905][SQL] Add `TYPE` in the ANSI non-reserved list

AmplabJenkins commented on pull request #28773: URL: https://github.com/apache/spark/pull/28773#issuecomment-641651407 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins removed a comment on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641656999 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641610426 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

SparkQA commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641609954 **[Test build #123706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123706/testReport)** for PR 28772 at commit

[GitHub] [spark] gengliangwang commented on a change in pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

gengliangwang commented on a change in pull request #28733: URL: https://github.com/apache/spark/pull/28733#discussion_r437753240 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala ## @@ -198,6 +199,88 @@ trait PredicateHelper

[GitHub] [spark] AmplabJenkins commented on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins commented on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641635578 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

AmplabJenkins removed a comment on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641635578 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28772: [SPARK-31944] Add instance weight support in LinearRegressionSummary

SparkQA removed a comment on pull request #28772: URL: https://github.com/apache/spark/pull/28772#issuecomment-641614935 **[Test build #123709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123709/testReport)** for PR 28772 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #28768: [SPARK-31941][CORE][WEBUI] Handle SparkException in SparkUI

HeartSaVioR commented on pull request #28768: URL: https://github.com/apache/spark/pull/28768#issuecomment-641644829 The change looks good - we may need to reflect the "actual change" into PR title and description, as it's no longer same as initial proposal.

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641649996 **[Test build #123708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123708/testReport)** for PR 28708 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641650246 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-641614950 **[Test build #123708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123708/testReport)** for PR 28708 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641649819 I guess the interface matters - in-memory KV store doesn't need to have close in its iterator of course, but level DB KV store should. The code picked "Iterator" as common

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR edited a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641649819 I guess the interface matters - in-memory KV store doesn't need to have close in its iterator of course (it has, but no-op), but level DB KV store should. The code

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AmplabJenkins removed a comment on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641664046 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] shaneknapp commented on pull request #28627: [SPARK-31756][WEBUI] Add real headless browser support for UI test

shaneknapp commented on pull request #28627: URL: https://github.com/apache/spark/pull/28627#issuecomment-641652740 sorry, i don't watch github for @mentions... if this is still a problem, file a build/infra jira and assign it to me. :) On Thu, May 28, 2020 at 11:00 PM Kousuke

[GitHub] [spark] HeartSaVioR commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641652906 Oh I was pointing out innocent one. `KVStoreView` is the culprit, though the class doc describes the warning.

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR edited a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641652906 Oh I was pointing out innocent one. `KVStoreView` is the culprit, though the class doc describes the warning.

[GitHub] [spark] SparkQA commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

SparkQA commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641656471 **[Test build #123712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123712/testReport)** for PR 28704 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

SparkQA removed a comment on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641549973 **[Test build #123705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123705/testReport)** for PR 28733 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28733: [SPARK-31705][SQL] Push more possible predicates through Join via CNF conversion

AmplabJenkins commented on pull request #28733: URL: https://github.com/apache/spark/pull/28733#issuecomment-641664046 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] redsanket commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

redsanket commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437780461 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1167,6 +1172,17 @@ private[history] class

[GitHub] [spark] srowen commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

srowen commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641636134 I think this 'works', though I do wonder how many iterators are open at once. It can't be like thousands, right? I guess we don't otherwise have a guess about what is

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437788564 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

HeartSaVioR commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r437788564 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,185 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins removed a comment on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641655003 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] zhli1142015 commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641655009 if we see description from below comments, i think even caller calls close, it may still have leaking issue.

[GitHub] [spark] AmplabJenkins commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641655003 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] srowen commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

srowen commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641655314 I'm not sure, I don't see any usages of it that don't close the iterator in a finally block. I do wonder where these are allocated, if there is a straightforward fix to the

[GitHub] [spark] viirya edited a comment on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

viirya edited a comment on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641655170 Added fold number check and test case both in Scala and Python. This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

AmplabJenkins removed a comment on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641655007 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] viirya commented on pull request #28704: [SPARK-31777][ML][PySpark] Add user-specified fold column to CrossValidator

viirya commented on pull request #28704: URL: https://github.com/apache/spark/pull/28704#issuecomment-641655170 Added fold number check both in Scala and Python. This is an automated message from the Apache Git Service. To

[GitHub] [spark] HeartSaVioR commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641668567 IMHO, KVStoreView shouldn't implement Iterable directly - this leads callers to simply call `iterator` or wrap with asScala and completely forget about the resource close.

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR edited a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-641668567 IMHO, KVStoreView shouldn't implement `Iterable` directly - this leads callers to simply call `iterator` or wrap with `asScala` and completely forget about the

[GitHub] [spark] HyukjinKwon commented on pull request #28745: [SPARK-31915][SQL][PYTHON] Remove projection that adds grouping keys in grouped and cogrouped pandas UDFs

HyukjinKwon commented on pull request #28745: URL: https://github.com/apache/spark/pull/28745#issuecomment-641674305 > will this cause perf regression? e.g. if grouping expr is expensive, with the Project we only need to evaluate it once. I would say this is kind of a design choice.

[GitHub] [spark] AmplabJenkins commented on pull request #28774: [SPARK-31945][SQL][PYSPARK] Enable cache for the same Python function.

AmplabJenkins commented on pull request #28774: URL: https://github.com/apache/spark/pull/28774#issuecomment-641674275 This is an automated message from the Apache Git Service. To respond to the message, please log on to

  1   2   3   4   5   6   7   >