[GitHub] [spark] zero323 commented on a change in pull request #29639: [SPARK-32186][DOCS][PYTHON] Development - Debugging

2020-09-03 Thread GitBox
zero323 commented on a change in pull request #29639: URL: https://github.com/apache/spark/pull/29639#discussion_r483402746 ## File path: python/docs/source/development/debugging.rst ## @@ -0,0 +1,187 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686925870 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686925870 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
SparkQA commented on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686925219 **[Test build #128277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128277/testReport)** for PR 29634 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29640: URL: https://github.com/apache/spark/pull/29640#discussion_r483400388 ## File path: python/docs/source/getting_started/installation.rst ## @@ -0,0 +1,119 @@ +.. Licensed to the Apache Software Foundation (ASF) under

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29634: URL: https://github.com/apache/spark/pull/29634#discussion_r483399207 ## File path: python/docs/source/development/testing.rst ## @@ -0,0 +1,61 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or

[GitHub] [spark] cloud-fan commented on a change in pull request #29593: [SPARK-32753][SQL] Only copy tags to node with no tags

2020-09-03 Thread GitBox
cloud-fan commented on a change in pull request #29593: URL: https://github.com/apache/spark/pull/29593#discussion_r483396414 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala ## @@ -91,7 +91,9 @@ abstract class TreeNode[BaseType <:

[GitHub] [spark] maropu commented on a change in pull request #29643: [SPARK-32638][SQL][FOLLOWUP] Move the plan rewriting methods to QueryPlan

2020-09-03 Thread GitBox
maropu commented on a change in pull request #29643: URL: https://github.com/apache/spark/pull/29643#discussion_r483396030 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala ## @@ -168,6 +170,85 @@ abstract class QueryPlan[PlanType

[GitHub] [spark] maropu commented on a change in pull request #29643: [SPARK-32638][SQL][FOLLOWUP] Move the plan rewriting methods to QueryPlan

2020-09-03 Thread GitBox
maropu commented on a change in pull request #29643: URL: https://github.com/apache/spark/pull/29643#discussion_r483396030 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala ## @@ -168,6 +170,85 @@ abstract class QueryPlan[PlanType

[GitHub] [spark] jainshashank24 removed a comment on pull request #28828: [SPARK-24634][SS][FOLLOWUP] Rename the variable from "numLateInputs" to "numRowsDroppedByWatermark"

2020-09-03 Thread GitBox
jainshashank24 removed a comment on pull request #28828: URL: https://github.com/apache/spark/pull/28828#issuecomment-686435584 Hi i have used this MR and the other one https://github.com/apache/spark/pull/28607/files even after that i cant see counter value increasing Though i can

[GitHub] [spark] cloud-fan commented on a change in pull request #29643: [SPARK-32638][SQL][FOLLOWUP] Move the plan rewriting methods to QueryPlan

2020-09-03 Thread GitBox
cloud-fan commented on a change in pull request #29643: URL: https://github.com/apache/spark/pull/29643#discussion_r483394790 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala ## @@ -168,6 +170,85 @@ abstract class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29593: [SPARK-32753][SQL] Only copy tags to node with no tags

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29593: URL: https://github.com/apache/spark/pull/29593#issuecomment-686909589 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29593: [SPARK-32753][SQL] Only copy tags to node with no tags

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29593: URL: https://github.com/apache/spark/pull/29593#issuecomment-686909589 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29593: [SPARK-32753][SQL] Only copy tags to node with no tags

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #29593: URL: https://github.com/apache/spark/pull/29593#issuecomment-686823149 **[Test build #128270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128270/testReport)** for PR 29593 at commit

[GitHub] [spark] SparkQA commented on pull request #29593: [SPARK-32753][SQL] Only copy tags to node with no tags

2020-09-03 Thread GitBox
SparkQA commented on pull request #29593: URL: https://github.com/apache/spark/pull/29593#issuecomment-686908644 **[Test build #128270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128270/testReport)** for PR 29593 at commit

[GitHub] [spark] maropu commented on a change in pull request #29643: [SPARK-32638][SQL][FOLLOWUP] Move the plan rewriting methods to QueryPlan

2020-09-03 Thread GitBox
maropu commented on a change in pull request #29643: URL: https://github.com/apache/spark/pull/29643#discussion_r483387646 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala ## @@ -168,6 +170,85 @@ abstract class QueryPlan[PlanType

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686906378 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686906378 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686906035 **[Test build #128275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128275/testReport)** for PR 29639 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686895732 **[Test build #128275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128275/testReport)** for PR 29639 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

2020-09-03 Thread GitBox
AngersZh commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-686902515 @HyukjinKwon as metioned in https://github.com/apache/spark/pull/29087#discussion_r454101882. can you also help review that pr

[GitHub] [spark] LuciferYang edited a comment on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang edited a comment on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686862866 > @LuciferYang are you sure? tests passed for that PR. Build jobs seem fine. https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/ What are

[GitHub] [spark] maropu commented on a change in pull request #29643: [SPARK-32638][SQL][FOLLOWUP] Move the plan rewriting methods to QueryPlan

2020-09-03 Thread GitBox
maropu commented on a change in pull request #29643: URL: https://github.com/apache/spark/pull/29643#discussion_r483382255 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -123,127 +123,6 @@ object AnalysisContext { } }

[GitHub] [spark] LuciferYang edited a comment on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang edited a comment on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686890566 > Hmm, why this is needed? Firstly I thought CostBasedJoinReorder will produce non-deterministic for same query. But I looked at the JIRA description, seems for

[GitHub] [spark] LuciferYang commented on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang commented on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686899631 @viirya I'm also entangled in this issue :( This is an automated message from the Apache Git Service. To

[GitHub] [spark] LuciferYang edited a comment on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang edited a comment on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686890566 > Hmm, why this is needed? Firstly I thought CostBasedJoinReorder will produce non-deterministic for same query. But I looked at the JIRA description, seems for

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-686898464 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] LuciferYang edited a comment on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang edited a comment on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686890566 > Hmm, why this is needed? Firstly I thought CostBasedJoinReorder will produce non-deterministic for same query. But I looked at the JIRA description, seems for

[GitHub] [spark] AmplabJenkins commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-686898464 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

2020-09-03 Thread GitBox
SparkQA commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-686897991 **[Test build #128276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128276/testReport)** for PR 29087 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

2020-09-03 Thread GitBox
HyukjinKwon commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-686896927 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686896591 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686896591 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686888751 **[Test build #128274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128274/testReport)** for PR 29634 at commit

[GitHub] [spark] SparkQA commented on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
SparkQA commented on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686896263 **[Test build #128274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128274/testReport)** for PR 29634 at commit

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686895732 **[Test build #128275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128275/testReport)** for PR 29639 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686893915 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686893915 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686886945 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686886945 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28269: [SPARK-31493][SQL] Optimize InSet to In according partition size at InSubqueryExec

2020-09-03 Thread GitBox
SparkQA commented on pull request #28269: URL: https://github.com/apache/spark/pull/28269#issuecomment-686887154 **[Test build #128272 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128272/testReport)** for PR 28269 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28269: [SPARK-31493][SQL] Optimize InSet to In according partition size at InSubqueryExec

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #28269: URL: https://github.com/apache/spark/pull/28269#issuecomment-686887480 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29634: URL: https://github.com/apache/spark/pull/29634#discussion_r483369994 ## File path: python/docs/source/development/testing.rst ## @@ -0,0 +1,61 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or

[GitHub] [spark] SparkQA removed a comment on pull request #28269: [SPARK-31493][SQL] Optimize InSet to In according partition size at InSubqueryExec

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #28269: URL: https://github.com/apache/spark/pull/28269#issuecomment-686835937 **[Test build #128272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128272/testReport)** for PR 28269 at commit

[GitHub] [spark] LuciferYang commented on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang commented on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686890566 > Hmm, why this is needed? Firstly I thought CostBasedJoinReorder will produce non-deterministic for same query. But I looked at the JIRA description, seems for different

[GitHub] [spark] HyukjinKwon commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
HyukjinKwon commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686890672 Thanks @srowen and @viirya. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
SparkQA commented on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686888751 **[Test build #128274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128274/testReport)** for PR 29634 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28269: [SPARK-31493][SQL] Optimize InSet to In according partition size at InSubqueryExec

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #28269: URL: https://github.com/apache/spark/pull/28269#issuecomment-686887489 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28269: [SPARK-31493][SQL] Optimize InSet to In according partition size at InSubqueryExec

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #28269: URL: https://github.com/apache/spark/pull/28269#issuecomment-686887480 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686889107 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29634: [SPARK-32783][DOCS][PYTHON] Development - Testing PySpark

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29634: URL: https://github.com/apache/spark/pull/29634#issuecomment-686889107 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686812047 **[Test build #128269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128269/testReport)** for PR 29364 at commit

[GitHub] [spark] SparkQA commented on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
SparkQA commented on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686886390 **[Test build #128269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128269/testReport)** for PR 29364 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686884086 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686884077 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] viirya commented on a change in pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
viirya commented on a change in pull request #29639: URL: https://github.com/apache/spark/pull/29639#discussion_r483366839 ## File path: python/docs/source/development/debugging.rst ## @@ -0,0 +1,187 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686884077 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686874465 **[Test build #128273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128273/testReport)** for PR 29639 at commit

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686883777 **[Test build #128273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128273/testReport)** for PR 29639 at commit

[GitHub] [spark] KevinSmile removed a comment on pull request #29644: [SPARK-32598][Scheduler] fix missing driver logs in UI Executors tab in standalone mode

2020-09-03 Thread GitBox
KevinSmile removed a comment on pull request #29644: URL: https://github.com/apache/spark/pull/29644#issuecomment-686875689 Direct bug reason: the original author forgot to implement `getDriverLogUrls` in `StandaloneSchedulerBackend`

[GitHub] [spark] KevinSmile edited a comment on pull request #29644: [SPARK-32598][Scheduler] fix missing driver logs in UI Executors tab in standalone mode

2020-09-03 Thread GitBox
KevinSmile edited a comment on pull request #29644: URL: https://github.com/apache/spark/pull/29644#issuecomment-686875689 Direct bug reason: the original author forgot to implement `getDriverLogUrls` in `StandaloneSchedulerBackend`

[GitHub] [spark] KevinSmile commented on pull request #29644: [SPARK-32598][Scheduler] fix missing driver logs in UI Executors tab in standalone mode

2020-09-03 Thread GitBox
KevinSmile commented on pull request #29644: URL: https://github.com/apache/spark/pull/29644#issuecomment-686875689 Direct bug reason: the original author forgot to implement `getDriverLogUrls` in `StandaloneSchedulerBackend`

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686874895 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29644: [SPARK-32598][Scheduler] fix missing driver logs in UI Executors tab in standalone mode

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29644: URL: https://github.com/apache/spark/pull/29644#issuecomment-686874349 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686874895 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29644: [SPARK-32598][Scheduler] fix missing driver logs in UI Executors tab in standalone mode

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29644: URL: https://github.com/apache/spark/pull/29644#issuecomment-686874710 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29639: [SPARK-32186][DOCS][PYTHON] User Guide - Debugging

2020-09-03 Thread GitBox
SparkQA commented on pull request #29639: URL: https://github.com/apache/spark/pull/29639#issuecomment-686874465 **[Test build #128273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128273/testReport)** for PR 29639 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29644: [SPARK-32598][Scheduler] fix missing driver logs in UI Executors tab in standalone mode

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29644: URL: https://github.com/apache/spark/pull/29644#issuecomment-686874349 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] KevinSmile opened a new pull request #29644: [SPARK-32598][Scheduler] fix missing driver logs in UI Executors tab in standalone mode

2020-09-03 Thread GitBox
KevinSmile opened a new pull request #29644: URL: https://github.com/apache/spark/pull/29644 ### What changes were proposed in this pull request? fix [SPARK-32598] (missing driver logs in UI Executors tab in standalone mode) by solving 2 more general problems: 1. Currently, a

[GitHub] [spark] HeartSaVioR edited a comment on pull request #29630: [SPARK-32097] Enable Spark History Server to read from multiple directories

2020-09-03 Thread GitBox
HeartSaVioR edited a comment on pull request #29630: URL: https://github.com/apache/spark/pull/29630#issuecomment-686870943 I don't think Spark has the concept of "clusters". Even I don't think Spark has the concept of "cluster", unless you use standalone mode. More specifically, there's

[GitHub] [spark] HeartSaVioR edited a comment on pull request #29630: [SPARK-32097] Enable Spark History Server to read from multiple directories

2020-09-03 Thread GitBox
HeartSaVioR edited a comment on pull request #29630: URL: https://github.com/apache/spark/pull/29630#issuecomment-686870943 I don't think Spark has the concept of "clusters". Even I don't think Spark has the concept of "cluster", unless you use standalone mode. More specifically, there's

[GitHub] [spark] HeartSaVioR edited a comment on pull request #29630: [SPARK-32097] Enable Spark History Server to read from multiple directories

2020-09-03 Thread GitBox
HeartSaVioR edited a comment on pull request #29630: URL: https://github.com/apache/spark/pull/29630#issuecomment-686870943 I don't think Spark has the concept of "clusters". Even I don't think Spark has the concept of "cluster", unless you use standalone mode. More specifically, there's

[GitHub] [spark] HeartSaVioR edited a comment on pull request #29630: [SPARK-32097] Enable Spark History Server to read from multiple directories

2020-09-03 Thread GitBox
HeartSaVioR edited a comment on pull request #29630: URL: https://github.com/apache/spark/pull/29630#issuecomment-686870943 I don't think Spark has the concept of "clusters". Even I don't think Spark has the concept of "cluster", unless you use standalone mode. More specifically, there's

[GitHub] [spark] HeartSaVioR commented on pull request #29630: [SPARK-32097] Enable Spark History Server to read from multiple directories

2020-09-03 Thread GitBox
HeartSaVioR commented on pull request #29630: URL: https://github.com/apache/spark/pull/29630#issuecomment-686870943 I don't think Spark has the concept of "clusters". Even I don't think Spark has the concept of "cluster", unless you use standalone mode. If the rationalization of

[GitHub] [spark] LuciferYang edited a comment on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang edited a comment on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686862866 > @LuciferYang are you sure? tests passed for that PR. Build jobs seem fine. https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/ What are

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686864065 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686864065 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686786051 **[Test build #128268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128268/testReport)** for PR 29364 at commit

[GitHub] [spark] SparkQA commented on pull request #29364: [SPARK-32548][SQL] - Add Application attemptId support to SQL Rest API

2020-09-03 Thread GitBox
SparkQA commented on pull request #29364: URL: https://github.com/apache/spark/pull/29364#issuecomment-686863480 **[Test build #128268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128268/testReport)** for PR 29364 at commit

[GitHub] [spark] LuciferYang commented on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang commented on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686862866 > @LuciferYang are you sure? tests passed for that PR. Build jobs seem fine. https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/ What are you

[GitHub] [spark] LuciferYang edited a comment on pull request #29638: [SPARK-32687][SQL] Let CostBasedJoinReorder produce relatively deterministic optimization result

2020-09-03 Thread GitBox
LuciferYang edited a comment on pull request #29638: URL: https://github.com/apache/spark/pull/29638#issuecomment-686862866 > @LuciferYang are you sure? tests passed for that PR. Build jobs seem fine. https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/ What are

[GitHub] [spark] wangyum commented on pull request #29641: [SPARK-32791][SQL] Non-partitioned table metric should not have dynamic partition pruning time

2020-09-03 Thread GitBox
wangyum commented on pull request #29641: URL: https://github.com/apache/spark/pull/29641#issuecomment-686859697 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] srowen commented on a change in pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
srowen commented on a change in pull request #29640: URL: https://github.com/apache/spark/pull/29640#discussion_r483338273 ## File path: python/docs/source/getting_started/index.rst ## @@ -20,7 +20,10 @@ Getting Started === +This page lists an overview of the

[GitHub] [spark] HyukjinKwon commented on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
HyukjinKwon commented on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686848371 Looks fine otherwise. @holdenk and @srowen can you take a look when you guys are available? This is an

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29640: URL: https://github.com/apache/spark/pull/29640#discussion_r483337245 ## File path: python/docs/source/getting_started/installation.rst ## @@ -0,0 +1,119 @@ +.. Licensed to the Apache Software Foundation (ASF) under

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29640: URL: https://github.com/apache/spark/pull/29640#discussion_r483337127 ## File path: python/docs/source/getting_started/installation.rst ## @@ -0,0 +1,119 @@ +.. Licensed to the Apache Software Foundation (ASF) under

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29640: URL: https://github.com/apache/spark/pull/29640#discussion_r483336831 ## File path: python/docs/source/getting_started/installation.rst ## @@ -0,0 +1,119 @@ +.. Licensed to the Apache Software Foundation (ASF) under

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29640: URL: https://github.com/apache/spark/pull/29640#discussion_r483336672 ## File path: python/docs/source/getting_started/index.rst ## @@ -20,7 +20,10 @@ Getting Started === +This page lists an overview of

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29640: URL: https://github.com/apache/spark/pull/29640#discussion_r483336906 ## File path: python/docs/source/getting_started/installation.rst ## @@ -0,0 +1,119 @@ +.. Licensed to the Apache Software Foundation (ASF) under

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29642: [SPARK-32792][SQL] Improve in filter pushdown for ParquetFilters

2020-09-03 Thread GitBox
HyukjinKwon commented on a change in pull request #29642: URL: https://github.com/apache/spark/pull/29642#discussion_r483336491 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala ## @@ -597,9 +597,9 @@ class

[GitHub] [spark] leanken commented on pull request #29614: [SPARK-32765][SQL] EliminateJoinToEmptyRelation should respect exchange behavior when canChangeNumPartitions == false

2020-09-03 Thread GitBox
leanken commented on pull request #29614: URL: https://github.com/apache/spark/pull/29614#issuecomment-686846914 @cloud-fan @maropu If this is no longer considered as a bug, then I will close this PR and JIRA. Is that OK?

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686844132 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686844132 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
SparkQA removed a comment on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686835914 **[Test build #128271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128271/testReport)** for PR 29640 at commit

[GitHub] [spark] SparkQA commented on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
SparkQA commented on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686843894 **[Test build #128271 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128271/testReport)** for PR 29640 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
AmplabJenkins removed a comment on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686836271 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu edited a comment on pull request #29636: [SPARK-32786][SQL][TEST] Improve performance for some slow DPP tests

2020-09-03 Thread GitBox
maropu edited a comment on pull request #29636: URL: https://github.com/apache/spark/pull/29636#issuecomment-686832660 Thanks! Merged to master/branch-3.0. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins commented on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
AmplabJenkins commented on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686836271 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29640: [SPARK-32180][PYTHON][DOCS] Installation page of Getting Started in PySpark documentation

2020-09-03 Thread GitBox
SparkQA commented on pull request #29640: URL: https://github.com/apache/spark/pull/29640#issuecomment-686835914 **[Test build #128271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128271/testReport)** for PR 29640 at commit

[GitHub] [spark] SparkQA commented on pull request #28269: [SPARK-31493][SQL] Optimize InSet to In according partition size at InSubqueryExec

2020-09-03 Thread GitBox
SparkQA commented on pull request #28269: URL: https://github.com/apache/spark/pull/28269#issuecomment-686835937 **[Test build #128272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128272/testReport)** for PR 28269 at commit

  1   2   3   4   5   6   7   >