[GitHub] [spark] LuciferYang commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
LuciferYang commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674156875 @dongjoon-hyun "abort the job if total size of results is too large" in TaskSetManagerSuite failed, a little strange

[GitHub] [spark] AmplabJenkins commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674160674 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] LuciferYang commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
LuciferYang commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674160116 Manual test ``` mvn clean install -pl core -am -DwildcardSuites=org.apache.spark.scheduler.TaskSetManagerSuite -Dtest=none Run completed in 46 seconds, 135

[GitHub] [spark] srowen commented on a change in pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
srowen commented on a change in pull request #29434: URL: https://github.com/apache/spark/pull/29434#discussion_r470793047 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala ## @@ -351,6 +351,18 @@ class

[GitHub] [spark] SparkQA commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
SparkQA commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674229394 **[Test build #127464 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127464/testReport)** for PR 29434 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
SparkQA removed a comment on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674160102 **[Test build #127464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127464/testReport)** for PR 29434 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29435: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29435: URL: https://github.com/apache/spark/pull/29435#issuecomment-674243886 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674160674 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] holdenk commented on pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

2020-08-14 Thread GitBox
holdenk commented on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-674186603 Thank you for taking the time to resolve this and make such a clear writeup of the root cause. From an in-production not-in-test question: if the executor exits we also want to

[GitHub] [spark] holdenk commented on a change in pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

2020-08-14 Thread GitBox
holdenk commented on a change in pull request #29422: URL: https://github.com/apache/spark/pull/29422#discussion_r470765386 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -323,6 +326,7 @@ private[spark] class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674201907 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674201907 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29435: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29435: URL: https://github.com/apache/spark/pull/29435#issuecomment-674241724 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] viirya commented on a change in pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
viirya commented on a change in pull request #29396: URL: https://github.com/apache/spark/pull/29396#discussion_r470728557 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScan.scala ## @@ -0,0 +1,50 @@ +/* + * Licensed to the

[GitHub] [spark] viirya commented on a change in pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
viirya commented on a change in pull request #29396: URL: https://github.com/apache/spark/pull/29396#discussion_r470732952 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala ## @@ -0,0 +1,70 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
SparkQA commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674160102 **[Test build #127464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127464/testReport)** for PR 29434 at commit

[GitHub] [spark] mingjialiu commented on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
mingjialiu commented on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674230500 > This is against 2.4. Could you also check whether the master branch still has such an issue? I cannot repro the issue at master branch. 3.0. in unit test.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674230204 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] viirya commented on a change in pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
viirya commented on a change in pull request #29396: URL: https://github.com/apache/spark/pull/29396#discussion_r470732059 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScanBuilder.scala ## @@ -0,0 +1,70 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA removed a comment on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
SparkQA removed a comment on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674085136 **[Test build #127460 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127460/testReport)** for PR 29431 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674210880 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29428: [SPARK-32608][SQL] Script Transform ROW FORMAT DELIMIT value should format value

2020-08-14 Thread GitBox
SparkQA removed a comment on pull request #29428: URL: https://github.com/apache/spark/pull/29428#issuecomment-674101424 **[Test build #127462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127462/testReport)** for PR 29428 at commit

[GitHub] [spark] mingjialiu opened a new pull request #29435: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
mingjialiu opened a new pull request #29435: URL: https://github.com/apache/spark/pull/29435 ### What changes were proposed in this pull request? Copy to master branch the unit test added for branch-2.4(https://github.com/apache/spark/pull/29430). ### Why are

[GitHub] [spark] dongjoon-hyun commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
dongjoon-hyun commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674157678 ok to test This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] viirya commented on a change in pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
viirya commented on a change in pull request #29396: URL: https://github.com/apache/spark/pull/29396#discussion_r470736186 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCWriteBuilder.scala ## @@ -0,0 +1,46 @@ +/* + * Licensed to

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470753712 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,215 @@ case class

[GitHub] [spark] SparkQA removed a comment on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
SparkQA removed a comment on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674088406 **[Test build #127461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127461/testReport)** for PR 29431 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674210880 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29428: [SPARK-32608][SQL] Script Transform ROW FORMAT DELIMIT value should format value

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29428: URL: https://github.com/apache/spark/pull/29428#issuecomment-674220724 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29428: [SPARK-32608][SQL] Script Transform ROW FORMAT DELIMIT value should format value

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29428: URL: https://github.com/apache/spark/pull/29428#issuecomment-674220724 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] emkornfield commented on a change in pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
emkornfield commented on a change in pull request #29430: URL: https://github.com/apache/spark/pull/29430#discussion_r470845244 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala ## @@ -371,6 +371,25 @@ class DataSourceV2Suite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674121960 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29427: [SPARK-25557][SQL][TEST][Followup] Add case-sensitivity test for ORC predicate pushdown

2020-08-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29427: URL: https://github.com/apache/spark/pull/29427#discussion_r470731273 ## File path: sql/core/v1.2/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala ## @@ -513,5 +513,98 @@ class

[GitHub] [spark] SparkQA commented on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
SparkQA commented on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674201047 **[Test build #127460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127460/testReport)** for PR 29431 at commit

[GitHub] [spark] SparkQA commented on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
SparkQA commented on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-674210062 **[Test build #127461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127461/testReport)** for PR 29431 at commit

[GitHub] [spark] SparkQA commented on pull request #29428: [SPARK-32608][SQL] Script Transform ROW FORMAT DELIMIT value should format value

2020-08-14 Thread GitBox
SparkQA commented on pull request #29428: URL: https://github.com/apache/spark/pull/29428#issuecomment-674217964 **[Test build #127462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127462/testReport)** for PR 29428 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-674230204 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
SparkQA commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-674238955 **[Test build #127463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127463/testReport)** for PR 29396 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
SparkQA removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-674125075 **[Test build #127463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127463/testReport)** for PR 29396 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-674239618 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-674239618 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29435: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29435: URL: https://github.com/apache/spark/pull/29435#issuecomment-674241724 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29378: [SPARK-30069][CORE][YARN] Clean up non-shuffle disk block manager files following executor exists on YARN

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29378: URL: https://github.com/apache/spark/pull/29378#issuecomment-673917333 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] leanken opened a new pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementEx…

2020-08-14 Thread GitBox
leanken opened a new pull request #29431: URL: https://github.com/apache/spark/pull/29431 ### What changes were proposed in this pull request? Found java.util.NoSuchElementException in UT log of AdaptiveQueryExecSuite. During AQE, when sub-plan changed, LiveExecutionData is using the

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29428: [SPARK-32608][SQL] Script Transform ROW FORMAT DELIMIT value should format value

2020-08-14 Thread GitBox
AngersZh commented on a change in pull request #29428: URL: https://github.com/apache/spark/pull/29428#discussion_r470449703 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala ## @@ -330,4 +331,44 @@ class SparkSqlParserSuite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-673947748 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29431: [SPARK-32615][SQL] Fix AQE aggregateMetrics java.util.NoSuchElementException

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29431: URL: https://github.com/apache/spark/pull/29431#issuecomment-673947748 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-14 Thread GitBox
SparkQA commented on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-673947714 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32066/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-673956114 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] sarutak commented on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-14 Thread GitBox
sarutak commented on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-673964388 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #26319: [SPARK-29594][SQL] Provide better error message when creating a Dataset from a Sequence of Case class where a field name started with a numbe

2020-08-14 Thread GitBox
SparkQA commented on pull request #26319: URL: https://github.com/apache/spark/pull/26319#issuecomment-673991562 **[Test build #127452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127452/testReport)** for PR 26319 at commit

[GitHub] [spark] ulysses-you commented on pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

2020-08-14 Thread GitBox
ulysses-you commented on pull request #29423: URL: https://github.com/apache/spark/pull/29423#issuecomment-673991291 thanks for merging! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-14 Thread GitBox
cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r470522985 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1479,6 +1479,33 @@ class Analyzer( //

[GitHub] [spark] maropu commented on pull request #29432: [SPARK-32616][SQL] Window operators should be added determinedly

2020-08-14 Thread GitBox
maropu commented on pull request #29432: URL: https://github.com/apache/spark/pull/29432#issuecomment-673995926 oh.. good catch! LGTM, pending Jenkins. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29428: [SPARK-32608][SQL] Script Transform ROW FORMAT DELIMIT value should format value

2020-08-14 Thread GitBox
AngersZh commented on a change in pull request #29428: URL: https://github.com/apache/spark/pull/29428#discussion_r470529869 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala ## @@ -311,6 +311,77 @@ abstract class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-674021539 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-674021532 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470845507 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -66,6 +66,30 @@ private[execution] sealed trait

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470855625 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -66,6 +66,30 @@ private[execution] sealed trait

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470883805 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,215 @@ case class

[GitHub] [spark] agrawaldevesh commented on pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

2020-08-14 Thread GitBox
agrawaldevesh commented on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-674294907 > Thank you for taking the time to resolve this and make such a clear writeup of the root cause. From an in-production not-in-test question: if the executor exits we also

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

2020-08-14 Thread GitBox
agrawaldevesh commented on a change in pull request #29422: URL: https://github.com/apache/spark/pull/29422#discussion_r470888548 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -212,22 +226,27 @@ class DecommissionWorkerSuite

[GitHub] [spark] rohitmishr1484 commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
rohitmishr1484 commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674294866 @HyukjinKwon, Thanks for your helpful comment. I have done the suggested changes but if you still find something which requires modification, please let me know, I

[GitHub] [spark] AmplabJenkins commented on pull request #29436: [SPARK-32620][SQL] Reset the numPartitions metric when DPP is enabled

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29436: URL: https://github.com/apache/spark/pull/29436#issuecomment-674325021 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29436: [SPARK-32620][SQL] Reset the numPartitions metric when DPP is enabled

2020-08-14 Thread GitBox
SparkQA commented on pull request #29436: URL: https://github.com/apache/spark/pull/29436#issuecomment-674324895 **[Test build #127468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127468/testReport)** for PR 29436 at commit

[GitHub] [spark] SparkQA commented on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
SparkQA commented on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674328817 **[Test build #127466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127466/testReport)** for PR 29430 at commit

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470854938 ## File path: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ## @@ -428,6 +428,68 @@ public MapIterator destructiveIterator() {

[GitHub] [spark] AmplabJenkins commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674274794 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674274794 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] asfgit closed pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-14 Thread GitBox
asfgit closed pull request #28939: URL: https://github.com/apache/spark/pull/28939 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] mridulm commented on pull request #28939: [SPARK-32119][CORE] ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-14 Thread GitBox
mridulm commented on pull request #28939: URL: https://github.com/apache/spark/pull/28939#issuecomment-674293037 Thanks @sarutak ! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins commented on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674297234 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674297234 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470898482 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -66,6 +66,30 @@ private[execution] sealed trait

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470908654 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,215 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-674315733 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-674315733 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] agrawaldevesh commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

2020-08-14 Thread GitBox
agrawaldevesh commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-674318466 @holdenk can this PR be abandoned/closed now since this is finally in ? This is an automated message from

[GitHub] [spark] wangyum opened a new pull request #29436: [SPARK-32620][SQL] Reset the numPartitions metric when DPP is enabled

2020-08-14 Thread GitBox
wangyum opened a new pull request #29436: URL: https://github.com/apache/spark/pull/29436 ### What changes were proposed in this pull request? This pr reset the `numPartitions` metric when DPP is enabled. ### Why are the changes needed? Fix metric issue.

[GitHub] [spark] SparkQA removed a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
SparkQA removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674274381 **[Test build #127465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127465/testReport)** for PR 29410 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674282565 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
SparkQA commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674282273 **[Test build #127465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127465/testReport)** for PR 29410 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674282565 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] mingjialiu commented on a change in pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
mingjialiu commented on a change in pull request #29430: URL: https://github.com/apache/spark/pull/29430#discussion_r470889123 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/DataSourceV2Suite.scala ## @@ -371,6 +371,25 @@ class DataSourceV2Suite extends

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

2020-08-14 Thread GitBox
agrawaldevesh commented on a change in pull request #29422: URL: https://github.com/apache/spark/pull/29422#discussion_r470889426 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -136,7 +137,9 @@ private[spark] class

[GitHub] [spark] mingjialiu commented on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
mingjialiu commented on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674302449 > I think it might make sense to add a test with a self join between the two dfs yields the correct results to mirror the issue observed The issue observed cannot be

[GitHub] [spark] huaxingao commented on a change in pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-14 Thread GitBox
huaxingao commented on a change in pull request #29396: URL: https://github.com/apache/spark/pull/29396#discussion_r470898996 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCScan.scala ## @@ -0,0 +1,50 @@ +/* + * Licensed to the

[GitHub] [spark] rohitmishr1484 commented on a change in pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
rohitmishr1484 commented on a change in pull request #29410: URL: https://github.com/apache/spark/pull/29410#discussion_r470866807 ## File path: python/docs/source/getting_started/index.rst ## @@ -20,3 +20,13 @@ Getting Started === +**PySpark** is the Python

[GitHub] [spark] rohitmishr1484 commented on a change in pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
rohitmishr1484 commented on a change in pull request #29410: URL: https://github.com/apache/spark/pull/29410#discussion_r470866885 ## File path: python/docs/source/getting_started/index.rst ## @@ -20,3 +20,13 @@ Getting Started === +**PySpark** is the Python

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
agrawaldevesh commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470873077 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,215 @@ case class

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r470884722 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,215 @@ case class

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

2020-08-14 Thread GitBox
agrawaldevesh commented on a change in pull request #29422: URL: https://github.com/apache/spark/pull/29422#discussion_r470890156 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -323,6 +326,7 @@ private[spark] class

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29422: [SPARK-32613][CORE] Fix regressions in DecommissionWorkerSuite

2020-08-14 Thread GitBox
agrawaldevesh commented on a change in pull request #29422: URL: https://github.com/apache/spark/pull/29422#discussion_r470890206 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -294,10 +294,13 @@ private[spark] class

[GitHub] [spark] SparkQA commented on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
SparkQA commented on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674296775 **[Test build #127466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127466/testReport)** for PR 29430 at commit

[GitHub] [spark] SparkQA commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

2020-08-14 Thread GitBox
SparkQA commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-674316955 **[Test build #127467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127467/testReport)** for PR 29342 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
AmplabJenkins commented on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674328932 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
SparkQA removed a comment on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674296775 **[Test build #127466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127466/testReport)** for PR 29430 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29430: [SPARK-32609] Incorrect exchange reuse with DataSourceV2

2020-08-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29430: URL: https://github.com/apache/spark/pull/29430#issuecomment-674328932 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] rohitmishr1484 commented on a change in pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
rohitmishr1484 commented on a change in pull request #29410: URL: https://github.com/apache/spark/pull/29410#discussion_r470866980 ## File path: python/docs/source/getting_started/installation.rst ## @@ -0,0 +1,62 @@ +.. Licensed to the Apache Software Foundation (ASF) under

[GitHub] [spark] SparkQA commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

2020-08-14 Thread GitBox
SparkQA commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-674274381 **[Test build #127465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127465/testReport)** for PR 29410 at commit

  1   2   3   4   5   >