[GitHub] [spark] HyukjinKwon edited a comment on pull request #29108: [SPARK-32309][PYSPARK] Import missing sys import

2020-07-14 Thread GitBox
HyukjinKwon edited a comment on pull request #29108: URL: https://github.com/apache/spark/pull/29108#issuecomment-658439161 Thanks @Fokko and @dongjoon-hyun This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] HyukjinKwon commented on pull request #29108: [SPARK-32309][PYSPARK] Import missing sys import

2020-07-14 Thread GitBox
HyukjinKwon commented on pull request #29108: URL: https://github.com/apache/spark/pull/29108#issuecomment-658439161 Thanks @Fokko This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-658438694 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-658438694 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
HyukjinKwon commented on a change in pull request #29110: URL: https://github.com/apache/spark/pull/29110#discussion_r454674733 ## File path: python/pyspark/_globals.py ## @@ -59,10 +59,6 @@ def __new__(cls): cls.__instance = super(_NoValueType, cls).__new__(cls)

[GitHub] [spark] SparkQA removed a comment on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting featur

2020-07-14 Thread GitBox
SparkQA removed a comment on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-658350804 **[Test build #125852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125852/testReport)** for PR 28874 at commit [`1ba58fb`](https://gi

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
HyukjinKwon commented on a change in pull request #29110: URL: https://github.com/apache/spark/pull/29110#discussion_r454674370 ## File path: python/pyspark/cloudpickle.py ## @@ -1010,11 +1006,7 @@ def save_ellipsis(self, obj): def save_not_implemented(self, obj):

[GitHub] [spark] SparkQA commented on pull request #28874: [SPARK-32036] Replace references to blacklist/whitelist language with more appropriate terminology, excluding the blacklisting feature.

2020-07-14 Thread GitBox
SparkQA commented on pull request #28874: URL: https://github.com/apache/spark/pull/28874#issuecomment-658437961 **[Test build #125852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125852/testReport)** for PR 28874 at commit [`1ba58fb`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29112: [SPARK-32310][ML][PySpark] ML params default value parity part 1

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29112: URL: https://github.com/apache/spark/pull/29112#issuecomment-658427540 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29112: [SPARK-32310][ML][PySpark] ML params default value parity part 1

2020-07-14 Thread GitBox
SparkQA commented on pull request #29112: URL: https://github.com/apache/spark/pull/29112#issuecomment-658430071 **[Test build #125861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125861/testReport)** for PR 29112 at commit [`72c17e9`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29112: [SPARK-32310][ML][PySpark] ML params default value parity part 1

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29112: URL: https://github.com/apache/spark/pull/29112#issuecomment-658427540 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] huaxingao opened a new pull request #29112: [SPARK-32310][ML][PySpark] ML params default value parity part 1

2020-07-14 Thread GitBox
huaxingao opened a new pull request #29112: URL: https://github.com/apache/spark/pull/29112 ### What changes were proposed in this pull request? set params default values in trait ...Params in both Scala and Python. I will do this in two PRs. I will change classification, regression,

[GitHub] [spark] aokolnychyi commented on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
aokolnychyi commented on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658425886 Those failures seem to belong to the old commit before [this](https://github.com/apache/spark/pull/29089/commits/c58ad12609b732fe8539277f3dcfb046268c5740) change where I rem

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29067: [SPARK-32274][SQL] Make SQL cache serialization pluggable

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29067: URL: https://github.com/apache/spark/pull/29067#issuecomment-658424511 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29067: [SPARK-32274][SQL] Make SQL cache serialization pluggable

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29067: URL: https://github.com/apache/spark/pull/29067#issuecomment-658424511 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] viirya commented on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
viirya commented on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658424153 Some tests in `EliminateSortsSuite` were failed? This is an automated message from the Apache Git Service. To res

[GitHub] [spark] SparkQA commented on pull request #29067: [SPARK-32274][SQL] Make SQL cache serialization pluggable

2020-07-14 Thread GitBox
SparkQA commented on pull request #29067: URL: https://github.com/apache/spark/pull/29067#issuecomment-658423981 **[Test build #125860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125860/testReport)** for PR 29067 at commit [`ad11a79`](https://github.com

[GitHub] [spark] revans2 commented on pull request #29067: [SPARK-32274][SQL] Make SQL cache serialization pluggable

2020-07-14 Thread GitBox
revans2 commented on pull request #29067: URL: https://github.com/apache/spark/pull/29067#issuecomment-658422580 Thanks @tgravescs those were issues related to metrics. One that I was over counting, and another that I ended up under counting. They should be fixed now. --

[GitHub] [spark] ueshin commented on pull request #29100: [MINOR][R] Match collectAsArrowToR with non-streaming collectAsArrowToPython

2020-07-14 Thread GitBox
ueshin commented on pull request #29100: URL: https://github.com/apache/spark/pull/29100#issuecomment-658413818 > Note that you cannot create no partition Spark DataFrame in SparkR How about something like `filter(df, FALSE)`?

[GitHub] [spark] srowen commented on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
srowen commented on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658409026 Yep, as I feared: ``` [error] * method this()Unit in class org.apache.spark.ml.Estimator does not have a correspondent in current version [error]filter with: Pr

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658408216 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658408206 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
SparkQA removed a comment on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658402613 **[Test build #125859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125859/testReport)** for PR 29111 at commit [`8b9f4a8`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658408206 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29111: URL: https://github.com/apache/spark/pull/29111#discussion_r454639188 ## File path: mllib/src/main/scala/org/apache/spark/ml/Estimator.scala ## @@ -26,7 +27,7 @@ import org.apache.spark.sql.Dataset /** * Abstract cl

[GitHub] [spark] SparkQA commented on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
SparkQA commented on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658408157 **[Test build #125859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125859/testReport)** for PR 29111 at commit [`8b9f4a8`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

2020-07-14 Thread GitBox
dongjoon-hyun commented on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-658406021 Thank you, @viirya . cc @cloud-fan This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658403223 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658403223 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
SparkQA commented on pull request #29111: URL: https://github.com/apache/spark/pull/29111#issuecomment-658402613 **[Test build #125859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125859/testReport)** for PR 29111 at commit [`8b9f4a8`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #29098: [SPARK-32300][PYTHON][2.4] toPandas should work from a Spark DataFrame with no partitions

2020-07-14 Thread GitBox
dongjoon-hyun commented on pull request #29098: URL: https://github.com/apache/spark/pull/29098#issuecomment-658401580 +1, late LGTM. Thanks all! This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] srowen commented on a change in pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
srowen commented on a change in pull request #29111: URL: https://github.com/apache/spark/pull/29111#discussion_r454628725 ## File path: examples/src/main/java/org/apache/spark/examples/ml/JavaTokenizerExample.java ## @@ -23,7 +23,7 @@ import java.util.Arrays; import java.ut

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658398551 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125

[GitHub] [spark] srowen opened a new pull request #29111: [SPARK-29292][SQL][ML] Update rest of default modules (Hive, ML, etc) for Scala 2.13 compilation

2020-07-14 Thread GitBox
srowen opened a new pull request #29111: URL: https://github.com/apache/spark/pull/29111 ### What changes were proposed in this pull request? Same as https://github.com/apache/spark/pull/29078 and https://github.com/apache/spark/pull/28971 . This makes the rest of the default module

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658398544 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-658398526 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-658398511 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

2020-07-14 Thread GitBox
SparkQA removed a comment on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-658292521 **[Test build #125845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125845/testReport)** for PR 29107 at commit [`93c5ea1`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658398544 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-658398511 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
SparkQA removed a comment on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658325499 **[Test build #125847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125847/testReport)** for PR 29089 at commit [`83791b7`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

2020-07-14 Thread GitBox
SparkQA commented on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-658398100 **[Test build #125845 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125845/testReport)** for PR 29107 at commit [`93c5ea1`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
SparkQA commented on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658397973 **[Test build #125847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125847/testReport)** for PR 29089 at commit [`83791b7`](https://github.co

[GitHub] [spark] BryanCutler closed pull request #29098: [SPARK-32300][PYTHON][2.4] toPandas should work from a Spark DataFrame with no partitions

2020-07-14 Thread GitBox
BryanCutler closed pull request #29098: URL: https://github.com/apache/spark/pull/29098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] BryanCutler commented on pull request #29098: [SPARK-32300][PYTHON][2.4] toPandas should work from a Spark DataFrame with no partitions

2020-07-14 Thread GitBox
BryanCutler commented on pull request #29098: URL: https://github.com/apache/spark/pull/29098#issuecomment-658396921 merged to branch-2.4, thanks @HyukjinKwon ! This is an automated message from the Apache Git Service. To res

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658396071 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658396071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
SparkQA commented on pull request #29089: URL: https://github.com/apache/spark/pull/29089#issuecomment-658395393 **[Test build #125858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125858/testReport)** for PR 29089 at commit [`ba6a1bb`](https://github.com

[GitHub] [spark] aokolnychyi commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
aokolnychyi commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454621400 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsBeforeRepartitionSuite.scala ## @@ -0,0 +1,179 @@ +/*

[GitHub] [spark] aokolnychyi commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
aokolnychyi commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454621506 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsBeforeRepartitionSuite.scala ## @@ -0,0 +1,179 @@ +/*

[GitHub] [spark] aokolnychyi commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
aokolnychyi commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454621594 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsBeforeRepartitionSuite.scala ## @@ -0,0 +1,179 @@ +/*

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-658392436 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] aokolnychyi commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
aokolnychyi commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454619436 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -981,6 +982,10 @@ object EliminateSorts exten

[GitHub] [spark] AmplabJenkins commented on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-658392436 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

2020-07-14 Thread GitBox
SparkQA commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-658391915 **[Test build #125857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125857/testReport)** for PR 29066 at commit [`5e4d304`](https://github.com

[GitHub] [spark] rdblue commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
rdblue commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454617890 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsBeforeRepartitionSuite.scala ## @@ -0,0 +1,179 @@ +/* + *

[GitHub] [spark] dongjoon-hyun commented on pull request #29102: [SPARK-32303][PYTHON][TESTS] Remove leftover from editable mode installation in PIP test

2020-07-14 Thread GitBox
dongjoon-hyun commented on pull request #29102: URL: https://github.com/apache/spark/pull/29102#issuecomment-658390332 I re-trigger Jenkins because this PR should be tested on Jenkins. This is an automated message from the Ap

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-658389813 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-658389813 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] aokolnychyi commented on a change in pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

2020-07-14 Thread GitBox
aokolnychyi commented on a change in pull request #29066: URL: https://github.com/apache/spark/pull/29066#discussion_r454616197 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2Writes.scala ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apach

[GitHub] [spark] SparkQA removed a comment on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
SparkQA removed a comment on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658372070 **[Test build #125854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125854/testReport)** for PR 29110 at commit [`2e88907`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658389310 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658389310 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29102: [SPARK-32303][PYTHON][TESTS] Remove leftover from editable mode installation in PIP test

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29102: URL: https://github.com/apache/spark/pull/29102#issuecomment-658388942 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
SparkQA commented on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658389062 **[Test build #125854 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125854/testReport)** for PR 29110 at commit [`2e88907`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-658389037 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

2020-07-14 Thread GitBox
SparkQA removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-658211049 **[Test build #125840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125840/testReport)** for PR 29085 at commit [`ec754e2`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

2020-07-14 Thread GitBox
SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-658388736 **[Test build #125840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125840/testReport)** for PR 29085 at commit [`ec754e2`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29102: [SPARK-32303][PYTHON][TESTS] Remove leftover from editable mode installation in PIP test

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29102: URL: https://github.com/apache/spark/pull/29102#issuecomment-658388942 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-658389037 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29102: [SPARK-32303][PYTHON][BUILD] Remove leftover from editable mode installation in PIP test

2020-07-14 Thread GitBox
SparkQA commented on pull request #29102: URL: https://github.com/apache/spark/pull/29102#issuecomment-658388294 **[Test build #125855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125855/testReport)** for PR 29102 at commit [`5879260`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

2020-07-14 Thread GitBox
SparkQA commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-658388368 **[Test build #125856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125856/testReport)** for PR 29066 at commit [`d75f0e4`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29102: [SPARK-32303][PYTHON][BUILD] Remove leftover from editable mode installation in PIP test

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29102: URL: https://github.com/apache/spark/pull/29102#discussion_r454613712 ## File path: dev/run-pip-tests ## @@ -96,6 +96,8 @@ for python in "${PYTHON_EXECS[@]}"; do cd "$FWDIR"/python # Delete the egg info file

[GitHub] [spark] dongjoon-hyun commented on pull request #29102: [SPARK-32303][PYTHON][BUILD] Remove leftover from editable mode installation in PIP test

2020-07-14 Thread GitBox
dongjoon-hyun commented on pull request #29102: URL: https://github.com/apache/spark/pull/29102#issuecomment-658387579 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454610134 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -981,6 +982,10 @@ object EliminateSorts ext

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454609943 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala ## @@ -1299,8 +1299,9 @@ class ArrowConverter

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454607941 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsBeforeRepartitionSuite.scala ## @@ -0,0 +1,179 @@ +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454608042 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsBeforeRepartitionSuite.scala ## @@ -0,0 +1,179 @@ +

[GitHub] [spark] Fokko commented on pull request #29109: [SPARK-32311][PYSPARK][TESTS] Remove duplicate import

2020-07-14 Thread GitBox
Fokko commented on pull request #29109: URL: https://github.com/apache/spark/pull/29109#issuecomment-658377938 Thanks for picking this up so quickly @dongjoon-hyun This is an automated message from the Apache Git Service. To

[GitHub] [spark] aokolnychyi commented on a change in pull request #29089: [SPARK-32276][SQL] Remove redundant sorts before repartition nodes

2020-07-14 Thread GitBox
aokolnychyi commented on a change in pull request #29089: URL: https://github.com/apache/spark/pull/29089#discussion_r454601951 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/EliminateSortsBeforeRepartitionSuite.scala ## @@ -0,0 +1,179 @@ +/*

[GitHub] [spark] dongjoon-hyun closed pull request #29109: [SPARK-32311][PYSPARK][TESTS] Remove duplicate import

2020-07-14 Thread GitBox
dongjoon-hyun closed pull request #29109: URL: https://github.com/apache/spark/pull/29109 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29109: [SPARK-32311][PYSPARK][TESTS] Remove duplicate import

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29109: URL: https://github.com/apache/spark/pull/29109#discussion_r454600908 ## File path: python/pyspark/sql/tests/test_pandas_udf_scalar.py ## @@ -14,7 +14,6 @@ # See the License for the specific language governing permiss

[GitHub] [spark] dongjoon-hyun closed pull request #29103: [SPARK-32305][BUILD] Make `mvn clean` remove `metastore_db` and `spark-warehouse`

2020-07-14 Thread GitBox
dongjoon-hyun closed pull request #29103: URL: https://github.com/apache/spark/pull/29103 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658372433 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658326014 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
SparkQA commented on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658372070 **[Test build #125854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125854/testReport)** for PR 29110 at commit [`2e88907`](https://github.com

[GitHub] [spark] Fokko commented on a change in pull request #29109: [SPARK-32311][PYSPARK][TESTS] Remove duplicate import

2020-07-14 Thread GitBox
Fokko commented on a change in pull request #29109: URL: https://github.com/apache/spark/pull/29109#discussion_r454595646 ## File path: python/pyspark/sql/tests/test_pandas_udf_scalar.py ## @@ -14,7 +14,6 @@ # See the License for the specific language governing permissions and

[GitHub] [spark] AmplabJenkins commented on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658372433 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun commented on pull request #29110: [SPARK-32313][PYSPARK] Remove Python 2 artifacts

2020-07-14 Thread GitBox
dongjoon-hyun commented on pull request #29110: URL: https://github.com/apache/spark/pull/29110#issuecomment-658371609 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29109: [SPARK-32311][PYSPARK][TESTS] Remove duplicate import

2020-07-14 Thread GitBox
dongjoon-hyun commented on a change in pull request #29109: URL: https://github.com/apache/spark/pull/29109#discussion_r454594452 ## File path: python/pyspark/sql/tests/test_pandas_udf_scalar.py ## @@ -14,7 +14,6 @@ # See the License for the specific language governing permiss

[GitHub] [spark] dongjoon-hyun closed pull request #29108: [SPARK-32309][PYSPARK] Import missing sys import

2020-07-14 Thread GitBox
dongjoon-hyun closed pull request #29108: URL: https://github.com/apache/spark/pull/29108 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #29106: [SPARK-32307][SQL] ScalaUDF's canonicalized expression should exclude inputEncoders

2020-07-14 Thread GitBox
dongjoon-hyun commented on pull request #29106: URL: https://github.com/apache/spark/pull/29106#issuecomment-658368554 BTW, @cloud-fan and @HyukjinKwon . I merged this because `GitHub Action` passed. This is an automated mes

[GitHub] [spark] adjordan commented on pull request #29080: [SPARK-32271][ML] Update CrossValidator to train folds in parallel

2020-07-14 Thread GitBox
adjordan commented on pull request #29080: URL: https://github.com/apache/spark/pull/29080#issuecomment-658366393 @viirya This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [spark] dongjoon-hyun closed pull request #29106: [SPARK-32307][SQL] ScalaUDF's canonicalized expression should exclude inputEncoders

2020-07-14 Thread GitBox
dongjoon-hyun closed pull request #29106: URL: https://github.com/apache/spark/pull/29106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28950: [SPARK-32094][PYTHON] Update cloudpickle to v1.4.1

2020-07-14 Thread GitBox
AmplabJenkins removed a comment on pull request #28950: URL: https://github.com/apache/spark/pull/28950#issuecomment-658363361 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28950: [SPARK-32094][PYTHON] Update cloudpickle to v1.4.1

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #28950: URL: https://github.com/apache/spark/pull/28950#issuecomment-658363361 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28950: [SPARK-32094][PYTHON] Update cloudpickle to v1.4.1

2020-07-14 Thread GitBox
SparkQA removed a comment on pull request #28950: URL: https://github.com/apache/spark/pull/28950#issuecomment-658285064 **[Test build #125844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125844/testReport)** for PR 28950 at commit [`d731f63`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28950: [SPARK-32094][PYTHON] Update cloudpickle to v1.4.1

2020-07-14 Thread GitBox
SparkQA commented on pull request #28950: URL: https://github.com/apache/spark/pull/28950#issuecomment-658362313 **[Test build #125844 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125844/testReport)** for PR 28950 at commit [`d731f63`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #29106: [SPARK-32307][SQL] ScalaUDF's canonicalized expression should exclude inputEncoders

2020-07-14 Thread GitBox
dongjoon-hyun commented on pull request #29106: URL: https://github.com/apache/spark/pull/29106#issuecomment-658359436 If this exists on `branch-3.0` and is not released yet, the affected version should be `3.0.1` instead of `3.0.0`. ---

[GitHub] [spark] AmplabJenkins commented on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

2020-07-14 Thread GitBox
AmplabJenkins commented on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-658354949 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

<    1   2   3   4   5   6   7   8   9   10   >