[GitHub] [spark] SparkQA commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853588189 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43791/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-02 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-853588123 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43792/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32761: [SPARK-35621][SQL] Add rule id pruning to the TypeCoercion rule

2021-06-02 Thread GitBox
SparkQA commented on pull request #32761: URL: https://github.com/apache/spark/pull/32761#issuecomment-853586808 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43785/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32741: [SPARK-35568][SQL] Add the BroadcastExchange after re-optimizing the physical plan to fix the UnsupportedOperationException when enabling bo

2021-06-02 Thread GitBox
SparkQA commented on pull request #32741: URL: https://github.com/apache/spark/pull/32741#issuecomment-853586016 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43790/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #32754: [SPARK-35613][CORE][SQL] Cache commonly occurring strings in SQLMetrics, JSONProtocol and AccumulatorV2 classes

2021-06-02 Thread GitBox
SparkQA removed a comment on pull request #32754: URL: https://github.com/apache/spark/pull/32754#issuecomment-853525564 **[Test build #139262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139262/testReport)** for PR 32754 at commit

[GitHub] [spark] SparkQA commented on pull request #32754: [SPARK-35613][CORE][SQL] Cache commonly occurring strings in SQLMetrics, JSONProtocol and AccumulatorV2 classes

2021-06-02 Thread GitBox
SparkQA commented on pull request #32754: URL: https://github.com/apache/spark/pull/32754#issuecomment-853580364 **[Test build #139262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139262/testReport)** for PR 32754 at commit

[GitHub] [spark] SparkQA commented on pull request #32737: [SPARK-35606][PYTHON][INFRA] List Python 3.9 installed libraries in build_and_test workflow

2021-06-02 Thread GitBox
SparkQA commented on pull request #32737: URL: https://github.com/apache/spark/pull/32737#issuecomment-853578682 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43788/ -- This is an automated message from the

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32745: [SPARK-35523] Fix the default value in Data Source Options page

2021-06-02 Thread GitBox
HyukjinKwon commented on a change in pull request #32745: URL: https://github.com/apache/spark/pull/32745#discussion_r644497726 ## File path: docs/sql-data-sources-text.md ## @@ -57,7 +57,7 @@ Data source options of text can be set via: Review comment: Can we

[GitHub] [spark] pingsutw commented on a change in pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-02 Thread GitBox
pingsutw commented on a change in pull request #32738: URL: https://github.com/apache/spark/pull/32738#discussion_r644490173 ## File path: python/pyspark/pandas/indexing.py ## @@ -608,7 +608,9 @@ def __setitem__(self, key, value): if cond is None:

[GitHub] [spark] HyukjinKwon commented on pull request #32745: [SPARK-35523] Fix the default value in Data Source Options page

2021-06-02 Thread GitBox
HyukjinKwon commented on pull request #32745: URL: https://github.com/apache/spark/pull/32745#issuecomment-853577413 can we add default values at https://github.com/apache/spark/blob/5ff5770e5c4aeeec9c5f0ab173c49dfe003e5eba/docs/sql-data-sources-jdbc.md too? -- This is an automated

[GitHub] [spark] pingsutw commented on a change in pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-02 Thread GitBox
pingsutw commented on a change in pull request #32738: URL: https://github.com/apache/spark/pull/32738#discussion_r644495912 ## File path: python/pyspark/pandas/indexing.py ## @@ -1246,7 +1255,8 @@ def _select_cols_by_iterable( % (len(cast(Sized,

[GitHub] [spark] SparkQA commented on pull request #32751: [SPARK-35612][SQL] Support LZ4 compression in ORC data source

2021-06-02 Thread GitBox
SparkQA commented on pull request #32751: URL: https://github.com/apache/spark/pull/32751#issuecomment-853575054 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43789/ -- This is an automated message from the

[GitHub] [spark] pingsutw commented on a change in pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-02 Thread GitBox
pingsutw commented on a change in pull request #32738: URL: https://github.com/apache/spark/pull/32738#discussion_r644493906 ## File path: python/pyspark/pandas/indexing.py ## @@ -1138,7 +1146,8 @@ def _select_rows_else( ) def _get_from_multiindex_column( -

[GitHub] [spark] AmplabJenkins commented on pull request #32735: [SPARK-35580][SQL] Implement canonicalized method for HigherOrderFunction

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32735: URL: https://github.com/apache/spark/pull/32735#issuecomment-853570290 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43787/ --

[GitHub] [spark] SparkQA commented on pull request #32735: [SPARK-35580][SQL] Implement canonicalized method for HigherOrderFunction

2021-06-02 Thread GitBox
SparkQA commented on pull request #32735: URL: https://github.com/apache/spark/pull/32735#issuecomment-853570260 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43787/ -- This is an automated message from the

[GitHub] [spark] pingsutw commented on a change in pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-02 Thread GitBox
pingsutw commented on a change in pull request #32738: URL: https://github.com/apache/spark/pull/32738#discussion_r644490173 ## File path: python/pyspark/pandas/indexing.py ## @@ -608,7 +608,9 @@ def __setitem__(self, key, value): if cond is None:

[GitHub] [spark] pingsutw commented on a change in pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-02 Thread GitBox
pingsutw commented on a change in pull request #32738: URL: https://github.com/apache/spark/pull/32738#discussion_r644489741 ## File path: python/pyspark/pandas/indexing.py ## @@ -514,7 +514,7 @@ def __getitem__(self, key) -> Union["Series", "DataFrame"]: except

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853566086 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139258/

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32737: [SPARK-35606][PYTHON][INFRA] List Python 3.9 installed libraries in build_and_test workflow

2021-06-02 Thread GitBox
HyukjinKwon commented on a change in pull request #32737: URL: https://github.com/apache/spark/pull/32737#discussion_r644487409 ## File path: .github/workflows/build_and_test.yml ## @@ -217,6 +217,11 @@ jobs: run: | python3.6 -m pip install numpy

[GitHub] [spark] LuciferYang commented on pull request #32710: [SPARK-35574][BUILD] Add a compile arg to turn compilation warnings related to `procedure syntax` to compilation errors in Scala 2.13

2021-06-02 Thread GitBox
LuciferYang commented on pull request #32710: URL: https://github.com/apache/spark/pull/32710#issuecomment-853566158 thx @srowen @HyukjinKwon @gengliangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853566086 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139258/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
SparkQA removed a comment on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853507209 **[Test build #139258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139258/testReport)** for PR 32750 at commit

[GitHub] [spark] SparkQA commented on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
SparkQA commented on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853565054 **[Test build #139258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139258/testReport)** for PR 32750 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32754: [SPARK-35613][CORE][SQL] Cache commonly occurring strings in SQLMetrics, JSONProtocol and AccumulatorV2 classes

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32754: URL: https://github.com/apache/spark/pull/32754#issuecomment-853564361 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43786/

[GitHub] [spark] AmplabJenkins commented on pull request #32754: [SPARK-35613][CORE][SQL] Cache commonly occurring strings in SQLMetrics, JSONProtocol and AccumulatorV2 classes

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32754: URL: https://github.com/apache/spark/pull/32754#issuecomment-853564361 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43786/ --

[GitHub] [spark] SparkQA commented on pull request #32737: [SPARK-35606][PYTHON][INFRA] List Python 3.9 installed libraries in build_and_test workflow

2021-06-02 Thread GitBox
SparkQA commented on pull request #32737: URL: https://github.com/apache/spark/pull/32737#issuecomment-853564322 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43788/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32754: [SPARK-35613][CORE][SQL] Cache commonly occurring strings in SQLMetrics, JSONProtocol and AccumulatorV2 classes

2021-06-02 Thread GitBox
SparkQA commented on pull request #32754: URL: https://github.com/apache/spark/pull/32754#issuecomment-853564346 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43786/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853563766 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139259/

[GitHub] [spark] AmplabJenkins commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853563766 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139259/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
SparkQA removed a comment on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853508975 **[Test build #139259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139259/testReport)** for PR 32726 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32743: [SPARK-35396][CORE][FOLLOWUP] Free memory entry immediately

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32743: URL: https://github.com/apache/spark/pull/32743#issuecomment-853563089 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139255/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32762: [SPARK-35081][DOCS] Add Data Source Option links to missing documents

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32762: URL: https://github.com/apache/spark/pull/32762#issuecomment-853563087 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43784/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32756: [SPARK-35589][CORE][3.1] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32756: URL: https://github.com/apache/spark/pull/32756#issuecomment-853563093 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139254/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853563086 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139257/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853563090 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43783/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853563085 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43782/

[GitHub] [spark] AmplabJenkins commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853563090 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43783/ --

[GitHub] [spark] SparkQA commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853563062 **[Test build #139259 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139259/testReport)** for PR 32726 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853563085 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43782/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853563086 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139257/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32743: [SPARK-35396][CORE][FOLLOWUP] Free memory entry immediately

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32743: URL: https://github.com/apache/spark/pull/32743#issuecomment-853563089 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139255/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32762: [SPARK-35081][DOCS] Add Data Source Option links to missing documents

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32762: URL: https://github.com/apache/spark/pull/32762#issuecomment-853563087 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43784/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32756: [SPARK-35589][CORE][3.1] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32756: URL: https://github.com/apache/spark/pull/32756#issuecomment-853563093 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139254/ -- This

[GitHub] [spark] SparkQA commented on pull request #32751: [SPARK-35612][SQL] Support LZ4 compression in ORC data source

2021-06-02 Thread GitBox
SparkQA commented on pull request #32751: URL: https://github.com/apache/spark/pull/32751#issuecomment-853561844 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43789/ -- This is an automated message from the Apache

[GitHub] [spark] otterc commented on a change in pull request #32140: [WIP][SPARK-32922][SHUFFLE][CORE] Adds support for executors to fetch local and remote merged shuffle data

2021-06-02 Thread GitBox
otterc commented on a change in pull request #32140: URL: https://github.com/apache/spark/pull/32140#discussion_r640210756 ## File path: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala ## @@ -712,38 +824,66 @@ final class

[GitHub] [spark] HyukjinKwon closed pull request #32762: [SPARK-35081][DOCS] Add Data Source Option links to missing documents

2021-06-02 Thread GitBox
HyukjinKwon closed pull request #32762: URL: https://github.com/apache/spark/pull/32762 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HyukjinKwon commented on pull request #32762: [SPARK-35081][DOCS] Add Data Source Option links to missing documents

2021-06-02 Thread GitBox
HyukjinKwon commented on pull request #32762: URL: https://github.com/apache/spark/pull/32762#issuecomment-853560437 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon closed pull request #32710: [SPARK-35574][BUILD] Add a compile arg to turn compilation warnings related to `procedure syntax` to compilation errors in Scala 2.13

2021-06-02 Thread GitBox
HyukjinKwon closed pull request #32710: URL: https://github.com/apache/spark/pull/32710 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HyukjinKwon commented on pull request #32710: [SPARK-35574][BUILD] Add a compile arg to turn compilation warnings related to `procedure syntax` to compilation errors in Scala 2.13

2021-06-02 Thread GitBox
HyukjinKwon commented on pull request #32710: URL: https://github.com/apache/spark/pull/32710#issuecomment-853560256 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] pingsutw commented on a change in pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-02 Thread GitBox
pingsutw commented on a change in pull request #32738: URL: https://github.com/apache/spark/pull/32738#discussion_r644481542 ## File path: python/pyspark/pandas/generic.py ## @@ -3064,25 +3064,25 @@ def ffill(self, axis=None, inplace=False, limit=None) -> Union["DataFrame",

[GitHub] [spark] SparkQA removed a comment on pull request #32756: [SPARK-35589][CORE][3.1] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

2021-06-02 Thread GitBox
SparkQA removed a comment on pull request #32756: URL: https://github.com/apache/spark/pull/32756#issuecomment-853503097 **[Test build #139254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139254/testReport)** for PR 32756 at commit

[GitHub] [spark] SparkQA commented on pull request #32735: [SPARK-35580][SQL] Implement canonicalized method for HigherOrderFunction

2021-06-02 Thread GitBox
SparkQA commented on pull request #32735: URL: https://github.com/apache/spark/pull/32735#issuecomment-853558709 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43787/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32756: [SPARK-35589][CORE][3.1] BlockManagerMasterEndpoint should not ignore index-only shuffle file during updating

2021-06-02 Thread GitBox
SparkQA commented on pull request #32756: URL: https://github.com/apache/spark/pull/32756#issuecomment-853558663 **[Test build #139254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139254/testReport)** for PR 32756 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
SparkQA removed a comment on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853507176 **[Test build #139257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139257/testReport)** for PR 32760 at commit

[GitHub] [spark] SparkQA commented on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
SparkQA commented on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853557358 **[Test build #139257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139257/testReport)** for PR 32760 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32743: [SPARK-35396][CORE][FOLLOWUP] Free memory entry immediately

2021-06-02 Thread GitBox
SparkQA removed a comment on pull request #32743: URL: https://github.com/apache/spark/pull/32743#issuecomment-853503136 **[Test build #139255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139255/testReport)** for PR 32743 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32605: [SPARK-35446] Override getJDBCType in MySQLDialect to map FloatType to FLOAT

2021-06-02 Thread GitBox
HyukjinKwon commented on a change in pull request #32605: URL: https://github.com/apache/spark/pull/32605#discussion_r644477321 ## File path: sql/core/src/main/scala/org/apache/spark/sql/jdbc/MySQLDialect.scala ## @@ -94,4 +95,11 @@ private case object MySQLDialect extends

[GitHub] [spark] SparkQA commented on pull request #32762: [SPARK-35081][DOCS] Add Data Source Option links to missing documents

2021-06-02 Thread GitBox
SparkQA commented on pull request #32762: URL: https://github.com/apache/spark/pull/32762#issuecomment-853554047 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43784/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853553814 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43783/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
SparkQA commented on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853552400 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43782/ -- This is an automated message from the

[GitHub] [spark] viirya commented on a change in pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-02 Thread GitBox
viirya commented on a change in pull request #32582: URL: https://github.com/apache/spark/pull/32582#discussion_r644473480 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBFileManager.scala ## @@ -17,18 +17,265 @@ package

[GitHub] [spark] SparkQA commented on pull request #32743: [SPARK-35396][CORE][FOLLOWUP] Free memory entry immediately

2021-06-02 Thread GitBox
SparkQA commented on pull request #32743: URL: https://github.com/apache/spark/pull/32743#issuecomment-853551215 **[Test build #139255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139255/testReport)** for PR 32743 at commit

[GitHub] [spark] SparkQA commented on pull request #32761: [SPARK-35621][SQL] Add rule id pruning to the TypeCoercion rule

2021-06-02 Thread GitBox
SparkQA commented on pull request #32761: URL: https://github.com/apache/spark/pull/32761#issuecomment-853550890 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43785/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853547190 **[Test build #139271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139271/testReport)** for PR 32726 at commit

[GitHub] [spark] SparkQA commented on pull request #32763: [SPARK-35058][SQL] Group exception messages in hive/client

2021-06-02 Thread GitBox
SparkQA commented on pull request #32763: URL: https://github.com/apache/spark/pull/32763#issuecomment-853545991 **[Test build #139270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139270/testReport)** for PR 32763 at commit

[GitHub] [spark] beliefer opened a new pull request #32763: [SPARK-35058][SQL] Group exception messages in hive/client

2021-06-02 Thread GitBox
beliefer opened a new pull request #32763: URL: https://github.com/apache/spark/pull/32763 ### What changes were proposed in this pull request? This PR group exception messages in `sql/hive/src/main/scala/org/apache/spark/sql/hive/client`. ### Why are the changes needed?

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-02 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-853545043 **[Test build #139269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139269/testReport)** for PR 32303 at commit

[GitHub] [spark] SparkQA commented on pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

2021-06-02 Thread GitBox
SparkQA commented on pull request #31102: URL: https://github.com/apache/spark/pull/31102#issuecomment-853543985 **[Test build #139268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139268/testReport)** for PR 31102 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32751: [SPARK-35612][SQL] Support LZ4 compression in ORC data source

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32751: URL: https://github.com/apache/spark/pull/32751#issuecomment-853462929 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139240/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32758: [SPARK-35416][K8S][FOLLOWUP] Use Set instead of ArrayBuffer

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32758: URL: https://github.com/apache/spark/pull/32758#issuecomment-853508200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32743: [SPARK-35396][CORE][FOLLOWUP] Free memory entry immediately

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32743: URL: https://github.com/apache/spark/pull/32743#issuecomment-853543360 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43780/

[GitHub] [spark] SparkQA commented on pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
SparkQA commented on pull request #32726: URL: https://github.com/apache/spark/pull/32726#issuecomment-853543625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32759: [SPARK-35619][ML] Refactor LinearRegression - make huber support virtual centering

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32759: URL: https://github.com/apache/spark/pull/32759#issuecomment-853543354 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43779/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-853543357 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139250/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853543353 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43781/

[GitHub] [spark] SparkQA commented on pull request #32741: [SPARK-35568][SQL] Add the BroadcastExchange after re-optimizing the physical plan to fix the UnsupportedOperationException when enabling bo

2021-06-02 Thread GitBox
SparkQA commented on pull request #32741: URL: https://github.com/apache/spark/pull/32741#issuecomment-853543621 **[Test build #139266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139266/testReport)** for PR 32741 at commit

[GitHub] [spark] SparkQA commented on pull request #32751: [SPARK-35612][SQL] Support LZ4 compression in ORC data source

2021-06-02 Thread GitBox
SparkQA commented on pull request #32751: URL: https://github.com/apache/spark/pull/32751#issuecomment-853543583 **[Test build #139265 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139265/testReport)** for PR 32751 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-853543357 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139250/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32743: [SPARK-35396][CORE][FOLLOWUP] Free memory entry immediately

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32743: URL: https://github.com/apache/spark/pull/32743#issuecomment-853543360 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43780/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32759: [SPARK-35619][ML] Refactor LinearRegression - make huber support virtual centering

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32759: URL: https://github.com/apache/spark/pull/32759#issuecomment-853543354 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43779/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32758: [SPARK-35416][K8S][FOLLOWUP] Use Set instead of ArrayBuffer

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32758: URL: https://github.com/apache/spark/pull/32758#issuecomment-853543356 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43775/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
AmplabJenkins commented on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853543353 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43781/ --

[GitHub] [spark] SparkQA commented on pull request #32762: [SPARK-35081][DOCS] Add Data Source Option links to missing documents

2021-06-02 Thread GitBox
SparkQA commented on pull request #32762: URL: https://github.com/apache/spark/pull/32762#issuecomment-853543142 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43784/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32750: [SPARK-33696][BUILD][SQL] Upgrade built-in Hive to 2.3.9

2021-06-02 Thread GitBox
SparkQA commented on pull request #32750: URL: https://github.com/apache/spark/pull/32750#issuecomment-853541720 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43782/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon closed pull request #32757: [SPARK-35528][DOCS] Add more options at Data Source Options pages

2021-06-02 Thread GitBox
HyukjinKwon closed pull request #32757: URL: https://github.com/apache/spark/pull/32757 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HyukjinKwon commented on pull request #32757: [SPARK-35528][DOCS] Add more options at Data Source Options pages

2021-06-02 Thread GitBox
HyukjinKwon commented on pull request #32757: URL: https://github.com/apache/spark/pull/32757#issuecomment-853540348 tests passed at https://github.com/itholic/spark/actions/runs/901299055. Merged to master. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] HyukjinKwon closed pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
HyukjinKwon closed pull request #32760: URL: https://github.com/apache/spark/pull/32760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] viirya commented on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-02 Thread GitBox
viirya commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-853540267 Sorry for late. I will take look this soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HyukjinKwon commented on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
HyukjinKwon commented on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853540210 Thanks @itholic and @srowen. Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #32760: [SPARK-35620][BUILD][PYTHON] Remove documentation build in Python linter

2021-06-02 Thread GitBox
SparkQA commented on pull request #32760: URL: https://github.com/apache/spark/pull/32760#issuecomment-853538801 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43781/ --

[GitHub] [spark] SparkQA commented on pull request #32743: [SPARK-35396][CORE][FOLLOWUP] Free memory entry immediately

2021-06-02 Thread GitBox
SparkQA commented on pull request #32743: URL: https://github.com/apache/spark/pull/32743#issuecomment-853538561 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43780/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32759: [SPARK-35619][ML] Refactor LinearRegression - make huber support virtual centering

2021-06-02 Thread GitBox
SparkQA commented on pull request #32759: URL: https://github.com/apache/spark/pull/32759#issuecomment-853537654 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43779/ -- This is an automated message from the

[GitHub] [spark] sumeetgajjar commented on pull request #32114: [SPARK-35011][CORE] Avoid Block Manager registrations when StopExecutor msg is in-flight

2021-06-02 Thread GitBox
sumeetgajjar commented on pull request #32114: URL: https://github.com/apache/spark/pull/32114#issuecomment-853537413 Thank you @Ngone51, @mridulm and @attilapiros for your detailed reviews and insights. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] fornaix commented on a change in pull request #32751: [SPARK-35612][SQL] Support LZ4 compression in ORC data source

2021-06-02 Thread GitBox
fornaix commented on a change in pull request #32751: URL: https://github.com/apache/spark/pull/32751#discussion_r644460380 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala ## @@ -649,4 +649,14 @@ class OrcSourceSuite

[GitHub] [spark] fornaix commented on a change in pull request #32751: [SPARK-35612][SQL] Support LZ4 compression in ORC data source

2021-06-02 Thread GitBox
fornaix commented on a change in pull request #32751: URL: https://github.com/apache/spark/pull/32751#discussion_r644460235 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala ## @@ -314,6 +314,7 @@ private[orc] object OrcFileFormat extends

[GitHub] [spark] xuanyuanking commented on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-02 Thread GitBox
xuanyuanking commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-853535338 ``` It would be nice if we can do the same we did for reviewing the first PR; if remaining review comments are planned to be addressed via further PR, please raise the

[GitHub] [spark] HyukjinKwon commented on pull request #32758: [SPARK-35416][K8S][FOLLOWUP] Use Set instead of ArrayBuffer

2021-06-02 Thread GitBox
HyukjinKwon commented on pull request #32758: URL: https://github.com/apache/spark/pull/32758#issuecomment-853533179 I'll leave it to @mridulm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32726: [SPARK-35587][PYTHON][DOCS] Initial porting of Koalas documentation

2021-06-02 Thread GitBox
HyukjinKwon commented on a change in pull request #32726: URL: https://github.com/apache/spark/pull/32726#discussion_r644456412 ## File path: .github/workflows/build_and_test.yml ## @@ -381,6 +381,7 @@ jobs: # Jinja2 3.0.0+ causes error when building with Sphinx.

[GitHub] [spark] SparkQA commented on pull request #32758: [SPARK-35416][K8S][FOLLOWUP] Use Set instead of ArrayBuffer

2021-06-02 Thread GitBox
SparkQA commented on pull request #32758: URL: https://github.com/apache/spark/pull/32758#issuecomment-853532217 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43775/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-02 Thread GitBox
SparkQA removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-853482286 **[Test build #139250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139250/testReport)** for PR 32303 at commit

  1   2   3   4   5   6   7   8   >