[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r469043792 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -242,8 +242,10 @@ class DecommissionWorkerSuite

[GitHub] [spark] c21 commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672644707 > I have a crazy idea: we can make the value length negative to indicate it's matched. Then the "matched bit" takes no extra memory. But I'm not sure how hard it is to update BytesT

[GitHub] [spark] cloud-fan commented on pull request #29404: [SPARK-32018][SQL][FollowUp][3.0] Throw exception on decimal value overflow of sum aggregation

cloud-fan commented on pull request #29404: URL: https://github.com/apache/spark/pull/29404#issuecomment-672642993 > Is this saying the isEmpty in Sum cannot be backported to branch-3.0? I think so. 3.0 doesn't even have mechanisms to detect incompatible state store format, and it ma

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

HyukjinKwon edited a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-672642497 FYI, I saw the flakiness of the test `org.apache.spark.deploy.DecommissionWorkerSuite.decommission workers ensure that fetch failures lead to rerun` here, https://am

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

HyukjinKwon edited a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-672642497 FYI, I saw the flakiness of the test here, https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127357/testReport/org.apache.spark.deploy/DecommissionWor

[GitHub] [spark] HyukjinKwon commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

HyukjinKwon commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-672642497 FYI, I saw the flakiness of the test here, https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127357/testReport/org.apache.spark.deploy/DecommissionWorkerSuit

[GitHub] [spark] bogdanghit commented on pull request #28991: [SPARK-26533][SQL][test-hive1.2][test-hadoop2.7] Support query auto timeout cancel on thriftserver

bogdanghit commented on pull request #28991: URL: https://github.com/apache/spark/pull/28991#issuecomment-672642527 Made a pass. I would reduce the unnecessary changes in hive-1.2 and avoid collapsing the two `newExecuteStatementOperation` methods into one. ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672641802 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672641802 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] bogdanghit commented on a change in pull request #28991: [SPARK-26533][SQL][test-hive1.2][test-hadoop2.7] Support query auto timeout cancel on thriftserver

bogdanghit commented on a change in pull request #28991: URL: https://github.com/apache/spark/pull/28991#discussion_r469038005 ## File path: sql/hive-thriftserver/v1.2/src/main/java/org/apache/hive/service/cli/operation/OperationManager.java ## @@ -87,7 +87,7 @@ private void i

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

AmplabJenkins removed a comment on pull request #29412: URL: https://github.com/apache/spark/pull/29412#issuecomment-672641495 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] bogdanghit commented on a change in pull request #28991: [SPARK-26533][SQL][test-hive1.2][test-hadoop2.7] Support query auto timeout cancel on thriftserver

bogdanghit commented on a change in pull request #28991: URL: https://github.com/apache/spark/pull/28991#discussion_r469037661 ## File path: sql/hive-thriftserver/v2.3/src/main/java/org/apache/hive/service/cli/operation/OperationManager.java ## @@ -86,20 +86,15 @@ private void

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672641442 **[Test build #127369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127369/testReport)** for PR 28841 at commit [`3ded6d0`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

AmplabJenkins commented on pull request #29412: URL: https://github.com/apache/spark/pull/29412#issuecomment-672641495 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] cloud-fan closed pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

cloud-fan closed pull request #29383: URL: https://github.com/apache/spark/pull/29383 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] c21 commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672640787 @cloud-fan, @agrawaldevesh, @maropu and @viirya - updated the PR with latest proposed change (I still need to add unit test for `BytesToBytesMap` and `HashedRelation`, but the added

[GitHub] [spark] cloud-fan commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

cloud-fan commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-672640923 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] SparkQA removed a comment on pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

SparkQA removed a comment on pull request #29412: URL: https://github.com/apache/spark/pull/29412#issuecomment-672439150 **[Test build #127353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127353/testReport)** for PR 29412 at commit [`962bcc6`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

SparkQA commented on pull request #29412: URL: https://github.com/apache/spark/pull/29412#issuecomment-672640391 **[Test build #127353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127353/testReport)** for PR 29412 at commit [`962bcc6`](https://github.co

[GitHub] [spark] cloud-fan commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672640531 let's compare the overheads of these 2 approaches. The current approach (put "matched bit" in the value payload): 1. needs to do a project over the build side rows to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

AmplabJenkins removed a comment on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672639695 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/127

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

AmplabJenkins removed a comment on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672639689 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

AmplabJenkins commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672639689 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

SparkQA removed a comment on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672557252 **[Test build #127357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127357/testReport)** for PR 29411 at commit [`cc006a3`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

AmplabJenkins commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672638799 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

SparkQA commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672639030 **[Test build #127357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127357/testReport)** for PR 29411 at commit [`cc006a3`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

AmplabJenkins removed a comment on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672638799 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

SparkQA commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-672638259 **[Test build #127368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127368/testReport)** for PR 29342 at commit [`bd7261e`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

viirya commented on a change in pull request #29412: URL: https://github.com/apache/spark/pull/29412#discussion_r469033829 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFiltersBase.scala ## @@ -67,18 +65,12 @@ trait OrcFiltersBase {

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

HyukjinKwon commented on a change in pull request #29412: URL: https://github.com/apache/spark/pull/29412#discussion_r469030506 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFiltersBase.scala ## @@ -67,18 +65,12 @@ trait OrcFiltersBase

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r469030334 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -242,8 +242,10 @@ class DecommissionWorkerSuite

[GitHub] [spark] viirya commented on pull request #29404: [SPARK-32018][SQL][FollowUp][3.0] Throw exception on decimal value overflow of sum aggregation

viirya commented on pull request #29404: URL: https://github.com/apache/spark/pull/29404#issuecomment-672633918 And looks like we don't have many choices. I think correctness should be considered first. This is an automated

[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

yanxiaole commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-672630663 the filter is called after the assignment of variable `stale`, but the race condition is happened in the assignment, the `asScala.toList`. ---

[GitHub] [spark] AmplabJenkins commented on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

AmplabJenkins commented on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-672630113 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

AmplabJenkins removed a comment on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-672629246 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

AmplabJenkins commented on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-672629246 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SaurabhChawla100 opened a new pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

SaurabhChawla100 opened a new pull request #29413: URL: https://github.com/apache/spark/pull/29413 ### What changes were proposed in this pull request? There are scenarios where we have seen the event drop in spark, resulting in the inconsistent state for the spark Application(s

[GitHub] [spark] cloud-fan closed pull request #29401: [SPARK-32400][SQL] Improve test coverage of HiveScriptTransformationExec

cloud-fan closed pull request #29401: URL: https://github.com/apache/spark/pull/29401 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #29401: [SPARK-32400][SQL] Improve test coverage of HiveScriptTransformationExec

cloud-fan commented on pull request #29401: URL: https://github.com/apache/spark/pull/29401#issuecomment-672625176 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] cloud-fan commented on pull request #29328: [SPARK-32516][SQL] 'path' option cannot co-exist with load()'s path parameters

cloud-fan commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-672624029 LGTM except one question: do we still need this fix? https://github.com/apache/spark/pull/29328/files#r468330253 -

[GitHub] [spark] viirya commented on pull request #29404: [SPARK-32018][SQL][FollowUp][3.0] Throw exception on decimal value overflow of sum aggregation

viirya commented on pull request #29404: URL: https://github.com/apache/spark/pull/29404#issuecomment-672622923 > However, the aggregation buffer is written in steaming checkpoints. Thus, we can't change to aggregation buffer to resolve the issue. Is this saying the `isEmpty` in `Sum

[GitHub] [spark] cloud-fan commented on pull request #29404: [SPARK-32018][SQL][FollowUp][3.0] Throw exception on decimal value overflow of sum aggregation

cloud-fan commented on pull request #29404: URL: https://github.com/apache/spark/pull/29404#issuecomment-672601482 This adds perf overhead as we need to check overflow after each Add operation, while the master branch only checks overflow at the end because we have an extra agg buffer slot

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29403: [SPARK-32585][SQL] Support scala enumeration in ScalaReflection

AmplabJenkins removed a comment on pull request #29403: URL: https://github.com/apache/spark/pull/29403#issuecomment-672600993 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] viirya commented on a change in pull request #29412: [SPARK-25557][SQL][Followup] Remove CaseInsensitiveMap in OrcFiltersBase

viirya commented on a change in pull request #29412: URL: https://github.com/apache/spark/pull/29412#discussion_r469015649 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFiltersBase.scala ## @@ -67,18 +65,12 @@ trait OrcFiltersBase {

[GitHub] [spark] AmplabJenkins commented on pull request #29403: [SPARK-32585][SQL] Support scala enumeration in ScalaReflection

AmplabJenkins commented on pull request #29403: URL: https://github.com/apache/spark/pull/29403#issuecomment-672600993 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] zhouyejoe commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

zhouyejoe commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-672598759 Hi, @yanxiaole. I double checked the codes in checkForLogs(). I think SPARK-29043 does actually handles the race condition by filtering out the stale.filterNot(isProcessing),

[GitHub] [spark] SparkQA commented on pull request #29403: [SPARK-32585][SQL] Support scala enumeration in ScalaReflection

SparkQA commented on pull request #29403: URL: https://github.com/apache/spark/pull/29403#issuecomment-672596737 **[Test build #127352 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127352/testReport)** for PR 29403 at commit [`462b0f7`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29403: [SPARK-32585][SQL] Support scala enumeration in ScalaReflection

SparkQA removed a comment on pull request #29403: URL: https://github.com/apache/spark/pull/29403#issuecomment-672414635 **[Test build #127352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127352/testReport)** for PR 29403 at commit [`462b0f7`](https://gi

[GitHub] [spark] mridulm commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

mridulm commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672589797 pip issues, jira update ended up failing ... This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] asfgit closed pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

asfgit closed pull request #29411: URL: https://github.com/apache/spark/pull/29411 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

HyukjinKwon edited a comment on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672579570 @mridulm, I remember you have to manually be added to have that Jenkins permission IIRC. The same thing happened to me too before. I will send an email offline to you

[GitHub] [spark] mridulm commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

mridulm commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672581541 Thx @venkata91, merging to master This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [spark] HyukjinKwon commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

HyukjinKwon commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672579570 @mridulm, I remember you have to manually be added to the Jenkins admin. The same thing happened to me too before. I will send an email offline to you to invite. -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

AmplabJenkins removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672576899 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/127

[GitHub] [spark] mridulm commented on pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

mridulm commented on pull request #29411: URL: https://github.com/apache/spark/pull/29411#issuecomment-672578174 @HyukjinKwon Any idea why jenkins does not want to listen to me ? :-) https://github.com/apache/spark/pull/29411#issuecomment-672367418 --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

AmplabJenkins removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672576869 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

SparkQA removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672567923 **[Test build #127366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127366/testReport)** for PR 29410 at commit [`e93b4e1`](https://gi

[GitHub] [spark] LuciferYang commented on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

LuciferYang commented on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672576953 @srowen After [5328d33](https://github.com/apache/spark/pull/29370/commits/5328d33cc0fe138a5d1de9c73285ef262209e6dc), I think that there are no more similar problems that c

[GitHub] [spark] AmplabJenkins commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

AmplabJenkins commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672576869 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

SparkQA commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672575936 **[Test build #127366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127366/testReport)** for PR 29410 at commit [`e93b4e1`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

AmplabJenkins removed a comment on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-672570802 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

AmplabJenkins commented on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-672570802 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] venkata91 commented on a change in pull request #29411: [SPARK-32596][CORE] Clear Ivy resolution files as part of finally block

venkata91 commented on a change in pull request #29411: URL: https://github.com/apache/spark/pull/29411#discussion_r469002807 ## File path: core/src/test/scala/org/apache/spark/deploy/SparkSubmitUtilsSuite.scala ## @@ -79,7 +79,7 @@ class SparkSubmitUtilsSuite extends SparkFun

[GitHub] [spark] SparkQA removed a comment on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

SparkQA removed a comment on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-672393733 **[Test build #127351 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127351/testReport)** for PR 29322 at commit [`ad111ba`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29322: [SPARK-32511][SQL] Add dropFields method to Column class

SparkQA commented on pull request #29322: URL: https://github.com/apache/spark/pull/29322#issuecomment-672570103 **[Test build #127351 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127351/testReport)** for PR 29322 at commit [`ad111ba`](https://github.co

[GitHub] [spark] HyukjinKwon commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

HyukjinKwon commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-672569879 Okay, GitHub team suggested a scenario that it could work out of the box. It's a bit complicated then I thought but I will try. Yes, it is a nice-to-have but I think it is v

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

HyukjinKwon commented on a change in pull request #29410: URL: https://github.com/apache/spark/pull/29410#discussion_r468999216 ## File path: python/docs/source/getting_started/index.rst ## @@ -20,3 +20,13 @@ Getting Started === +**PySpark** is the Python API fo

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

AmplabJenkins removed a comment on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672568214 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

AmplabJenkins commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672568232 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

AmplabJenkins removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672568232 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

AmplabJenkins commented on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672568214 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] rohitmishr1484 edited a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

rohitmishr1484 edited a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672391444 Hi @HyukjinKwon, I was not sure how to add you as a Reviewer for this Pull request, thus adding this comment. I would like to mention a few points: 1

[GitHub] [spark] SparkQA commented on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

SparkQA commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672567923 **[Test build #127366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127366/testReport)** for PR 29410 at commit [`e93b4e1`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

SparkQA commented on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672567955 **[Test build #127367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127367/testReport)** for PR 29370 at commit [`75d85ee`](https://github.com

[GitHub] [spark] rohitmishr1484 edited a comment on pull request #29410: [SPARK-32180][PYTHON][DOCS] Installation page in Getting Started in PySpark documentation

rohitmishr1484 edited a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672391444 Hi @HyukjinKwon, I was not sure how to add you as a Reviewer for this Pull request, thus adding this comment. I would like to mention a few points: 1

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29410: [SPARK-32180][PYSPARK][DOCS] Getting started-Installation guide for pyspark doc

AmplabJenkins removed a comment on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672278185 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] LuciferYang edited a comment on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

LuciferYang edited a comment on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672566297 The main change of Address [5328d33](https://github.com/apache/spark/pull/29370/commits/5328d33cc0fe138a5d1de9c73285ef262209e6dc) is add a sort - Add a `sorted

[GitHub] [spark] HyukjinKwon commented on pull request #29410: [SPARK-32180][PYSPARK][DOCS] Getting started-Installation guide for pyspark doc

HyukjinKwon commented on pull request #29410: URL: https://github.com/apache/spark/pull/29410#issuecomment-672566533 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [spark] HyukjinKwon closed pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

HyukjinKwon closed pull request #29409: URL: https://github.com/apache/spark/pull/29409 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672566390 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on pull request #29409: [SPARK-32594][SQL] Fix serialization of dates inserted to Hive tables

HyukjinKwon commented on pull request #29409: URL: https://github.com/apache/spark/pull/29409#issuecomment-672566456 Merged to master and branch-3.0. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672566390 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] LuciferYang commented on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

LuciferYang commented on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672566297 The main change of Address [5328d33](https://github.com/apache/spark/pull/29370/commits/5328d33cc0fe138a5d1de9c73285ef262209e6dc) is add a sort - Add a `sorted` to en

[GitHub] [spark] cchighman commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

cchighman commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672566232 @maropu Everything should be updated as requested. I appreciate your thorough review and believe we arrived at a better result in the process. -

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672566066 **[Test build #127365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127365/testReport)** for PR 28841 at commit [`54dd3cf`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

AmplabJenkins removed a comment on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672564475 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

AmplabJenkins commented on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672564475 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

SparkQA commented on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672564201 **[Test build #127364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127364/testReport)** for PR 29370 at commit [`5328d33`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

SparkQA commented on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672564174 **[Test build #127363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127363/testReport)** for PR 29406 at commit [`e2edd36`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

AmplabJenkins removed a comment on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672562750 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672562791 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

AmplabJenkins removed a comment on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672562761 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon closed pull request #29405: [SPARK-32586][SQL] Fix NumberFormatException error message when ansi is enabled

HyukjinKwon closed pull request #29405: URL: https://github.com/apache/spark/pull/29405 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #29406: [SPARK-32352][SQL] Partially push down support data filter if it mixed in partition filters

AmplabJenkins commented on pull request #29406: URL: https://github.com/apache/spark/pull/29406#issuecomment-672562761 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672562791 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #29370: [SPARK-32526][SQL]Fix some test cases of `sql/catalyst` module in scala 2.13

AmplabJenkins commented on pull request #29370: URL: https://github.com/apache/spark/pull/29370#issuecomment-672562750 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HyukjinKwon commented on pull request #29405: [SPARK-32586][SQL] Fix NumberFormatException error message when ansi is enabled

HyukjinKwon commented on pull request #29405: URL: https://github.com/apache/spark/pull/29405#issuecomment-672562701 Merged to master and branch-3.0. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-672562400 **[Test build #127362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127362/testReport)** for PR 28841 at commit [`50ebdfc`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins removed a comment on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672561572 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/127

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins removed a comment on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-672561566 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

  1   2   3   4   5   6   7   >