[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982286223 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50211/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #34684: [SPARK-37442][SQL] InMemoryRelation statistics bug causing broadcast join failures with AQE enabled

2021-11-29 Thread GitBox
cloud-fan commented on a change in pull request #34684: URL: https://github.com/apache/spark/pull/34684#discussion_r758931580 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala ## @@ -237,7 +238,23 @@ case class

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982285782 **[Test build #145746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145746/testReport)** for PR 34746 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #34635: [SPARK-37205][YARN] Introduce a new config 'spark.yarn.am.tokenConfRegex' to support renewing delegation tokens in a multi-cluster envi

2021-11-29 Thread GitBox
dongjoon-hyun commented on pull request #34635: URL: https://github.com/apache/spark/pull/34635#issuecomment-982285975 Could you review this once more please, @mridulm ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] HyukjinKwon commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

2021-11-29 Thread GitBox
HyukjinKwon commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982292481 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon closed pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

2021-11-29 Thread GitBox
HyukjinKwon closed pull request #34717: URL: https://github.com/apache/spark/pull/34717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

2021-11-29 Thread GitBox
SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982294398 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50218/ -- This is an automated message from the Apache

[GitHub] [spark] dchvn commented on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

2021-11-29 Thread GitBox
dchvn commented on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-982303245 Ping @HyukjinKwon Thanks :D -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dchvn opened a new pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

2021-11-29 Thread GitBox
dchvn opened a new pull request #34750: URL: https://github.com/apache/spark/pull/34750 ### What changes were proposed in this pull request? Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled ### Why are the changes needed? identical

[GitHub] [spark] SparkQA commented on pull request #34656: [SPARK-37376][SQL] Introduce a new DataSource V2 interface HasPartitionKey

2021-11-29 Thread GitBox
SparkQA commented on pull request #34656: URL: https://github.com/apache/spark/pull/34656#issuecomment-982331806 **[Test build #145753 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145753/testReport)** for PR 34656 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #34656: [SPARK-37376][SQL] Introduce a new DataSource V2 interface HasPartitionKey

2021-11-29 Thread GitBox
dongjoon-hyun commented on pull request #34656: URL: https://github.com/apache/spark/pull/34656#issuecomment-982334173 Thank you, @sunchao ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dongjoon-hyun commented on pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

2021-11-29 Thread GitBox
dongjoon-hyun commented on pull request #34264: URL: https://github.com/apache/spark/pull/34264#issuecomment-982333916 BTW, in general, I agree with your demands and requirements in this PR. The only concerns are - the better backward compatibility - the visibility of these

[GitHub] [spark] SparkQA commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

2021-11-29 Thread GitBox
SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982344119 **[Test build #145738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145738/testReport)** for PR 34723 at commit

[GitHub] [spark] summaryzb commented on a change in pull request #34749: [SPARK-37493][CORE] show driver's gc time and duration time in executors page

2021-11-29 Thread GitBox
summaryzb commented on a change in pull request #34749: URL: https://github.com/apache/spark/pull/34749#discussion_r758995760 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala ## @@ -89,7 +89,44 @@ private[spark] class AppStatusStore( } else {

[GitHub] [spark] SparkQA commented on pull request #34656: [SPARK-37376][SQL] Introduce a new DataSource V2 interface HasPartitionKey

2021-11-29 Thread GitBox
SparkQA commented on pull request #34656: URL: https://github.com/apache/spark/pull/34656#issuecomment-982357268 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50225/ -- This is an automated message from the Apache

[GitHub] [spark] yangwwei commented on pull request #34672: [SPARK-37394][CORE] Skip registering with ESS if a customized shuffle manager is configured

2021-11-29 Thread GitBox
yangwwei commented on pull request #34672: URL: https://github.com/apache/spark/pull/34672#issuecomment-982363102 @mridulm , @attilapiros , @tgravescs could you pls help to review the changes again? Per @attilapiros 's suggestion, I have added a method in the ShuffleManager trait and

[GitHub] [spark] SparkQA commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

2021-11-29 Thread GitBox
SparkQA commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-982372984 **[Test build #145756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145756/testReport)** for PR 34750 at commit

[GitHub] [spark] yangwwei edited a comment on pull request #34672: [SPARK-37394][CORE] Skip registering with ESS if a customized shuffle manager is configured

2021-11-29 Thread GitBox
yangwwei edited a comment on pull request #34672: URL: https://github.com/apache/spark/pull/34672#issuecomment-982363102 @mridulm , @attilapiros , @tgravescs could you pls help to review the changes again? Per @attilapiros 's suggestion, I have added a method in the ShuffleManager

[GitHub] [spark] SparkQA commented on pull request #34635: [SPARK-37205][YARN] Introduce a new config 'spark.yarn.am.tokenConfRegex' to support renewing delegation tokens in a multi-cluster environmen

2021-11-29 Thread GitBox
SparkQA commented on pull request #34635: URL: https://github.com/apache/spark/pull/34635#issuecomment-982355152 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50224/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982354923 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145740/ -- This

[GitHub] [spark] sadikovi commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-29 Thread GitBox
sadikovi commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r75908 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala ## @@ -160,6 +169,17 @@ class CSVInferSchema(val

[GitHub] [spark] SparkQA removed a comment on pull request #34751: [SPARK-37497][K8S] Promote `ExecutorPods[PollingSnapshot|WatchSnapshot]Source` to DeveloperApi

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34751: URL: https://github.com/apache/spark/pull/34751#issuecomment-982354267 **[Test build #145755 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145755/testReport)** for PR 34751 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982354923 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145740/

[GitHub] [spark] LuciferYang commented on a change in pull request #34743: [SPARK-37488][CORE] When `TaskLocation` is `HDFSCacheTaskLocation` or `HostTaskLocation`, check if executor is alive on the h

2021-11-29 Thread GitBox
LuciferYang commented on a change in pull request #34743: URL: https://github.com/apache/spark/pull/34743#discussion_r758996701 ## File path: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ## @@ -291,6 +291,21 @@ class TaskSetManagerSuite

[GitHub] [spark] SparkQA commented on pull request #34635: [SPARK-37205][YARN] Introduce a new config 'spark.yarn.am.tokenConfRegex' to support renewing delegation tokens in a multi-cluster environmen

2021-11-29 Thread GitBox
SparkQA commented on pull request #34635: URL: https://github.com/apache/spark/pull/34635#issuecomment-982356662 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50223/ -- This is an automated message from the Apache

[GitHub] [spark] guiyanakuang commented on a change in pull request #34743: [SPARK-37488][CORE] When `TaskLocation` is `HDFSCacheTaskLocation` or `HostTaskLocation`, check if executor is alive on the

2021-11-29 Thread GitBox
guiyanakuang commented on a change in pull request #34743: URL: https://github.com/apache/spark/pull/34743#discussion_r759001312 ## File path: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ## @@ -291,6 +291,21 @@ class TaskSetManagerSuite

[GitHub] [spark] beliefer opened a new pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

2021-11-29 Thread GitBox
beliefer opened a new pull request #31847: URL: https://github.com/apache/spark/pull/31847 ### What changes were proposed in this pull request? Data Type Formatting Functions: `to_number` and `to_char` is very useful. There are some mainstream database support the syntax.

[GitHub] [spark] SparkQA commented on pull request #34750: [SPARK-37495][PYTHON] Skip identical index checking of Series.compare when config 'compute.eager_check' is disabled

2021-11-29 Thread GitBox
SparkQA commented on pull request #34750: URL: https://github.com/apache/spark/pull/34750#issuecomment-982370952 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50221/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34751: [SPARK-37497][K8S] Promote `ExecutorPods[PollingSnapshot|WatchSnapshot]Source` to DeveloperApi

2021-11-29 Thread GitBox
SparkQA commented on pull request #34751: URL: https://github.com/apache/spark/pull/34751#issuecomment-982364378 **[Test build #145755 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145755/testReport)** for PR 34751 at commit

[GitHub] [spark] tdg5 commented on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

2021-11-29 Thread GitBox
tdg5 commented on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-981800250 @gaborgsomogyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] tdg5 commented on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

2021-11-29 Thread GitBox
tdg5 commented on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-981800777 This is not a work in progress as far as I am concerned/aware, but I marked it as a WIP until @gaborgsomogyi can give it an initial thumbs up. -- This is an automated message

[GitHub] [spark] tdg5 removed a comment on pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

2021-11-29 Thread GitBox
tdg5 removed a comment on pull request #34745: URL: https://github.com/apache/spark/pull/34745#issuecomment-981800250 @gaborgsomogyi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] nicolasazrak commented on a change in pull request #34509: [SPARK-34521][PYTHON][SQL] Fix spark.createDataFrame when using pandas with StringDtype

2021-11-29 Thread GitBox
nicolasazrak commented on a change in pull request #34509: URL: https://github.com/apache/spark/pull/34509#discussion_r758535773 ## File path: python/pyspark/sql/pandas/serializers.py ## @@ -169,6 +169,8 @@ def create_array(s, t): elif

[GitHub] [spark] nicolasazrak commented on a change in pull request #34509: [SPARK-34521][PYTHON][SQL] Fix spark.createDataFrame when using pandas with StringDtype

2021-11-29 Thread GitBox
nicolasazrak commented on a change in pull request #34509: URL: https://github.com/apache/spark/pull/34509#discussion_r758535773 ## File path: python/pyspark/sql/pandas/serializers.py ## @@ -169,6 +169,8 @@ def create_array(s, t): elif

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981806797 **[Test build #145727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145727/testReport)** for PR 34746 at commit

[GitHub] [spark] gengliangwang opened a new pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

2021-11-29 Thread GitBox
gengliangwang opened a new pull request #34747: URL: https://github.com/apache/spark/pull/34747 ### What changes were proposed in this pull request? Show extra hint in the error message if analysis failed only with ANSI type coercion: ``` To fix the error, you might

[GitHub] [spark] gengliangwang commented on pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

2021-11-29 Thread GitBox
gengliangwang commented on pull request #34747: URL: https://github.com/apache/spark/pull/34747#issuecomment-981814923 cc @cloud-fan @entong -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] tdg5 commented on a change in pull request #34745: [WIP][SPARK-37391][SQL] JdbcConnectionProvider must indicate if it needs lock

2021-11-29 Thread GitBox
tdg5 commented on a change in pull request #34745: URL: https://github.com/apache/spark/pull/34745#discussion_r758551435 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/BasicConnectionProvider.scala ## @@ -48,4 +48,12 @@

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981818991 **[Test build #145725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145725/testReport)** for PR 34746 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #34733: [SPARK-36346][SQL][FOLLOWUP] Rename `withAllOrcReaders` to `withAllNativeOrcReaders`

2021-11-29 Thread GitBox
dongjoon-hyun commented on pull request #34733: URL: https://github.com/apache/spark/pull/34733#issuecomment-981821101 To @MaxGekk , as @cloud-fan mentioned, it's a big change and not a good approach because `SQL` module had batter be complete by itself. To @cloud-fan , yes, the

[GitHub] [spark] dongjoon-hyun closed pull request #34733: [SPARK-36346][SQL][FOLLOWUP] Rename `withAllOrcReaders` to `withAllNativeOrcReaders`

2021-11-29 Thread GitBox
dongjoon-hyun closed pull request #34733: URL: https://github.com/apache/spark/pull/34733 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] dongjoon-hyun commented on pull request #34733: [SPARK-36346][SQL][FOLLOWUP] Rename `withAllOrcReaders` to `withAllNativeOrcReaders`

2021-11-29 Thread GitBox
dongjoon-hyun commented on pull request #34733: URL: https://github.com/apache/spark/pull/34733#issuecomment-981821610 Thank you all! Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-29 Thread GitBox
dongjoon-hyun commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-981822819 #34733 is merged. Could you rebase this PR to the master branch, @beliefer ? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun commented on pull request #34723: [MINOR][SQL] Optimize some Orc test code

2021-11-29 Thread GitBox
dongjoon-hyun commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-981824965 BTW, is this PR include all instances like this, @beliefer ? Could you check other places too? -- This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981795246 **[Test build #145725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145725/testReport)** for PR 34746 at commit

[GitHub] [spark] SparkQA commented on pull request #34744: [SPARK-37454][SQL][FOLLOWUP] Time travel timestamp expression should support RuntimeReplaceable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34744: URL: https://github.com/apache/spark/pull/34744#issuecomment-981828148 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50196/ --

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981832733 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50193/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981833984 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50195/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34716: [SPARK-37468][SQL] Support ANSI intervals and TimestampNTZ for UnionEstimation

2021-11-29 Thread GitBox
SparkQA commented on pull request #34716: URL: https://github.com/apache/spark/pull/34716#issuecomment-981837233 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50194/ -- This is an automated message from the

[GitHub] [spark] viirya commented on a change in pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

2021-11-29 Thread GitBox
viirya commented on a change in pull request #34717: URL: https://github.com/apache/spark/pull/34717#discussion_r758572738 ## File path: python/docs/source/user_guide/sql/arrow_pandas.rst ## @@ -387,7 +387,7 @@ working with timestamps in ``pandas_udf``\s to get the best

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981844777 **[Test build #145727 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145727/testReport)** for PR 34746 at commit

[GitHub] [spark] c21 commented on pull request #34702: [SPARK-37455][SQL] Replace hash with sort aggregate if child is already sorted

2021-11-29 Thread GitBox
c21 commented on pull request #34702: URL: https://github.com/apache/spark/pull/34702#issuecomment-981846435 This PR is ready for review again thanks @cloud-fan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981845342 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50193/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34716: [SPARK-37468][SQL] Support ANSI intervals and TimestampNTZ for UnionEstimation

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34716: URL: https://github.com/apache/spark/pull/34716#issuecomment-981845336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50194/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981845333 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34744: [SPARK-37454][SQL][FOLLOWUP] Time travel timestamp expression should support RuntimeReplaceable

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34744: URL: https://github.com/apache/spark/pull/34744#issuecomment-981845337 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50196/

[GitHub] [spark] AmplabJenkins commented on pull request #34744: [SPARK-37454][SQL][FOLLOWUP] Time travel timestamp expression should support RuntimeReplaceable

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34744: URL: https://github.com/apache/spark/pull/34744#issuecomment-981845337 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50196/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34716: [SPARK-37468][SQL] Support ANSI intervals and TimestampNTZ for UnionEstimation

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34716: URL: https://github.com/apache/spark/pull/34716#issuecomment-981845336 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50194/ --

[GitHub] [spark] SparkQA removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981806797 **[Test build #145727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145727/testReport)** for PR 34746 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981845502 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145727/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981845342 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50193/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981845333 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145725/ -- This

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

2021-11-29 Thread GitBox
SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-981848687 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50197/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

2021-11-29 Thread GitBox
SparkQA commented on pull request #34747: URL: https://github.com/apache/spark/pull/34747#issuecomment-981849118 **[Test build #145728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145728/testReport)** for PR 34747 at commit

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-29 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981720533 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50190/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

2021-11-29 Thread GitBox
SparkQA commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-981722288 **[Test build #145718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145718/testReport)** for PR 34740 at commit

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-29 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981724319 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50192/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-29 Thread GitBox
SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-981726534 **[Test build #145709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145709/testReport)** for PR 34611 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-981583423 **[Test build #145718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145718/testReport)** for PR 34740 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34735: [SPARK-37481][Core] Fix disappearance of skipped stages after they retry

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34735: URL: https://github.com/apache/spark/pull/34735#issuecomment-981583332 **[Test build #145719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145719/testReport)** for PR 34735 at commit

[GitHub] [spark] SparkQA commented on pull request #34735: [SPARK-37481][Core] Fix disappearance of skipped stages after they retry

2021-11-29 Thread GitBox
SparkQA commented on pull request #34735: URL: https://github.com/apache/spark/pull/34735#issuecomment-981723374 **[Test build #145719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145719/testReport)** for PR 34735 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-981496480 **[Test build #145709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145709/testReport)** for PR 34611 at commit

[GitHub] [spark] SparkQA commented on pull request #34741: [SPARK-37463][SQL] Read/Write Timestamp ntz from/to Orc uses UTC time zone

2021-11-29 Thread GitBox
SparkQA commented on pull request #34741: URL: https://github.com/apache/spark/pull/34741#issuecomment-981733534 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50191/ -- This is an automated message from the Apache

[GitHub] [spark] srowen commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

2021-11-29 Thread GitBox
srowen commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-981732487 Looks fine; are there any more you know of? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] KevinAppelBofa commented on pull request #34709: [WIP][SPARK-37259] Add option to unwrap query to support CTE for MSSQL JDBC driver

2021-11-29 Thread GitBox
KevinAppelBofa commented on pull request #34709: URL: https://github.com/apache/spark/pull/34709#issuecomment-981732735 @akhalymon-cv thank you for working on the patch, I was able to test this and both the test queries and the actual temp and CTE queries are working. the item in the

[GitHub] [spark] AmplabJenkins commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981734229 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50189/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34735: [SPARK-37481][Core] Fix disappearance of skipped stages after they retry

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34735: URL: https://github.com/apache/spark/pull/34735#issuecomment-981734225 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145719/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981734223 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50190/ --

[GitHub] [spark] KevinAppelBofa edited a comment on pull request #34709: [WIP][SPARK-37259] Add option to unwrap query to support CTE for MSSQL JDBC driver

2021-11-29 Thread GitBox
KevinAppelBofa edited a comment on pull request #34709: URL: https://github.com/apache/spark/pull/34709#issuecomment-981732735 @akhalymon-cv thank you for working on the patch, I was able to test this and both the test queries and the actual temp and CTE queries are working. the item in

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-981734224 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145709/ -- This

[GitHub] [spark] srowen commented on pull request #34739: [SPARK-37484][CORE][SQL] Replace `get` and `getOrElse` with `getOrElse`

2021-11-29 Thread GitBox
srowen commented on pull request #34739: URL: https://github.com/apache/spark/pull/34739#issuecomment-981734041 Also fine; any others? I want to make sure we don't do this module by module -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] cloud-fan commented on a change in pull request #34729: [SPARK-37475][SQL] Add scale parameter to floor and ceil functions

2021-11-29 Thread GitBox
cloud-fan commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r758462752 ## File path: sql/core/src/test/resources/sql-tests/results/operators.sql.out ## @@ -357,87 +357,87 @@ struct -- !query select ceiling(0) -- !query

[GitHub] [spark] AmplabJenkins commented on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-981734222 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145718/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-981734224 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145709/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981734229 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50189/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34735: [SPARK-37481][Core] Fix disappearance of skipped stages after they retry

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34735: URL: https://github.com/apache/spark/pull/34735#issuecomment-981734225 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145719/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981734223 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50190/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34740: [SPARK-37485][CORE][SQL] Replace `map` with expressions which produce no result with `foreach`

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34740: URL: https://github.com/apache/spark/pull/34740#issuecomment-981734222 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145718/

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981735413 **[Test build #145723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145723/testReport)** for PR 34578 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #34729: [SPARK-37475][SQL] Add scale parameter to floor and ceil functions

2021-11-29 Thread GitBox
cloud-fan commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r758464183 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala ## @@ -243,40 +243,26 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #34729: [SPARK-37475][SQL] Add scale parameter to floor and ceil functions

2021-11-29 Thread GitBox
cloud-fan commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r758463687 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala ## @@ -243,40 +243,26 @@ case class

[GitHub] [spark] AmplabJenkins commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981737837 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145723/ -- This

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981737798 **[Test build #145723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145723/testReport)** for PR 34578 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981737837 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145723/

[GitHub] [spark] SparkQA removed a comment on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-981735413 **[Test build #145723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145723/testReport)** for PR 34578 at commit

[GitHub] [spark] MaxGekk commented on a change in pull request #34716: [SPARK-37468][SQL] Support ANSI intervals and TimestampNTZ for UnionEstimation

2021-11-29 Thread GitBox
MaxGekk commented on a change in pull request #34716: URL: https://github.com/apache/spark/pull/34716#discussion_r758469434 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -50,13 +50,23 @@

[GitHub] [spark] SparkQA commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-29 Thread GitBox
SparkQA commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-981741230 **[Test build #145710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145710/testReport)** for PR 34367 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-29 Thread GitBox
SparkQA removed a comment on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-981496871 **[Test build #145710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145710/testReport)** for PR 34367 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-29 Thread GitBox
AmplabJenkins removed a comment on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-981743370 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145710/

[GitHub] [spark] AmplabJenkins commented on pull request #34367: [SPARK-37099][SQL] Impl a rank-based filter to optimize top-k computation

2021-11-29 Thread GitBox
AmplabJenkins commented on pull request #34367: URL: https://github.com/apache/spark/pull/34367#issuecomment-981743370 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145710/ -- This

<    3   4   5   6   7   8   9   >