[GitHub] [spark] SparkQA commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773855770 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39498/ ---

[GitHub] [spark] SparkQA commented on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

SparkQA commented on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773851149 **[Test build #134920 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134920/testReport)** for PR 31483 at commit [`3a28304`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

SparkQA removed a comment on pull request #31471: URL: https://github.com/apache/spark/pull/31471#issuecomment-773743688 **[Test build #134899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134899/testReport)** for PR 31471 at commit [`ff2cce2`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

SparkQA commented on pull request #31471: URL: https://github.com/apache/spark/pull/31471#issuecomment-773848350 **[Test build #134899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134899/testReport)** for PR 31471 at commit [`ff2cce2`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773847254 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39498/ -

[GitHub] [spark] SparkQA commented on pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

SparkQA commented on pull request #31484: URL: https://github.com/apache/spark/pull/31484#issuecomment-773846745 **[Test build #134916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134916/testReport)** for PR 31484 at commit [`3515e38`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf

SparkQA commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773846744 **[Test build #134917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134917/testReport)** for PR 31482 at commit [`c9d2248`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31472: [SPARK-34356][ML] OVR transform fix potential column conflict

SparkQA commented on pull request #31472: URL: https://github.com/apache/spark/pull/31472#issuecomment-773845767 **[Test build #134918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134918/testReport)** for PR 31472 at commit [`02725b0`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

AmplabJenkins removed a comment on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773845253 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134898/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

AmplabJenkins commented on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773845409 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

AmplabJenkins commented on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773845253 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134898/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

SparkQA removed a comment on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773742165 **[Test build #134898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134898/testReport)** for PR 31477 at commit [`e2b2a81`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

SparkQA commented on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773844202 **[Test build #134898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134898/testReport)** for PR 31477 at commit [`e2b2a81`](https://github.co

[GitHub] [spark] maropu commented on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

maropu commented on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773843549 For reviews, could you put the screenshot of the updated doc in the PR description? This is an automated message

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

AmplabJenkins removed a comment on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773839682 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] maropu commented on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

maropu commented on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773843320 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan closed pull request #31478: [SPARK-34371][SQL][TESTS] Run the datetime rebasing tests for Parquet datasource v1 and v2

cloud-fan closed pull request #31478: URL: https://github.com/apache/spark/pull/31478 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] attilapiros commented on a change in pull request #31133: [SPARK-26836][SQL] Supporting Avro schema evolution for partitioned Hive tables with "avro.schema.literal"

attilapiros commented on a change in pull request #31133: URL: https://github.com/apache/spark/pull/31133#discussion_r570761922 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala ## @@ -1883,6 +1883,58 @@ class HiveDDLSuite } }

[GitHub] [spark] cloud-fan commented on pull request #31478: [SPARK-34371][SQL][TESTS] Run the datetime rebasing tests for Parquet datasource v1 and v2

cloud-fan commented on pull request #31478: URL: https://github.com/apache/spark/pull/31478#issuecomment-773842135 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] SparkQA commented on pull request #31133: [SPARK-26836][SQL] Supporting Avro schema evolution for partitioned Hive tables with "avro.schema.literal"

SparkQA commented on pull request #31133: URL: https://github.com/apache/spark/pull/31133#issuecomment-773840372 **[Test build #134919 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134919/testReport)** for PR 31133 at commit [`20dc167`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

AmplabJenkins commented on pull request #31483: URL: https://github.com/apache/spark/pull/31483#issuecomment-773839682 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AmplabJenkins removed a comment on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773839225 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39494/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally

AmplabJenkins removed a comment on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773839228 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

AmplabJenkins removed a comment on pull request #31204: URL: https://github.com/apache/spark/pull/31204#issuecomment-773839233 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39495/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31479: [SPARK-34373][SQL] HiveThriftServer2 startWithContext may hang with a race issue

AmplabJenkins removed a comment on pull request #31479: URL: https://github.com/apache/spark/pull/31479#issuecomment-773839231 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39486/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

AmplabJenkins removed a comment on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773839235 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39493/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins removed a comment on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773839230 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134903/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins commented on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773839230 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134903/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

AmplabJenkins commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773839235 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39493/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AmplabJenkins commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773839225 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39494/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may caus

AmplabJenkins commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773839238 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

AmplabJenkins commented on pull request #31204: URL: https://github.com/apache/spark/pull/31204#issuecomment-773839233 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39495/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31479: [SPARK-34373][SQL] HiveThriftServer2 startWithContext may hang with a race issue

AmplabJenkins commented on pull request #31479: URL: https://github.com/apache/spark/pull/31479#issuecomment-773839231 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39486/ -

[GitHub] [spark] LuciferYang opened a new pull request #31484: [SPARK-34374][SQL] Use standard methods to extract keys or values from a Map.

LuciferYang opened a new pull request #31484: URL: https://github.com/apache/spark/pull/31484 ### What changes were proposed in this pull request? Use standard methods to extract `keys` or `values` from a `Map`, it's semantically consistent and use the `DefaultKeySet` and `DefaultValue

[GitHub] [spark] SparkQA commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf

SparkQA commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773834171 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39496/ ---

[GitHub] [spark] Eric-Lemmon opened a new pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

Eric-Lemmon opened a new pull request #31483: URL: https://github.com/apache/spark/pull/31483 ### What changes were proposed in this pull request? Documentation for `SparkSession.conf.isModifiable` is missing from the Python API site, so I included the rst file for `pyspark.sql.conf.Run

[GitHub] [spark] SparkQA commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf

SparkQA commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773832211 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39496/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

SparkQA removed a comment on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773777161 **[Test build #134903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134903/testReport)** for PR 31480 at commit [`50e8d73`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

SparkQA commented on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773827836 **[Test build #134903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134903/testReport)** for PR 31480 at commit [`50e8d73`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31479: [SPARK-34373][SQL] HiveThriftServer2 startWithContext may hang with a race issue

SparkQA commented on pull request #31479: URL: https://github.com/apache/spark/pull/31479#issuecomment-773827300 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39486/ ---

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773825728 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39493/ ---

[GitHub] [spark] SparkQA commented on pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

SparkQA commented on pull request #31204: URL: https://github.com/apache/spark/pull/31204#issuecomment-773824407 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39495/ ---

[GitHub] [spark] SparkQA commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773824361 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39494/ ---

[GitHub] [spark] SparkQA commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773823173 **[Test build #134915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134915/testReport)** for PR 30957 at commit [`8dec5a1`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #31482: [SPARK-34346][CORE][SQL][3.1] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may caus

dongjoon-hyun commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773823111 Thank you for making a backport, @yaooqinn . This is an automated message from the Apache Git Service. To

[GitHub] [spark] zhengruifeng commented on a change in pull request #31472: [SPARK-34356][ML] OVR transform fix potential column conflict

zhengruifeng commented on a change in pull request #31472: URL: https://github.com/apache/spark/pull/31472#discussion_r570745838 ## File path: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala ## @@ -185,71 +185,56 @@ final class OneVsRestModel private[ml

[GitHub] [spark] SparkQA removed a comment on pull request #31482: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause p

SparkQA removed a comment on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773819012 **[Test build #134913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134913/testReport)** for PR 31482 at commit [`905405f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31482: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf regr

SparkQA commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773820019 **[Test build #134913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134913/testReport)** for PR 31482 at commit [`905405f`](https://github.co

[GitHub] [spark] zhengruifeng commented on a change in pull request #31472: [SPARK-34356][ML] OVR transform fix potential column conflict

zhengruifeng commented on a change in pull request #31472: URL: https://github.com/apache/spark/pull/31472#discussion_r570744818 ## File path: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala ## @@ -185,71 +185,56 @@ final class OneVsRestModel private[ml

[GitHub] [spark] zhengruifeng commented on a change in pull request #31472: [SPARK-34356][ML] OVR transform fix potential column conflict

zhengruifeng commented on a change in pull request #31472: URL: https://github.com/apache/spark/pull/31472#discussion_r570744758 ## File path: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala ## @@ -185,71 +185,56 @@ final class OneVsRestModel private[ml

[GitHub] [spark] zhengruifeng commented on a change in pull request #31472: [SPARK-34356][ML] OVR transform fix potential column conflict

zhengruifeng commented on a change in pull request #31472: URL: https://github.com/apache/spark/pull/31472#discussion_r570744185 ## File path: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala ## @@ -185,71 +185,56 @@ final class OneVsRestModel private[ml

[GitHub] [spark] SparkQA commented on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

SparkQA commented on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773819000 **[Test build #134914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134914/testReport)** for PR 31477 at commit [`90f9b44`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31482: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf regr

SparkQA commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773819012 **[Test build #134913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134913/testReport)** for PR 31482 at commit [`905405f`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AngersZh commented on a change in pull request #30957: URL: https://github.com/apache/spark/pull/30957#discussion_r570743087 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -47,7 +47,13 @@ trait BaseScriptTransf

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AngersZh commented on a change in pull request #30957: URL: https://github.com/apache/spark/pull/30957#discussion_r570743193 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala ## @@ -471,6 +473,100 @@ abstract class Bas

[GitHub] [spark] MaxGekk commented on pull request #31478: [SPARK-34371][SQL][TESTS] Run the datetime rebasing tests for Parquet datasource v1 and v2

MaxGekk commented on pull request #31478: URL: https://github.com/apache/spark/pull/31478#issuecomment-773816713 @cloud-fan @gengliangwang @HyukjinKwon Could you review this PR. This is an automated message from the Apache Gi

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

HyukjinKwon commented on a change in pull request #31466: URL: https://github.com/apache/spark/pull/31466#discussion_r570740350 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ## @@ -566,7 +563,14 @@ class SQLQueryTestSuite extends QueryTest w

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

AmplabJenkins removed a comment on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773813065 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134902/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31481: [SQL][MINOR][TEST][3.1] Re-enable some DS v2 char/varchar test

AmplabJenkins removed a comment on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773813067 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39492/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

AmplabJenkins removed a comment on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773813062 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins removed a comment on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773813064 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39485/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31437: [SPARK-34329][YARN] When hit ApplicationAttemptNotFoundException, we can't just stop app for all case

AmplabJenkins removed a comment on pull request #31437: URL: https://github.com/apache/spark/pull/31437#issuecomment-773813066 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39490/

[GitHub] [spark] AmplabJenkins commented on pull request #31481: [SQL][MINOR][TEST][3.1] Re-enable some DS v2 char/varchar test

AmplabJenkins commented on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773813067 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39492/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

AmplabJenkins commented on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773813062 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #31437: [SPARK-34329][YARN] When hit ApplicationAttemptNotFoundException, we can't just stop app for all case

AmplabJenkins commented on pull request #31437: URL: https://github.com/apache/spark/pull/31437#issuecomment-773813066 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39490/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins commented on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773813064 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39485/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

AmplabJenkins commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773813065 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134902/ -

[GitHub] [spark] SparkQA commented on pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

SparkQA commented on pull request #31204: URL: https://github.com/apache/spark/pull/31204#issuecomment-773812394 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39495/ -

[GitHub] [spark] SparkQA commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773812144 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39494/ -

[GitHub] [spark] yaooqinn commented on pull request #31482: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf reg

yaooqinn commented on pull request #31482: URL: https://github.com/apache/spark/pull/31482#issuecomment-773812166 cc @cloud-fan @maropu @HyukjinKwon here is the backport PR for 3.1, thanks This is an automated message from th

[GitHub] [spark] yaooqinn opened a new pull request #31482: [SPARK-34346][CORE][SQL] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may cause perf reg

yaooqinn opened a new pull request #31482: URL: https://github.com/apache/spark/pull/31482 In many real-world cases, when interacting with hive catalog through Spark SQL, users may just share the `hive-site.xml` for their hive jobs and make a copy to `SPARK_HOME`/conf w/o modificat

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773810259 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39493/ -

[GitHub] [spark] SparkQA commented on pull request #31481: [SQL][MINOR][TEST] Re-enable some DS v2 char/varchar test

SparkQA commented on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773807844 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39492/ ---

[GitHub] [spark] yaooqinn commented on pull request #31481: [SQL][MINOR][TEST] Re-enable some DS v2 char/varchar test

yaooqinn commented on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773807726 BTW, add [3.1] to the pr title This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [spark] SparkQA commented on pull request #31481: [SQL][MINOR][TEST] Re-enable some DS v2 char/varchar test

SparkQA commented on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773806209 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39492/ -

[GitHub] [spark] maropu commented on a change in pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

maropu commented on a change in pull request #30957: URL: https://github.com/apache/spark/pull/30957#discussion_r570731954 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -47,7 +47,13 @@ trait BaseScriptTransformati

[GitHub] [spark] maropu closed pull request #31456: [SPARK-34343][SQL][TESTS] Add missing test for some non-array types in PostgreSQL

maropu closed pull request #31456: URL: https://github.com/apache/spark/pull/31456 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] maropu commented on pull request #31456: [SPARK-34343][SQL][TESTS] Add missing test for some non-array types in PostgreSQL

maropu commented on pull request #31456: URL: https://github.com/apache/spark/pull/31456#issuecomment-773798533 Thanks! Merged to master. This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] yaooqinn commented on pull request #31481: [MINOR][TEST] Re-enable some DS v2 char/varchar test

yaooqinn commented on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773798240 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] SparkQA commented on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

SparkQA commented on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773797582 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39487/ ---

[GitHub] [spark] SparkQA commented on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

SparkQA commented on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-773797320 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39485/ ---

[GitHub] [spark] SparkQA removed a comment on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

SparkQA removed a comment on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773753068 **[Test build #134902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134902/testReport)** for PR 31348 at commit [`8582741`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

SparkQA commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773797032 **[Test build #134902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134902/testReport)** for PR 31348 at commit [`8582741`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

SparkQA commented on pull request #31204: URL: https://github.com/apache/spark/pull/31204#issuecomment-773796762 **[Test build #134911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134911/testReport)** for PR 31204 at commit [`02b0b46`](https://github.com

[GitHub] [spark] sririshindra commented on a change in pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

sririshindra commented on a change in pull request #31477: URL: https://github.com/apache/spark/pull/31477#discussion_r570727456 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -57,7 +57,8 @@ case class BroadcastHash

[GitHub] [spark] SparkQA commented on pull request #31481: [MINOR][TEST] Re-enable some DS v2 char/varchar test

SparkQA commented on pull request #31481: URL: https://github.com/apache/spark/pull/31481#issuecomment-773796124 **[Test build #134909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134909/testReport)** for PR 31481 at commit [`7e9fbcd`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

SparkQA removed a comment on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773692087 **[Test build #134895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134895/testReport)** for PR 31477 at commit [`e2b2a81`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31477: [SPARK-34369][SQL][WEBUI] Track number of pairs processed out of Join.

SparkQA commented on pull request #31477: URL: https://github.com/apache/spark/pull/31477#issuecomment-773795208 **[Test build #134895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134895/testReport)** for PR 31477 at commit [`e2b2a81`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31413: [SPARK-32985][SQL] Decouple bucket scan and bucket filter pruning for data source v1

SparkQA commented on pull request #31413: URL: https://github.com/apache/spark/pull/31413#issuecomment-773794905 **[Test build #134910 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134910/testReport)** for PR 31413 at commit [`03120af`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

AmplabJenkins commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-773794596 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39489/ -

[GitHub] [spark] SparkQA commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

SparkQA commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-773794573 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39489/ ---

[GitHub] [spark] SparkQA commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

SparkQA commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773794491 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39491/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

AmplabJenkins commented on pull request #31476: URL: https://github.com/apache/spark/pull/31476#issuecomment-773794508 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134896/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31348: [SPARK-34245][CORE] Ensure Master removes executors that failed to send finished state

AmplabJenkins commented on pull request #31348: URL: https://github.com/apache/spark/pull/31348#issuecomment-773794502 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39491/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

SparkQA removed a comment on pull request #31476: URL: https://github.com/apache/spark/pull/31476#issuecomment-773692487 **[Test build #134896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134896/testReport)** for PR 31476 at commit [`38fb966`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

SparkQA commented on pull request #31476: URL: https://github.com/apache/spark/pull/31476#issuecomment-773793576 **[Test build #134896 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134896/testReport)** for PR 31476 at commit [`38fb966`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

SparkQA commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-773792948 **[Test build #134912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134912/testReport)** for PR 30957 at commit [`b7e7f92`](https://github.com

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

AngersZh commented on a change in pull request #31204: URL: https://github.com/apache/spark/pull/31204#discussion_r570724448 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala ## @@ -138,14 +211,63 @@ private[spark] class AppStatusStore( }

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AngersZh commented on a change in pull request #30957: URL: https://github.com/apache/spark/pull/30957#discussion_r570723751 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/CatalystTypeConverters.scala ## @@ -174,6 +174,7 @@ object CatalystTypeConve

[GitHub] [spark] SparkQA commented on pull request #31479: [SPARK-34373][SQL] HiveThriftServer2 startWithContext may hang with a race issue

SparkQA commented on pull request #31479: URL: https://github.com/apache/spark/pull/31479#issuecomment-773791464 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39486/ -

  1   2   3   4   5   6   7   8   >