[GitHub] [spark] SparkQA commented on pull request #31305: [SPARK-34213][SQL][3.0] Refresh cached data of v1 table in `LOAD DATA`

SparkQA commented on pull request #31305: URL: https://github.com/apache/spark/pull/31305#issuecomment-766304228 **[Test build #134408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134408/testReport)** for PR 31305 at commit

[GitHub] [spark] MaxGekk commented on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

MaxGekk commented on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766303721 Here is the backport to `branch-3.0`: https://github.com/apache/spark/pull/31305 This is an automated message

[GitHub] [spark] MaxGekk opened a new pull request #31305: [SPARK-34213][SQL][3.0] Refresh cached data of v1 table in `LOAD DATA`

MaxGekk opened a new pull request #31305: URL: https://github.com/apache/spark/pull/31305 ### What changes were proposed in this pull request? Invoke `CatalogImpl.refreshTable()` instead of `SessionCatalog.refreshTable` in v1 implementation of the `LOAD DATA` command.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31024: [SPARK-33979][SQL] Reorder predicate

AmplabJenkins removed a comment on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766299111 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134407/

[GitHub] [spark] AmplabJenkins commented on pull request #31024: [SPARK-33979][SQL] Reorder predicate

AmplabJenkins commented on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766299111 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134407/

[GitHub] [spark] SparkQA removed a comment on pull request #31024: [SPARK-33979][SQL] Reorder predicate

SparkQA removed a comment on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766284680 **[Test build #134407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134407/testReport)** for PR 31024 at commit

[GitHub] [spark] SparkQA commented on pull request #31024: [SPARK-33979][SQL] Reorder predicate

SparkQA commented on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766299044 **[Test build #134407 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134407/testReport)** for PR 31024 at commit

[GitHub] [spark] viirya commented on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya commented on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766292873 BTW, I know PipedRDD (RDD.pipe) is widely used. At least I heard many times people use pipe to adapt existing program with Spark.

[GitHub] [spark] viirya commented on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya commented on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766292712 > Also, when they can do arbitrary aggregation in forked process per partition, the aggregation is only bound to the micro-batch, even only bound to the single partition, which

[GitHub] [spark] viirya edited a comment on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya edited a comment on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766291831 > I understand the functionality is lacking on SS. There's a workaround like foreachBatch -> toRDD -> pipe but streaming operations can't be added after calling pipe. So

[GitHub] [spark] viirya commented on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

viirya commented on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766291831 > I understand the functionality is lacking on SS. There's a workaround like foreachBatch -> toRDD -> pipe but streaming operations can't be added after calling pipe. So I'd

[GitHub] [spark] wangyum commented on a change in pull request #31303: [SPARK-34211][SQL][TESTS] Benchmark TPC-DS with 1GB scale factor

wangyum commented on a change in pull request #31303: URL: https://github.com/apache/spark/pull/31303#discussion_r563236578 ## File path: .github/workflows/build_and_test.yml ## @@ -430,3 +430,38 @@ jobs: - name: Build with SBT run: | ./build/sbt -Pyarn

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31024: [SPARK-33979][SQL] Reorder predicate

AmplabJenkins removed a comment on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766289094 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38993/

[GitHub] [spark] AmplabJenkins commented on pull request #31024: [SPARK-33979][SQL] Reorder predicate

AmplabJenkins commented on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766289094 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38993/

[GitHub] [spark] SparkQA commented on pull request #31024: [SPARK-33979][SQL] Reorder predicate

SparkQA commented on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766289090 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38993/

[GitHub] [spark] HyukjinKwon commented on pull request #31218: [SPARK-34147][SQL][TEST] Keep table partitioning in TPCDSQueryBenchmak when CBO is enabled

HyukjinKwon commented on pull request #31218: URL: https://github.com/apache/spark/pull/31218#issuecomment-766288411 @maropu can you take a quick look for doubly sure when you find some time? This is an automated message

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31218: [SPARK-34147][SQL][TEST] Keep table partitioning in TPCDSQueryBenchmak when CBO is enabled

HyukjinKwon commented on a change in pull request #31218: URL: https://github.com/apache/spark/pull/31218#discussion_r563234527 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala ## @@ -70,11 +67,13 @@ object

[GitHub] [spark] HyukjinKwon closed pull request #31243: [SPARK-34155][SQL][TEST] Add partition columns for TPCDS tables

HyukjinKwon closed pull request #31243: URL: https://github.com/apache/spark/pull/31243 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31243: [SPARK-34155][SQL][TEST] Add partition columns for TPCDS tables

HyukjinKwon commented on pull request #31243: URL: https://github.com/apache/spark/pull/31243#issuecomment-766288033 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31271: [SPARK-34185][DOCS] Review and fix issues in API docs

HyukjinKwon commented on a change in pull request #31271: URL: https://github.com/apache/spark/pull/31271#discussion_r563233910 ## File path: project/SparkBuild.scala ## @@ -910,7 +910,7 @@ object Unidoc {

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31271: [SPARK-34185][DOCS] Review and fix issues in API docs

HyukjinKwon commented on a change in pull request #31271: URL: https://github.com/apache/spark/pull/31271#discussion_r563233892 ## File path: project/SparkBuild.scala ## @@ -910,7 +910,7 @@ object Unidoc {

[GitHub] [spark] SparkQA commented on pull request #31024: [SPARK-33979][SQL] Reorder predicate

SparkQA commented on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766287673 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38993/

[GitHub] [spark] HyukjinKwon closed pull request #31300: [SPARK-34052][SQL][3.1] store SQL text for a temp view created using "CACHE TABLE .. AS SELECT ..."

HyukjinKwon closed pull request #31300: URL: https://github.com/apache/spark/pull/31300 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon closed pull request #31290: [SPARK-33813][SQL][3.0] Fix the issue that JDBC source can't treat MS SQL Server's spatial types

HyukjinKwon closed pull request #31290: URL: https://github.com/apache/spark/pull/31290 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31290: [SPARK-33813][SQL][3.0] Fix the issue that JDBC source can't treat MS SQL Server's spatial types

HyukjinKwon commented on pull request #31290: URL: https://github.com/apache/spark/pull/31290#issuecomment-766287402 Merged to branch-3.0. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

HyukjinKwon commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766285048 cc @Ngone51 who also works on decommissioning and torrent broadcast stuff. This is an automated message

[GitHub] [spark] HyukjinKwon commented on pull request #31300: [SPARK-34052][SQL][3.1] store SQL text for a temp view created using "CACHE TABLE .. AS SELECT ..."

HyukjinKwon commented on pull request #31300: URL: https://github.com/apache/spark/pull/31300#issuecomment-766284933 Merged to branch-3.1 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon closed pull request #29178: [SPARK-32380][SQL] fixed spark3.0 access hive table while data in hbase problem

HyukjinKwon closed pull request #29178: URL: https://github.com/apache/spark/pull/29178 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon closed pull request #31302: [SPARK-34210][SQL] After upgrading 3.0.1, Spark SQL access hive on HBase table access exception

HyukjinKwon closed pull request #31302: URL: https://github.com/apache/spark/pull/31302 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31302: [SPARK-34210][SQL] After upgrading 3.0.1, Spark SQL access hive on HBase table access exception

HyukjinKwon commented on pull request #31302: URL: https://github.com/apache/spark/pull/31302#issuecomment-766284757 Is this a duplicate of https://github.com/apache/spark/pull/29178 and https://github.com/apache/spark/pull/31147? I have the same question

[GitHub] [spark] HyukjinKwon closed pull request #31147: [SPARK-32380][SQL]fixed spark3.0 access hive table while data in hbase problem

HyukjinKwon closed pull request #31147: URL: https://github.com/apache/spark/pull/31147 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #31024: [SPARK-33979][SQL] Reorder predicate

SparkQA commented on pull request #31024: URL: https://github.com/apache/spark/pull/31024#issuecomment-766284680 **[Test build #134407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134407/testReport)** for PR 31024 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31303: [SPARK-34211][SQL][TEST] Benchmark TPC-DS with 1GB scale factor

HyukjinKwon commented on a change in pull request #31303: URL: https://github.com/apache/spark/pull/31303#discussion_r563230694 ## File path: .github/workflows/build_and_test.yml ## @@ -430,3 +430,38 @@ jobs: - name: Build with SBT run: | ./build/sbt

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31303: [SPARK-34211][SQL][TEST] Benchmark TPC-DS with 1GB scale factor

HyukjinKwon commented on a change in pull request #31303: URL: https://github.com/apache/spark/pull/31303#discussion_r563230668 ## File path: .github/workflows/build_and_test.yml ## @@ -430,3 +430,38 @@ jobs: - name: Build with SBT run: | ./build/sbt

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31303: [SPARK-34211][SQL][TEST] Benchmark TPC-DS with 1GB scale factor

HyukjinKwon commented on a change in pull request #31303: URL: https://github.com/apache/spark/pull/31303#discussion_r563230598 ## File path: .github/workflows/build_and_test.yml ## @@ -430,3 +430,38 @@ jobs: - name: Build with SBT run: | ./build/sbt

[GitHub] [spark] HyukjinKwon commented on pull request #31303: [SPARK-34211][SQL][TEST] Benchmark TPC-DS with 1GB scale factor

HyukjinKwon commented on pull request #31303: URL: https://github.com/apache/spark/pull/31303#issuecomment-766283987 cc @Ngone51, @dongjoon-hyun, @cloud-fan, @peter-toth FYI This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31303: [SPARK-34211][SQL][TEST] Benchmark TPC-DS with 1GB scale factor

AmplabJenkins removed a comment on pull request #31303: URL: https://github.com/apache/spark/pull/31303#issuecomment-765888057 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31303: [SPARK-34211][SQL][TEST] Benchmark TPC-DS with 1GB scale factor

HyukjinKwon commented on a change in pull request #31303: URL: https://github.com/apache/spark/pull/31303#discussion_r563230324 ## File path: .github/workflows/build_and_test.yml ## @@ -430,3 +430,38 @@ jobs: - name: Build with SBT run: | ./build/sbt

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

AmplabJenkins removed a comment on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766283831 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134406/

[GitHub] [spark] AmplabJenkins commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

AmplabJenkins commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766283831 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134406/

[GitHub] [spark] HeartSaVioR edited a comment on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

HeartSaVioR edited a comment on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766278704 Also, when they can do arbitrary aggregation in forked process per partition, the aggregation is only bound to the micro-batch, even only bound to the single

[GitHub] [spark] HeartSaVioR edited a comment on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

HeartSaVioR edited a comment on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766278704 Also, when they can do arbitrary aggregation in forked process per partition, the aggregation is only bound to the micro-batch, even only bound to the single

[GitHub] [spark] SparkQA removed a comment on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA removed a comment on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766247355 **[Test build #134406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134406/testReport)** for PR 31298 at commit

[GitHub] [spark] SparkQA commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766279632 **[Test build #134406 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134406/testReport)** for PR 31298 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

HeartSaVioR commented on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766278704 Also, when they can do arbitrary aggregation in forked process per partition, the aggregation is only bound to the micro-batch, even only bound to the single partition,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

AmplabJenkins removed a comment on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766276488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134405/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

AmplabJenkins removed a comment on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766276489 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38992/

[GitHub] [spark] AmplabJenkins commented on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

AmplabJenkins commented on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766276488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134405/

[GitHub] [spark] AmplabJenkins commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

AmplabJenkins commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766276489 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38992/

[GitHub] [spark] HeartSaVioR edited a comment on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

HeartSaVioR edited a comment on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766276315 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #31296: [SPARK-34205][SQL][SS] Add pipe to Dataset to enable Streaming Dataset pipe

HeartSaVioR commented on pull request #31296: URL: https://github.com/apache/spark/pull/31296#issuecomment-766276315 I understand the functionality is lacking on SS. There's a workaround like foreachBatch -> toRDD -> pipe but streaming operations can't be added after calling pipe. So I'd

[GitHub] [spark] SparkQA removed a comment on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

SparkQA removed a comment on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766199176 **[Test build #134405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134405/testReport)** for PR 31203 at commit

[GitHub] [spark] SparkQA commented on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

SparkQA commented on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766273876 **[Test build #134405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134405/testReport)** for PR 31203 at commit

[GitHub] [spark] SparkQA commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766273202 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38992/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

AmplabJenkins removed a comment on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766270156 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38991/

[GitHub] [spark] AmplabJenkins commented on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

AmplabJenkins commented on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766270156 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38991/

[GitHub] [spark] SparkQA commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766256745 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38992/

[GitHub] [spark] SparkQA commented on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

SparkQA commented on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766248177 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38991/

[GitHub] [spark] SparkQA commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766247355 **[Test build #134406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134406/testReport)** for PR 31298 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

AmplabJenkins removed a comment on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766243203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134403/

[GitHub] [spark] AmplabJenkins commented on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

AmplabJenkins commented on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766243203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134403/

[GitHub] [spark] holdenk commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

holdenk commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-766242622 Jenkins retest this please (kafka failure unrelated). This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun commented on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

dongjoon-hyun commented on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766236553 Could you make a backport to branch-3.0? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

SparkQA commented on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766236406 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38991/

[GitHub] [spark] dongjoon-hyun closed pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

dongjoon-hyun closed pull request #31304: URL: https://github.com/apache/spark/pull/31304 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563213871 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/parquet/ParquetPartitionReaderFactory.scala ## @@ -225,6

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563213809 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileMetaCacheManager.scala ## @@ -0,0 +1,75 @@ +/* + * Licensed

[GitHub] [spark] SparkQA removed a comment on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

SparkQA removed a comment on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766165976 **[Test build #134403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134403/testReport)** for PR 31304 at commit

[GitHub] [spark] SparkQA commented on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

SparkQA commented on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766215212 **[Test build #134403 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134403/testReport)** for PR 31304 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563213655 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileMetaCacheManager.scala ## @@ -0,0 +1,75 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563213572 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileMeta.scala ## @@ -0,0 +1,48 @@ +/* + * Licensed to the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563213415 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileMetaCacheManager.scala ## @@ -0,0 +1,75 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563213189 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileMetaCacheManager.scala ## @@ -0,0 +1,75 @@ +/* + * Licensed

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563212908 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -838,6 +845,20 @@ object SQLConf { .booleanConf

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563212780 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -838,6 +845,20 @@ object SQLConf { .booleanConf

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563212548 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -838,6 +845,20 @@ object SQLConf { .booleanConf

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563212466 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -785,6 +785,13 @@ object SQLConf { .booleanConf

[GitHub] [spark] SparkQA commented on pull request #31203: [SPARK-33212][FOLLOW-UP][BUILD] Bring back duplicate dependency check and add more strict Hadoop version check

SparkQA commented on pull request #31203: URL: https://github.com/apache/spark/pull/31203#issuecomment-766199176 **[Test build #134405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134405/testReport)** for PR 31203 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

dongjoon-hyun commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r563212195 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -785,6 +785,13 @@ object SQLConf { .booleanConf

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

AmplabJenkins removed a comment on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766197480 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

AmplabJenkins commented on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766197480 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

SparkQA removed a comment on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766174321 **[Test build #134404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134404/testReport)** for PR 31281 at commit

[GitHub] [spark] SparkQA commented on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

SparkQA commented on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766194925 **[Test build #134404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134404/testReport)** for PR 31281 at commit

[GitHub] [spark] SparkQA commented on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

SparkQA commented on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766193904 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38990/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

AmplabJenkins removed a comment on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766180383 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38989/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31265: [SPARK-34197][SQL] `SessionCatalog.refreshTable()` should not invalidate the relation cache for temporary views

AmplabJenkins removed a comment on pull request #31265: URL: https://github.com/apache/spark/pull/31265#issuecomment-766164385 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31265: [SPARK-34197][SQL] `SessionCatalog.refreshTable()` should not invalidate the relation cache for temporary views

AmplabJenkins commented on pull request #31265: URL: https://github.com/apache/spark/pull/31265#issuecomment-766180384 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134402/

[GitHub] [spark] AmplabJenkins commented on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

AmplabJenkins commented on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766180383 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38989/

[GitHub] [spark] imback82 commented on a change in pull request #31273: [WIP][Spark-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

imback82 commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r563194113 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ## @@ -558,20 +560,27 @@ object ViewHelper { catalog:

[GitHub] [spark] SparkQA commented on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

SparkQA commented on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766179587 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38990/

[GitHub] [spark] SparkQA removed a comment on pull request #31265: [SPARK-34197][SQL] `SessionCatalog.refreshTable()` should not invalidate the relation cache for temporary views

SparkQA removed a comment on pull request #31265: URL: https://github.com/apache/spark/pull/31265#issuecomment-766145188 **[Test build #134402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134402/testReport)** for PR 31265 at commit

[GitHub] [spark] SparkQA commented on pull request #31265: [SPARK-34197][SQL] `SessionCatalog.refreshTable()` should not invalidate the relation cache for temporary views

SparkQA commented on pull request #31265: URL: https://github.com/apache/spark/pull/31265#issuecomment-766178765 **[Test build #134402 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134402/testReport)** for PR 31265 at commit

[GitHub] [spark] SparkQA commented on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

SparkQA commented on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766174321 **[Test build #134404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134404/testReport)** for PR 31281 at commit

[GitHub] [spark] viirya commented on a change in pull request #31300: [SPARK-34052][SQL][3.1] store SQL text for a temp view created using "CACHE TABLE .. AS SELECT ..."

viirya commented on a change in pull request #31300: URL: https://github.com/apache/spark/pull/31300#discussion_r563194618 ## File path: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala ## @@ -928,33 +928,61 @@ class CachedTableSuite extends QueryTest with

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

AmplabJenkins removed a comment on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766164386 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

AmplabJenkins commented on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766172855 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134401/

[GitHub] [spark] imback82 commented on a change in pull request #31273: [WIP][Spark-34152][SQL] Make CreateViewStatement.child to be LogicalPlan's children so that it's resolved in analyze phase

imback82 commented on a change in pull request #31273: URL: https://github.com/apache/spark/pull/31273#discussion_r563194113 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ## @@ -558,20 +560,27 @@ object ViewHelper { catalog:

[GitHub] [spark] SparkQA removed a comment on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

SparkQA removed a comment on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766145652 **[Test build #134401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134401/testReport)** for PR 31281 at commit

[GitHub] [spark] SparkQA commented on pull request #31281: [SPARK-34192][SQL] Move char padding to write side and remove length check on read side too

SparkQA commented on pull request #31281: URL: https://github.com/apache/spark/pull/31281#issuecomment-766169311 **[Test build #134401 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134401/testReport)** for PR 31281 at commit

[GitHub] [spark] SparkQA commented on pull request #31304: [SPARK-34213][SQL] Refresh cached data of v1 table in `LOAD DATA`

SparkQA commented on pull request #31304: URL: https://github.com/apache/spark/pull/31304#issuecomment-766165976 **[Test build #134403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134403/testReport)** for PR 31304 at commit

  1   2   >