[GitHub] [spark] HeartSaVioR commented on pull request #34089: [SPARK-36837][BUILD] Upgrade Kafka to 3.0.0
HeartSaVioR commented on pull request #34089: URL: https://github.com/apache/spark/pull/34089#issuecomment-927008330 Personally I'm in favor of holding on upgrade for major version till a couple of bugfix versions based on the major version are released. There is around 6 months for Spark 3.3.0 to be released, and we can let early-adopters to experiment with Kafka 3.0.0 (even 3.0.x) clients in the meanwhile. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34104: [SPARK-36848][SQL] Migrate ShowCurrentNamespaceStatement to v2 command framework
AmplabJenkins removed a comment on pull request #34104: URL: https://github.com/apache/spark/pull/34104#issuecomment-927006905 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48131/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34104: [SPARK-36848][SQL] Migrate ShowCurrentNamespaceStatement to v2 command framework
AmplabJenkins commented on pull request #34104: URL: https://github.com/apache/spark/pull/34104#issuecomment-927006905 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48131/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34104: [SPARK-36848][SQL] Migrate ShowCurrentNamespaceStatement to v2 command framework
SparkQA commented on pull request #34104: URL: https://github.com/apache/spark/pull/34104#issuecomment-927004482 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48131/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-927003864 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143618/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-927003864 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143618/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926987611 **[Test build #143618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143618/testReport)** for PR 34100 at commit [`0c358b3`](https://github.com/apache/spark/commit/0c358b34a14c59158bff018777388605abf42dc3). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-927003708 **[Test build #143618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143618/testReport)** for PR 34100 at commit [`0c358b3`](https://github.com/apache/spark/commit/0c358b34a14c59158bff018777388605abf42dc3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
SparkQA removed a comment on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926946228 **[Test build #143614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143614/testReport)** for PR 34087 at commit [`334ce1f`](https://github.com/apache/spark/commit/334ce1fc713a5b328a06761c3f493a5d26a41c85). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-927001227 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48130/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
AmplabJenkins removed a comment on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-927001229 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48129/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
AmplabJenkins removed a comment on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-927001226 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143614/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-927001227 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48130/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
AmplabJenkins commented on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-927001229 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48129/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
AmplabJenkins commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-927001226 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143614/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
SparkQA commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-927000191 **[Test build #143614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143614/testReport)** for PR 34087 at commit [`334ce1f`](https://github.com/apache/spark/commit/334ce1fc713a5b328a06761c3f493a5d26a41c85). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
SparkQA commented on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-927000139 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48129/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34104: [SPARK-36848][SQL] Migrate ShowCurrentNamespaceStatement to v2 command framework
SparkQA commented on pull request #34104: URL: https://github.com/apache/spark/pull/34104#issuecomment-92743 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48131/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926998644 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48130/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
AmplabJenkins removed a comment on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-926995976 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143617/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
SparkQA removed a comment on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-926987646 **[Test build #143617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143617/testReport)** for PR 34103 at commit [`cb6b5b1`](https://github.com/apache/spark/commit/cb6b5b1fc87438264321497cd48fd32a47ba44a7). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
AmplabJenkins commented on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-926995976 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143617/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
SparkQA commented on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-926995908 **[Test build #143617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143617/testReport)** for PR 34103 at commit [`cb6b5b1`](https://github.com/apache/spark/commit/cb6b5b1fc87438264321497cd48fd32a47ba44a7). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
AmplabJenkins removed a comment on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926994863 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48128/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34104: [SPARK-36848][SQL] Migrate ShowCurrentNamespaceStatement to v2 command framework
SparkQA commented on pull request #34104: URL: https://github.com/apache/spark/pull/34104#issuecomment-926995264 **[Test build #143619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143619/testReport)** for PR 34104 at commit [`8bebbb5`](https://github.com/apache/spark/commit/8bebbb5c8fb79fa9b579359f2b2dbec1f507655d). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
AmplabJenkins commented on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926994863 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48128/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
SparkQA commented on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-926993578 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48129/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926993508 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48130/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ueshin commented on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
ueshin commented on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926993361 cc @HyukjinKwon @xinrong-databricks @itholic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
SparkQA commented on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926991616 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48128/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao opened a new pull request #34104: [SPARK-36848][SQL] Migrate ShowCurrentNamespaceStatement to v2 command framework
huaxingao opened a new pull request #34104: URL: https://github.com/apache/spark/pull/34104 ### What changes were proposed in this pull request? Migrate `ShowCurrentNamespaceStatement` to v2 command framework ### Why are the changes needed? Migrate to the standard V2 framework ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
AmplabJenkins removed a comment on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926987252 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143616/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926953096 **[Test build #143615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143615/testReport)** for PR 34100 at commit [`d73562e`](https://github.com/apache/spark/commit/d73562ed3635bb3454ac67029ca6541b30ae0c02). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926987299 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143615/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34103: [SPARK-32712][SQL] Support writing Hive bucketed table (Hive file formats with Hive hash)
SparkQA commented on pull request #34103: URL: https://github.com/apache/spark/pull/34103#issuecomment-926987646 **[Test build #143617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143617/testReport)** for PR 34103 at commit [`cb6b5b1`](https://github.com/apache/spark/commit/cb6b5b1fc87438264321497cd48fd32a47ba44a7). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926987611 **[Test build #143618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143618/testReport)** for PR 34100 at commit [`0c358b3`](https://github.com/apache/spark/commit/0c358b34a14c59158bff018777388605abf42dc3). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926987299 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143615/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
AmplabJenkins commented on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926987252 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143616/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926986994 **[Test build #143615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143615/testReport)** for PR 34100 at commit [`d73562e`](https://github.com/apache/spark/commit/d73562ed3635bb3454ac67029ca6541b30ae0c02). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] c21 commented on a change in pull request #34103: [SPARK-32712][SQL] Support to write Hive bucketed table (Hive file formats with Hive hash)
c21 commented on a change in pull request #34103:
URL: https://github.com/apache/spark/pull/34103#discussion_r715967575
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/sources/BucketedWriteWithHiveSupportSuite.scala
##
@@ -48,29 +49,37 @@ class BucketedWriteWithHiveSupportSuite extends
BucketedWriteSuite with TestHive
val table = "hive_bucketed_table"
fileFormatsToTest.foreach { format =>
- withTable(table) {
-sql(
- s"""
- |CREATE TABLE IF NOT EXISTS $table (i int, j string)
- |PARTITIONED BY(k string)
- |CLUSTERED BY (i, j) SORTED BY (i) INTO 8 BUCKETS
- |STORED AS $format
- """.stripMargin)
+ Seq("true", "false").foreach { enableConvertMetastore =>
+withSQLConf(HiveUtils.CONVERT_METASTORE_PARQUET.key ->
enableConvertMetastore,
+ HiveUtils.CONVERT_METASTORE_ORC.key -> enableConvertMetastore) {
+ withTable(table) {
+sql(
+ s"""
+ |CREATE TABLE IF NOT EXISTS $table (i int, j string)
+ |PARTITIONED BY(k string)
+ |CLUSTERED BY (i, j) SORTED BY (i) INTO 8 BUCKETS
+ |STORED AS $format
+ """.stripMargin)
-val df =
- (0 until 50).map(i => (i % 13, i.toString, i % 5)).toDF("i", "j",
"k")
-df.write.mode(SaveMode.Overwrite).insertInto(table)
+val df =
+ (0 until 50).map(i => (i % 13, i.toString, i % 5)).toDF("i",
"j", "k")
-for (k <- 0 until 5) {
- testBucketing(
-new File(tableDir(table), s"k=$k"),
-format,
-8,
-Seq("i", "j"),
-Seq("i"),
-df,
-bucketIdExpression,
-getBucketIdFromFileName)
+withSQLConf("hive.exec.dynamic.partition.mode" -> "nonstrict") {
Review comment:
This is added as Hive write code path enforces it -
https://github.com/apache/spark/blob/master/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala#L161
.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
SparkQA commented on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926986277 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48128/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] c21 opened a new pull request #34103: [SPARK-32712][SQL] Support to write Hive bucketed table (Hive file formats with Hive hash)
c21 opened a new pull request #34103: URL: https://github.com/apache/spark/pull/34103 ### What changes were proposed in this pull request? This is to support writing Hive bucketed table with Hive file formats (the code path for Hive table write - `InsertIntoHiveTable`). The bucketed table is partitioned with Hive hash, same as Hive, Presto and Trino. ### Why are the changes needed? To make Spark write other-SQL-engines-compatible bucketed table. Same motivation as https://github.com/apache/spark/pull/33432 . ### Does this PR introduce _any_ user-facing change? Yes. Before this PR, writing to these Hive bucketed table would throw an exception in Spark if config "hive.enforce.bucketing" or "hive.enforce.sorting" set to true. After this PR, writing to these Hive bucketed table would succeed. The table can be read back by Presto and Trino efficiently as other Hive bucketed table. ### How was this patch tested? Modified unit test in `BucketedWriteWithHiveSupportSuite.scala`, to verify bucket file names and each row in each bucket is written properly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao commented on a change in pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
sunchao commented on a change in pull request #34100: URL: https://github.com/apache/spark/pull/34100#discussion_r715966900 ## File path: pom.xml ## @@ -3273,7 +3273,7 @@ 2.7.1 2.4 hadoop-client - hadoop-client + hadoop-yarn-api Review comment: Actually it may not be so useful to change `hadoop-client-minicluster.artifact` since it is test scope while the other two are compile scope by default. For some reason it also changes `dev/deps/spark-deps-hadoop-2.7-hive-2.3` when I set it to something like `hadoop-mapreduce-client-jobclient`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
SparkQA removed a comment on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926979043 **[Test build #143616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143616/testReport)** for PR 34102 at commit [`5b3a5b8`](https://github.com/apache/spark/commit/5b3a5b8bae1ec2f2b8334c86fc6c887c6635007f). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
SparkQA commented on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926983556 **[Test build #143616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143616/testReport)** for PR 34102 at commit [`5b3a5b8`](https://github.com/apache/spark/commit/5b3a5b8bae1ec2f2b8334c86fc6c887c6635007f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
SparkQA commented on pull request #34102: URL: https://github.com/apache/spark/pull/34102#issuecomment-926979043 **[Test build #143616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143616/testReport)** for PR 34102 at commit [`5b3a5b8`](https://github.com/apache/spark/commit/5b3a5b8bae1ec2f2b8334c86fc6c887c6635007f). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ueshin opened a new pull request #34102: [SPARK-36847][PYTHON] Explicitly specify error codes when ignoring type hint errors
ueshin opened a new pull request #34102: URL: https://github.com/apache/spark/pull/34102 ### What changes were proposed in this pull request? Explicitly specifies error codes when ignoring type hint errors. ### Why are the changes needed? We use a lot of `type: ignore` annotation to ignore type hint errors in pandas-on-Spark. We should explicitly specify the error codes to make it clear what kind of error is being ignored, then the type hint checker can check more cases. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
AmplabJenkins removed a comment on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926977149 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48126/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926977150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48127/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #32385: [WIP][SPARK-35275][CORE] Add checksum for shuffle blocks and diagnose corruption
github-actions[bot] commented on pull request #32385: URL: https://github.com/apache/spark/pull/32385#issuecomment-926977283 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
AmplabJenkins commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926977149 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48126/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926977150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48127/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao edited a comment on pull request #33989: [SPARK-36676][SQL][BUILD] Create shaded Hive module and upgrade Guava version to 30.1.1-jre
sunchao edited a comment on pull request #33989: URL: https://github.com/apache/spark/pull/33989#issuecomment-926976194 Hmm interesting. After I changed the isolated class loader to pick guava classes from Hive jars (which is of 14.0.1), tests started to fail because it now seems to use Spark's built-in Guava which is 30.1.1-jre. This doesn't seem to make sense. ``` [error] sbt.ForkMain$ForkError: java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator [error] at org.apache.hadoop.hive.ql.exec.FetchOperator.(FetchOperator.java:108) [error] at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87) [error] at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541) [error] at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) [error] at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) [error] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) [error] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) [error] at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$runHive$1(HiveClientImpl.scala:831) ``` `Iterators.emptyIterator` here is no longer public in the newer versions of guava. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #33989: [SPARK-36676][SQL][BUILD] Create shaded Hive module and upgrade Guava version to 30.1.1-jre
sunchao commented on pull request #33989: URL: https://github.com/apache/spark/pull/33989#issuecomment-926976194 Hmm interesting. After I changed the isolated class loader to pick guava classes from Hive jars (which is of 14.0.1), tests started to fail because it now seems to uses Spark's built-in Guava which is 30.1.1-jre. This doesn't seem to make sense. ``` [error] sbt.ForkMain$ForkError: java.lang.IllegalAccessError: tried to access method com.google.common.collect.Iterators.emptyIterator()Lcom/google/common/collect/UnmodifiableIterator; from class org.apache.hadoop.hive.ql.exec.FetchOperator [error] at org.apache.hadoop.hive.ql.exec.FetchOperator.(FetchOperator.java:108) [error] at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:87) [error] at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:541) [error] at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) [error] at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) [error] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) [error] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) [error] at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$runHive$1(HiveClientImpl.scala:831) ``` `Iterators.emptyIterator` here is no longer public in the newer versions of guava. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926975265 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48127/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao commented on a change in pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
sunchao commented on a change in pull request #34100: URL: https://github.com/apache/spark/pull/34100#discussion_r715955928 ## File path: pom.xml ## @@ -3273,7 +3273,7 @@ 2.7.1 2.4 hadoop-client - hadoop-client + hadoop-yarn-api Review comment: Thanks for taking a look. Yes I think it's better to apply the same for `hadoop-client-minicluster.artifact`. Let me try that, and perhaps we won't need the changes in YARN's pom.xml with this. The side effect for this is seems to be that it affects the _distance_ of these dependencies to the root module and thus may make a difference when maven tries to resolve a dependency with multiple versions (see [here](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html) for reference). I was using `hadoop-common` (which carries lots of dependencies) instead of `hadoop-yarn-api` and it was not able to compile. Will update PR description and the comment in the above pom.xml. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao commented on a change in pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
sunchao commented on a change in pull request #34100: URL: https://github.com/apache/spark/pull/34100#discussion_r715955928 ## File path: pom.xml ## @@ -3273,7 +3273,7 @@ 2.7.1 2.4 hadoop-client - hadoop-client + hadoop-yarn-api Review comment: Thanks for taking a look. Yes I think it's better to apply the same for `hadoop-client-minicluster.artifact. Let me try that, and perhaps we won't need the changes in YARN's pom.xml with this. The side effect for this is seems to be that it affects the _distance_ of these dependencies to the root module and thus may make a difference when maven tries to resolve a dependency with multiple versions (see [here](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html) for reference). I was using `hadoop-common` (which carries lots of dependencies) instead of `hadoop-yarn-api` and it was not able to compile. Will update PR description and the comment in the above pom.xml. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
SparkQA commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926972509 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48126/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926965463 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48127/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen commented on a change in pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
JoshRosen commented on a change in pull request #34100: URL: https://github.com/apache/spark/pull/34100#discussion_r715946127 ## File path: pom.xml ## @@ -3273,7 +3273,7 @@ 2.7.1 2.4 hadoop-client - hadoop-client + hadoop-yarn-api Review comment: Ahhh, this is a clever fix: Instead of the `hadoop-2.7` profile resulting in a duplicate direct dependency on `hadoop-client`, we now just declare an explicit dependency on one of `hadoop-client`'s transitive dependencies (`hadoop-yarn-api` in this case). Anything which depends on `hadoop-client-runtime.artifact` must also depend on `hadoop-client-api.artifact`, so this doesn't end up changing the set of dependencies pulled in. It looks like we didn't need to do that for `hadoop-client-minicluster.artifact` because that's only used in the `resource-managers/yarn` POM and that's already using Maven profiles to control the dependency selection (so the other workaround is fairly non-invasive in that context). In principle, though, I guess we could have changed that to some other transitive dep. --- Could you maybe add a one or two line comment above these Hadoop 2.7 lines to explain what's going on? And maybe edit the comment at https://github.com/apache/spark/blob/d73562ed3635bb3454ac67029ca6541b30ae0c02/pom.xml#L251-L255 to reflect this change? This fix is clever but a little subtle, so I think a comment calling it out (and maybe mentioning SPARK-36835 might help future readers. **Edit:** could you also update the PR description to reflect this final fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen commented on a change in pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
JoshRosen commented on a change in pull request #34100: URL: https://github.com/apache/spark/pull/34100#discussion_r715946127 ## File path: pom.xml ## @@ -3273,7 +3273,7 @@ 2.7.1 2.4 hadoop-client - hadoop-client + hadoop-yarn-api Review comment: Ahhh, this is a clever fix: Instead of the `hadoop-2.7` profile resulting in a duplicate direct dependency on `hadoop-client`, we now just declare an explicit dependency on one of `hadoop-client`'s transitive dependencies (`hadoop-yarn-api` in this case). Anything which depends on `hadoop-client-runtime.artifact` must also depend on `hadoop-client-api.artifact`, so this doesn't end up changing the set of dependencies pulled in. It looks like we didn't need to do that for `hadoop-client-minicluster.artifact` because that's only used in the `resource-managers/yarn` POM and that's already using Maven profiles to control the dependency selection (so the other workaround is less invasive in that context). In principle, though, I guess we could have changed that to some other transitive dep. --- Could you maybe add a one or two line comment above these Hadoop 2.7 lines to explain what's going on? And maybe edit the comment at https://github.com/apache/spark/blob/d73562ed3635bb3454ac67029ca6541b30ae0c02/pom.xml#L251-L255 to reflect this change? This fix is clever but a little subtle, so I think a comment calling it out (and maybe mentioning SPARK-36835 might help future readers. **Edit:** could you also update the PR description to reflect this final fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen commented on a change in pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
JoshRosen commented on a change in pull request #34100: URL: https://github.com/apache/spark/pull/34100#discussion_r715946127 ## File path: pom.xml ## @@ -3273,7 +3273,7 @@ 2.7.1 2.4 hadoop-client - hadoop-client + hadoop-yarn-api Review comment: Ahhh, this is a clever fix: Instead of the `hadoop-2.7` profile resulting in a duplicate direct dependency on `hadoop-client`, we now just declare an explicit dependency on one of `hadoop-client`'s transitive dependencies (`hadoop-yarn-api` in this case). Anything which depends on `hadoop-client-runtime.artifact` must also depend on `hadoop-client-api.artifact`, so this doesn't end up changing the set of dependencies pulled in. It looks like we didn't need to do that for `hadoop-client-minicluster.artifact` because that's only used in the `resource-managers/yarn` POM and that's already using Maven profiles to control the dependency selection (so the other workaround is less invasive in that context). In principle, though, I guess we could have changed that to some other transitive dep. --- Could you maybe add a one or two line comment above these Hadoop 2.7 lines to explain what's going on? And maybe edit the comment at https://github.com/apache/spark/blob/d73562ed3635bb3454ac67029ca6541b30ae0c02/pom.xml#L251-L255 to reflect this change? This fix is clever but a little subtle, so I think a comment calling it out (and maybe mentioning SPARK-36835) might help future readers. **Edit:** could you also update the PR description to reflect this final fix? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen commented on a change in pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
JoshRosen commented on a change in pull request #34100: URL: https://github.com/apache/spark/pull/34100#discussion_r715946127 ## File path: pom.xml ## @@ -3273,7 +3273,7 @@ 2.7.1 2.4 hadoop-client - hadoop-client + hadoop-yarn-api Review comment: Ahhh, this is a clever fix: Instead of the `hadoop-2.7` profile resulting in a duplicate direct dependency on `hadoop-client`, we now just declare an explicit dependency on one of `hadoop-client`'s transitive dependencies (`hadoop-yarn-api` in this case). Anything which depends on `hadoop-client-runtime.artifact` must also depend on `hadoop-client-api.artifact`, so this doesn't end up changing the set of dependencies pulled in. It looks like we didn't need to do that for `hadoop-client-minicluster.artifact` because that's only used in the `resource-managers/yarn` POM and that's already using Maven profiles to control the dependency selection (so the other workaround is less invasive in that context). In principle, though, I guess we could have changed that to some other transitive dep. --- Could you maybe add a one or two line comment above these Hadoop 2.7 lines to explain what's going on? And maybe edit the comment at https://github.com/apache/spark/blob/d73562ed3635bb3454ac67029ca6541b30ae0c02/pom.xml#L251-L255 to reflect this change? This fix is clever but a little subtle, so I think a comment calling it out (and maybe mentioning SPARK-36835) might help future readers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
SparkQA commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926960970 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48126/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
AmplabJenkins removed a comment on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926954360 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48125/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
AmplabJenkins commented on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926954360 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48125/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
SparkQA commented on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926954350 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48125/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926953096 **[Test build #143615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143615/testReport)** for PR 34100 at commit [`d73562e`](https://github.com/apache/spark/commit/d73562ed3635bb3454ac67029ca6541b30ae0c02). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
sunchao commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926952762 updated the PR to use different name for `hadoop-client-runtime.artifact`, which is probably a simpler approach. Verified locally with: ``` build/mvn clean install -DskipTests -Phadoop-2.7 -Phive-2.3 -Pmesos -Phive-thriftserver -Pyarn -Pspark-ganglia-lgpl -Pkinesis-asl -Pkubernetes -Phadoop-cloud -Phive ``` and the build is successful. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
SparkQA commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926946228 **[Test build #143614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143614/testReport)** for PR 34087 at commit [`334ce1f`](https://github.com/apache/spark/commit/334ce1fc713a5b328a06761c3f493a5d26a41c85). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level
SparkQA removed a comment on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-926859915 **[Test build #143612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143612/testReport)** for PR 33253 at commit [`ac1659e`](https://github.com/apache/spark/commit/ac1659e156eca5899e1eff765698c9986eec5d4c). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level
AmplabJenkins removed a comment on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-926944613 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143612/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] flyrain commented on a change in pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
flyrain commented on a change in pull request #34087:
URL: https://github.com/apache/spark/pull/34087#discussion_r715929221
##
File path:
sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatchRow.java
##
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.vectorized;
+
+import org.apache.spark.sql.catalyst.InternalRow;
+import org.apache.spark.sql.catalyst.expressions.GenericInternalRow;
+import org.apache.spark.sql.types.*;
+import org.apache.spark.unsafe.types.CalendarInterval;
+import org.apache.spark.unsafe.types.UTF8String;
+
+/**
+ * This class wraps an array of {@link ColumnVector} and provides a row view.
+ */
+public final class ColumnarBatchRow extends InternalRow {
Review comment:
Made the change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] flyrain commented on a change in pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
flyrain commented on a change in pull request #34087:
URL: https://github.com/apache/spark/pull/34087#discussion_r715928920
##
File path:
sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java
##
@@ -32,11 +28,11 @@
*/
@Evolving
public class ColumnarBatch implements AutoCloseable {
Review comment:
Make sense to me. Made the change in the new commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level
AmplabJenkins commented on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-926944613 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143612/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
SparkQA commented on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926938373 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/48125/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level
SparkQA commented on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-926937820 **[Test build #143612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143612/testReport)** for PR 33253 at commit [`ac1659e`](https://github.com/apache/spark/commit/ac1659e156eca5899e1eff765698c9986eec5d4c). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class SkewJoinChildWrapper(plan: SparkPlan) extends LeafExecNode ` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
AmplabJenkins removed a comment on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926932659 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143613/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
SparkQA removed a comment on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926922191 **[Test build #143613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143613/testReport)** for PR 34101 at commit [`0a43396`](https://github.com/apache/spark/commit/0a43396ce3da47024db39f27ffcc9f28911cf1ab). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
AmplabJenkins commented on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926932659 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143613/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
SparkQA commented on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926932417 **[Test build #143613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143613/testReport)** for PR 34101 at commit [`0a43396`](https://github.com/apache/spark/commit/0a43396ce3da47024db39f27ffcc9f28911cf1ab). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34038: [SPARK-36797][SQL] Union should resolve nested columns as top-level columns
AmplabJenkins commented on pull request #34038: URL: https://github.com/apache/spark/pull/34038#issuecomment-926928227 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143607/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34038: [SPARK-36797][SQL] Union should resolve nested columns as top-level columns
AmplabJenkins removed a comment on pull request #34038: URL: https://github.com/apache/spark/pull/34038#issuecomment-926928227 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143607/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34038: [SPARK-36797][SQL] Union should resolve nested columns as top-level columns
SparkQA removed a comment on pull request #34038: URL: https://github.com/apache/spark/pull/34038#issuecomment-926769095 **[Test build #143607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143607/testReport)** for PR 34038 at commit [`80bb6e1`](https://github.com/apache/spark/commit/80bb6e135dddaa75aee1658e05681b992be91896). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34038: [SPARK-36797][SQL] Union should resolve nested columns as top-level columns
SparkQA commented on pull request #34038: URL: https://github.com/apache/spark/pull/34038#issuecomment-926927002 **[Test build #143607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143607/testReport)** for PR 34038 at commit [`80bb6e1`](https://github.com/apache/spark/commit/80bb6e135dddaa75aee1658e05681b992be91896). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
sunchao commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926922306 Actually it may not be so easy to use `ColumnarRow`, so I'm fine with exposing `ColumnarBatchRow` here. Eventually we might want to combine them since they look so similar .. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34097: [SPARK-36792][SQL][FOLLOWUP] Refactor InSet generated code
AmplabJenkins removed a comment on pull request #34097: URL: https://github.com/apache/spark/pull/34097#issuecomment-926920195 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143606/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34097: [SPARK-36792][SQL][FOLLOWUP] Refactor InSet generated code
SparkQA removed a comment on pull request #34097: URL: https://github.com/apache/spark/pull/34097#issuecomment-926759993 **[Test build #143606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143606/testReport)** for PR 34097 at commit [`6f13869`](https://github.com/apache/spark/commit/6f1386933d9678c1ca4976c518cd44fec73f8a06). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
SparkQA commented on pull request #34101: URL: https://github.com/apache/spark/pull/34101#issuecomment-926922191 **[Test build #143613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143613/testReport)** for PR 34101 at commit [`0a43396`](https://github.com/apache/spark/commit/0a43396ce3da47024db39f27ffcc9f28911cf1ab). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926920193 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143611/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA removed a comment on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926840225 **[Test build #143611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143611/testReport)** for PR 34100 at commit [`4456fc1`](https://github.com/apache/spark/commit/4456fc150a1ac0da6b8b2501976772311fefdb55). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34097: [SPARK-36792][SQL][FOLLOWUP] Refactor InSet generated code
AmplabJenkins commented on pull request #34097: URL: https://github.com/apache/spark/pull/34097#issuecomment-926920195 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143606/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
AmplabJenkins commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926920193 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/143611/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34097: [SPARK-36792][SQL][FOLLOWUP] Refactor InSet generated code
SparkQA commented on pull request #34097: URL: https://github.com/apache/spark/pull/34097#issuecomment-926918859 **[Test build #143606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143606/testReport)** for PR 34097 at commit [`6f13869`](https://github.com/apache/spark/commit/6f1386933d9678c1ca4976c518cd44fec73f8a06). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dbtsai commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
dbtsai commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926913808 +1 on using `Iterable` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ueshin opened a new pull request #34101: [SPARK-36846][PYTHON] Inline most of type hint files under pyspark/sql/pandas folder
ueshin opened a new pull request #34101: URL: https://github.com/apache/spark/pull/34101 ### What changes were proposed in this pull request? Inlines type hint files under `pyspark/sql/pandas` folder, except for `pyspark/sql/pandas/functions.pyi` and files under `pyspark/sql/pandas/_typing`. - Since the file contains a lot of overloads, we should revisit and manage it separately. - We can't inline files under `pyspark/sql/pandas/_typing` because it includes new syntax for type hints. ### Why are the changes needed? Currently there are type hint stub files (`*.pyi`) to show the expected types for functions, but we can also take advantage of static type checking within the functions by inlining the type hints. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #34087: [SPARK-36821][SQL] Make class ColumnarBatch extendable - addendum
sunchao commented on pull request #34087: URL: https://github.com/apache/spark/pull/34087#issuecomment-926912982 > We may still expose ColumnarBatchRow since any class extending the abstract class still needs it I'm thinking whether they can just use `ColumnarRow` instead. > One question for your code snippet, we should add the public method rowIterator, right? It is the major interface of the ColumnarBatch. Yea. We can have the class implement `Iterable` which is a more standard Java API. We'll need to replace all the places that use `rowInterator` with `iterator` though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34100: [SPARK-36835][FOLLOWUP][BUILD][TEST-HADOOP2.7] Fix maven issue for Hadoop 2.7 profile after enabling dependency reduced pom
SparkQA commented on pull request #34100: URL: https://github.com/apache/spark/pull/34100#issuecomment-926911899 **[Test build #143611 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/143611/testReport)** for PR 34100 at commit [`4456fc1`](https://github.com/apache/spark/commit/4456fc150a1ac0da6b8b2501976772311fefdb55). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] JoshRosen commented on pull request #33989: [SPARK-36676][SQL][BUILD] Create shaded Hive module and upgrade Guava version to 30.1.1-jre
JoshRosen commented on pull request #33989: URL: https://github.com/apache/spark/pull/33989#issuecomment-926906688 A cross-reference for other reviewers: Given that `hive-exec` shades Guava in Hive 2.3.8+ (https://github.com/apache/hive/pull/1356), I was initially confused about why we needed to do our own shading in this PR: I originally thought that it was done to shade a broader set of dependencies beyond just Guava, further isolating us from future dependency conflicts. As @viirya points out at https://github.com/apache/spark/pull/29326#issuecomment-875060042, though, Spark uses the `hive-exec-core` JAR, not `hive-exec`, so Hive's Guava shading doesn't apply (hence the need to shade here). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33253: [SPARK-36038][CORE] Speculation metrics summary at stage level
AmplabJenkins removed a comment on pull request #33253: URL: https://github.com/apache/spark/pull/33253#issuecomment-926903690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/48124/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
