[GitHub] [spark] mengxr commented on issue #24484: [SPARK-27472] add user guide for binary file data source
mengxr commented on issue #24484: [SPARK-27472] add user guide for binary file data source URL: https://github.com/apache/spark/pull/24484#issuecomment-487456424 cc: @WeichenXu123 @HyukjinKwon @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24484: [SPARK-27472] add user guide for binary file data source
SparkQA removed a comment on issue #24484: [SPARK-27472] add user guide for binary file data source URL: https://github.com/apache/spark/pull/24484#issuecomment-487454719 **[Test build #104984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104984/testReport)** for PR 24484 at commit [`bdd4529`](https://github.com/apache/spark/commit/bdd4529acce90b3ea889f8497aeeb11b326b9a64). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mengxr commented on issue #24483: [SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files
mengxr commented on issue #24483: [SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files URL: https://github.com/apache/spark/pull/24483#issuecomment-487456403 cc: @WeichenXu123 @HyukjinKwon @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24484: [SPARK-27472] add user guide for binary file data source
SparkQA commented on issue #24484: [SPARK-27472] add user guide for binary file data source URL: https://github.com/apache/spark/pull/24484#issuecomment-487455906 **[Test build #104984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104984/testReport)** for PR 24484 at commit [`bdd4529`](https://github.com/apache/spark/commit/bdd4529acce90b3ea889f8497aeeb11b326b9a64). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487455166 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
SparkQA removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487441758 **[Test build #104980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104980/testReport)** for PR 24375 at commit [`b148c6f`](https://github.com/apache/spark/commit/b148c6f119be4b0a5da0c2600add506cd930a647). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487455168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104980/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487455166 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
SparkQA commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487454951 **[Test build #104980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104980/testReport)** for PR 24375 at commit [`b148c6f`](https://github.com/apache/spark/commit/b148c6f119be4b0a5da0c2600add506cd930a647). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487454801
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104979/
Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24484: [SPARK-27472] add user guide for binary file data source
SparkQA commented on issue #24484: [SPARK-27472] add user guide for binary file data source URL: https://github.com/apache/spark/pull/24484#issuecomment-487454719 **[Test build #104984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104984/testReport)** for PR 24484 at commit [`bdd4529`](https://github.com/apache/spark/commit/bdd4529acce90b3ea889f8497aeeb11b326b9a64). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487454693 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
SparkQA removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487438204
**[Test build #104979 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104979/testReport)**
for PR 24482 at commit
[`6d13656`](https://github.com/apache/spark/commit/6d13656a4780a16b07c90683ca8e5a3c044a442d).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mengxr commented on a change in pull request #24484: [SPARK-27472] add user guide for binary file data source
mengxr commented on a change in pull request #24484: [SPARK-27472] add user guide for binary file data source URL: https://github.com/apache/spark/pull/24484#discussion_r279238983 ## File path: docs/sql-data-sources.md ## @@ -54,4 +54,5 @@ goes into specific options that are available for the built-in data sources. * [Compatibility with Databricks spark-avro](sql-data-sources-avro.html#compatibility-with-databricks-spark-avro) * [Supported types for Avro -> Spark SQL conversion](sql-data-sources-avro.html#supported-types-for-avro---spark-sql-conversion) * [Supported types for Spark SQL -> Avro conversion](sql-data-sources-avro.html#supported-types-for-spark-sql---avro-conversion) +* [Whole Binary Files](sql-data-sources-binaryFile.html) Review comment: Use `Whole Binary Files` to indicate that this is loading each file as a whole record. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487455168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104980/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487454798
Merged build finished. Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487454801
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104979/
Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487454798
Merged build finished. Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487454699 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104981/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487454699 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104981/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487454693 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
SparkQA commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487454603
**[Test build #104979 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104979/testReport)**
for PR 24482 at commit
[`6d13656`](https://github.com/apache/spark/commit/6d13656a4780a16b07c90683ca8e5a3c044a442d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487442420 **[Test build #104981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104981/testReport)** for PR 23546 at commit [`29ecdeb`](https://github.com/apache/spark/commit/29ecdebb3392213c1cc63bba7122bdf8ee671660). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mengxr opened a new pull request #24484: [SPARK-27472] add user guide for binary file data source
mengxr opened a new pull request #24484: [SPARK-27472] add user guide for binary file data source URL: https://github.com/apache/spark/pull/24484 ## What changes were proposed in this pull request? Add user guide for binary file data source. https://user-images.githubusercontent.com/829644/56877594-0488d300-6a04-11e9-9064-5047dfedd913.png;> ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) Please review http://spark.apache.org/contributing.html before opening a pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487454473 **[Test build #104981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104981/testReport)** for PR 23546 at commit [`29ecdeb`](https://github.com/apache/spark/commit/29ecdebb3392213c1cc63bba7122bdf8ee671660). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] William1104 commented on issue #24221: [SPARK-27248][SQL] refresh table should recreate cache with same cache name
William1104 commented on issue #24221: [SPARK-27248][SQL] refresh table should recreate cache with same cache name URL: https://github.com/apache/spark/pull/24221#issuecomment-487450791 Hi @cloud-fan , @gatorsmile and @srowen , Appreciate if you can take a look the PR again. I enhanced the test case to cover storage level. If we should remove the note in migration guide, I could update it accordingly as well. Many thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24483: [SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files
SparkQA commented on issue #24483: [SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files URL: https://github.com/apache/spark/pull/24483#issuecomment-487450170 **[Test build #104983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104983/testReport)** for PR 24483 at commit [`11ff2cc`](https://github.com/apache/spark/commit/11ff2cc83a591245a7c9fa1d4bfeae9419b6bfa7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mengxr opened a new pull request #24483: [SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files
mengxr opened a new pull request #24483: [SPARK-27588] Binary file data source fails fast and doesn't attempt to read very large files URL: https://github.com/apache/spark/pull/24483 ## What changes were proposed in this pull request? If a file is too big (>2GB), we should fail fast and do not try to read the file. ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) Please review http://spark.apache.org/contributing.html before opening a pull request. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487445325 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/10284/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487445337 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10284/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487445332 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487445332 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487445337 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10284/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] linehrr commented on issue #24461: [SPARK-27434][CORE] Fix mem leak
linehrr commented on issue #24461: [SPARK-27434][CORE] Fix mem leak URL: https://github.com/apache/spark/pull/24461#issuecomment-48740 @srowen if there is indeed somewhere else that's holding that filesystem object, them this won't solve the problem you are right. but if not, removing from the cache will de-reference it from the cache, therefore could make that object orphan and then can be cleaned by GC. I can't really guarantee that there aren't other places that could reference that object, but the fact that when spark context is closing makes me think there should really not be. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487443942 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/10284/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins
AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-487442918 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins
AmplabJenkins removed a comment on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-487442922 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10283/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins
AmplabJenkins commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-487442922 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10283/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins
AmplabJenkins commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-487442918 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins
wangyum commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-487442600 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487442420 **[Test build #104981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104981/testReport)** for PR 23546 at commit [`29ecdeb`](https://github.com/apache/spark/commit/29ecdebb3392213c1cc63bba7122bdf8ee671660). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins
SparkQA commented on issue #24044: [WIP][test-hadoop2.7][test-maven] Test Hadoop 3.2 on jenkins URL: https://github.com/apache/spark/pull/24044#issuecomment-487443066 **[Test build #104982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104982/testReport)** for PR 24044 at commit [`26f7e89`](https://github.com/apache/spark/commit/26f7e89cbdeff7e27facd4c0ea539c0bfb3b9017). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487442254 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
skonto commented on a change in pull request #23546: [SPARK-23153][K8s] Support
client dependencies with a Hadoop Compatible File System
URL: https://github.com/apache/spark/pull/23546#discussion_r279230019
##
File path:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/DepsTestsSuite.scala
##
@@ -0,0 +1,239 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.deploy.k8s.integrationtest
+
+import java.net.URL
+
+import scala.collection.JavaConverters._
+
+import com.amazonaws.auth.BasicAWSCredentials
+import com.amazonaws.services.s3.AmazonS3Client
+import io.fabric8.kubernetes.api.model._
+import io.fabric8.kubernetes.api.model.apps.StatefulSetBuilder
+import org.scalatest.concurrent.{Eventually, PatienceConfiguration}
+import org.scalatest.time.{Minutes, Span}
+
+import org.apache.spark.SparkException
+import
org.apache.spark.deploy.k8s.integrationtest.DepsTestsSuite.{DEPS_TIMEOUT,
FILE_CONTENTS, HOST_PATH}
+import org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.{INTERVAL,
MinikubeTag, TIMEOUT}
+import org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube
+
+private[spark] trait DepsTestsSuite { k8sSuite: KubernetesSuite =>
+ import KubernetesSuite.k8sTestTag
+
+ val cName = "ceph-nano"
+ val svcName = s"$cName-s3"
+ val bucket = "spark"
+
+ private def getCephContainer(): Container = {
+val envVars = Map ( "NETWORK_AUTO_DETECT" -> "4",
+ "RGW_CIVETWEB_PORT" -> "8000",
+ "SREE_PORT" -> "5001",
+ "CEPH_DEMO_UID" -> "nano",
+ "CEPH_DAEMON" -> "demo",
+ "DEBUG" -> "verbose"
+).map( envV =>
+ new EnvVarBuilder()
+.withName(envV._1)
+.withValue(envV._2)
+.build()
+).toArray
+
+val resources = Map(
+ "cpu" -> new QuantityBuilder()
+.withAmount("1")
+.build(),
+ "memory" -> new QuantityBuilder()
+.withAmount("512M")
+.build()
+).asJava
+
+new ContainerBuilder()
+ .withImage("ceph/daemon:v4.0.0-stable-4.0-master-centos-7-x86_64")
Review comment:
ceph/daemon:latest is broken (3 days ago), using stable, will open a bug.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins removed a comment on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487442257 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10282/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] skonto commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
skonto commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487442281 jenkins test this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487442254 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
AmplabJenkins commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487442257 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10282/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
SparkQA commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487441758 **[Test build #104980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104980/testReport)** for PR 24375 at commit [`b148c6f`](https://github.com/apache/spark/commit/b148c6f119be4b0a5da0c2600add506cd930a647). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times
cloud-fan commented on issue #24375: [SPARK-27474][CORE] avoid retrying a task failed with CommitDeniedException many times URL: https://github.com/apache/spark/pull/24375#issuecomment-487441589 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
cloud-fan commented on a change in pull request #24482: [SPARK-27581][SQL]
DataFrame countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#discussion_r279228389
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -358,9 +358,10 @@ object functions {
* @since 1.3.0
*/
@scala.annotation.varargs
- def countDistinct(expr: Column, exprs: Column*): Column = {
-withAggregateFunction(Count.apply((expr +: exprs).map(_.expr)), isDistinct
= true)
- }
+ def countDistinct(expr: Column, exprs: Column*): Column =
+// For usage like countDistinct("*"), we should let analyzer expand star
and
Review comment:
how do we handle it in `count`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #24442: [SPARK-27547][SQL] fix DataFrame self-join problems
cloud-fan commented on a change in pull request #24442: [SPARK-27547][SQL] fix
DataFrame self-join problems
URL: https://github.com/apache/spark/pull/24442#discussion_r279227348
##
File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
##
@@ -1292,10 +1323,24 @@ class Dataset[T] private[sql](
colRegex(colName)
} else {
val expr = resolve(colName)
-Column(expr)
+Column(addDataFrameIdToCol(expr))
Review comment:
Internally, I think we should never compare 2 attributes by `equal` not
`semanticEqual`.
Externally, if users rely on the equality of `Column`, yes this PR may break
it under some cases. If this matters, I can add a migration guide or update
`Column.equal`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value
AmplabJenkins commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487437861 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value
AmplabJenkins commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487437867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104978/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value
SparkQA commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487437686 **[Test build #104978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104978/testReport)** for PR 24465 at commit [`567e58e`](https://github.com/apache/spark/commit/567e58e51fc97b5a9da36456b87cc3ec6367b502). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value
SparkQA removed a comment on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487428177 **[Test build #104978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104978/testReport)** for PR 24465 at commit [`567e58e`](https://github.com/apache/spark/commit/567e58e51fc97b5a9da36456b87cc3ec6367b502). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
SparkQA commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487438204
**[Test build #104979 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104979/testReport)**
for PR 24482 at commit
[`6d13656`](https://github.com/apache/spark/commit/6d13656a4780a16b07c90683ca8e5a3c044a442d).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487438065
Merged build finished. Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487438067
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10281/
Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487438065
Merged build finished. Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487438067
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10281/
Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value
AmplabJenkins removed a comment on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487437867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104978/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value
AmplabJenkins removed a comment on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487437861 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module)
AmplabJenkins removed a comment on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module) URL: https://github.com/apache/spark/pull/24391#issuecomment-487437211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104975/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module)
AmplabJenkins removed a comment on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module) URL: https://github.com/apache/spark/pull/24391#issuecomment-487437208 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module)
AmplabJenkins commented on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module) URL: https://github.com/apache/spark/pull/24391#issuecomment-487437208 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module)
AmplabJenkins commented on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module) URL: https://github.com/apache/spark/pull/24391#issuecomment-487437211 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104975/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module)
SparkQA removed a comment on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module) URL: https://github.com/apache/spark/pull/24391#issuecomment-487416048 **[Test build #104975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104975/testReport)** for PR 24391 at commit [`262be84`](https://github.com/apache/spark/commit/262be84bc1991401422038b1d763bfd4762618d1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module)
SparkQA commented on issue #24391: [SPARK-27402][SQL][test-hadoop3.2][test-maven] Fix hadoop-3.2 test issue(except the hive-thriftserver module) URL: https://github.com/apache/spark/pull/24391#issuecomment-487436988 **[Test build #104975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104975/testReport)** for PR 24391 at commit [`262be84`](https://github.com/apache/spark/commit/262be84bc1991401422038b1d763bfd4762618d1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
viirya commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487434662
cc @dongjoon-hyun @cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #24467: [SPARK-27568][CORE] Fix readLock leak while calling take()/first() on a cached rdd
Ngone51 commented on issue #24467: [SPARK-27568][CORE] Fix readLock leak while calling take()/first() on a cached rdd URL: https://github.com/apache/spark/pull/24467#issuecomment-487433811 @srowen Yeah, agree with that. I'm thinking of it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value
SparkQA commented on issue #24465: [SPARK-27575][CORE][YARN] Yarn file-related confs should merge new value with existing value URL: https://github.com/apache/spark/pull/24465#issuecomment-487428177 **[Test build #104978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104978/testReport)** for PR 24465 at commit [`567e58e`](https://github.com/apache/spark/commit/567e58e51fc97b5a9da36456b87cc3ec6367b502). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487427301 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487427304 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104977/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487427301 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487427304 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104977/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487419066 **[Test build #104977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104977/testReport)** for PR 23546 at commit [`bea2ce9`](https://github.com/apache/spark/commit/bea2ce92156cd26b3815a508fcb9450873f962c1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487427206 **[Test build #104977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104977/testReport)** for PR 23546 at commit [`bea2ce9`](https://github.com/apache/spark/commit/bea2ce92156cd26b3815a508fcb9450873f962c1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487427043
Merged build finished. Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487427045
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104973/
Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487427045
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104973/
Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
AmplabJenkins commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487427043
Merged build finished. Test PASSed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
SparkQA removed a comment on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487416037
**[Test build #104973 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104973/testReport)**
for PR 24482 at commit
[`351cf1e`](https://github.com/apache/spark/commit/351cf1ed969798ea07fda3c5be9d824b98e59a39).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24482: [SPARK-27581][SQL] DataFrame countDistinct("*") shouldn't fail with AnalysisException
SparkQA commented on issue #24482: [SPARK-27581][SQL] DataFrame
countDistinct("*") shouldn't fail with AnalysisException
URL: https://github.com/apache/spark/pull/24482#issuecomment-487426924
**[Test build #104973 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104973/testReport)**
for PR 24482 at commit
[`351cf1e`](https://github.com/apache/spark/commit/351cf1ed969798ea07fda3c5be9d824b98e59a39).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487425336 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104976/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487425333 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487425336 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104976/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487425333 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487416058 **[Test build #104976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104976/testReport)** for PR 23546 at commit [`bea2ce9`](https://github.com/apache/spark/commit/bea2ce92156cd26b3815a508fcb9450873f962c1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
SparkQA commented on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487425190 **[Test build #104976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104976/testReport)** for PR 23546 at commit [`bea2ce9`](https://github.com/apache/spark/commit/bea2ce92156cd26b3815a508fcb9450873f962c1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1
AmplabJenkins removed a comment on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1 URL: https://github.com/apache/spark/pull/24481#issuecomment-487423886 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1
AmplabJenkins removed a comment on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1 URL: https://github.com/apache/spark/pull/24481#issuecomment-487423889 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104974/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1
AmplabJenkins commented on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1 URL: https://github.com/apache/spark/pull/24481#issuecomment-487423889 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/104974/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1
AmplabJenkins commented on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1 URL: https://github.com/apache/spark/pull/24481#issuecomment-487423886 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1
SparkQA commented on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1 URL: https://github.com/apache/spark/pull/24481#issuecomment-487423761 **[Test build #104974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104974/testReport)** for PR 24481 at commit [`1eeeb5e`](https://github.com/apache/spark/commit/1eeeb5ebdbd94379260f022e4caedb173e643ee7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1
SparkQA removed a comment on issue #24481: [WIP][SPARK-27467][BUILD] Upgrade Maven to 3.6.1 URL: https://github.com/apache/spark/pull/24481#issuecomment-487416038 **[Test build #104974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/104974/testReport)** for PR 24481 at commit [`1eeeb5e`](https://github.com/apache/spark/commit/1eeeb5ebdbd94379260f022e4caedb173e643ee7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] srowen commented on issue #24461: [SPARK-27434][CORE] Fix mem leak
srowen commented on issue #24461: [SPARK-27434][CORE] Fix mem leak URL: https://github.com/apache/spark/pull/24461#issuecomment-487422243 Hm your heap dump is convincing, although this `FileSystem` is just obtained from Hadoop and its cache. Generally that's not safe to close, but maybe here. I'm trying to figure out if closing it solves the problem though. You'd still have something holding a reference to this object, no? And that I don't quite see here, why that is. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] skonto removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
skonto removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487419487 @shaneknapp there is a weird error: ``` 9/04/28 14:34:17.480 OkHttp https://192.168.39.110:8443/... ERROR ExecWebSocketListener: Exec Failure: HTTP:400. Message:pod spark-test-app-ade7c8acff964400b212005a8ee56832 does not have a host assigned java.net.ProtocolException: Expected HTTP 101 response but was '400 Bad Request' at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:229) ``` I suspect the pod was never scheduled? What resources are assigned to minikube? This was previously working above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487420346 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10280/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System
AmplabJenkins removed a comment on issue #23546: [SPARK-23153][K8s] Support client dependencies with a Hadoop Compatible File System URL: https://github.com/apache/spark/pull/23546#issuecomment-487420345 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
