[GitHub] [spark] SparkQA commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
SparkQA commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504630827 **[Test build #106789 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106789/testReport)** for PR 24936 at commit [`c52cb06`](https://github.com/apache/spark/commit/c52cb0627c8f221763c9c42b681867bc1ff4fba8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504624104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106785/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504624102 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504624102 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504624104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106785/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
SparkQA removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504611055 **[Test build #106785 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106785/testReport)** for PR 24865 at commit [`1cd73c3`](https://github.com/apache/spark/commit/1cd73c375da4a72822ea3355170fd4af6ff622c5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504623962 **[Test build #106785 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106785/testReport)** for PR 24865 at commit [`1cd73c3`](https://github.com/apache/spark/commit/1cd73c375da4a72822ea3355170fd4af6ff622c5). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621884 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106788/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621882 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621882 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621884 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106788/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
SparkQA removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619664 **[Test build #106788 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106788/testReport)** for PR 24927 at commit [`9a04e70`](https://github.com/apache/spark/commit/9a04e70a1fa24595033472edd9c20b355a9479b6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621777 **[Test build #106788 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106788/testReport)** for PR 24927 at commit [`9a04e70`](https://github.com/apache/spark/commit/9a04e70a1fa24595033472edd9c20b355a9479b6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621080 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106787/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621077 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621080 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106787/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621077 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
SparkQA removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619203 **[Test build #106787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106787/testReport)** for PR 24927 at commit [`b9983c0`](https://github.com/apache/spark/commit/b9983c02a5319cdd9b3c6ab8411d6757141fbcd7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504621008 **[Test build #106787 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106787/testReport)** for PR 24927 at commit [`b9983c0`](https://github.com/apache/spark/commit/b9983c02a5319cdd9b3c6ab8411d6757141fbcd7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619664 **[Test build #106788 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106788/testReport)** for PR 24927 at commit [`9a04e70`](https://github.com/apache/spark/commit/9a04e70a1fa24595033472edd9c20b355a9479b6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619549 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619550 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12008/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619550 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12008/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
AmplabJenkins commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619549 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
HyukjinKwon removed a comment on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619121 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
SparkQA commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619203 **[Test build #106787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106787/testReport)** for PR 24927 at commit [`b9983c0`](https://github.com/apache/spark/commit/b9983c02a5319cdd9b3c6ab8411d6757141fbcd7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark
HyukjinKwon commented on issue #24927: [SPARK-28130][PYTHON] Print pretty messages for skipped tests when xmlrunner is available in PySpark URL: https://github.com/apache/spark/pull/24927#issuecomment-504619121 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
HyukjinKwon closed pull request #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
HyukjinKwon commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926#issuecomment-504618906 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24930: [SPARK-28132][PYTHON] Update document type conversion for Pandas UDFs (pyarrow 0.13.0, pandas 0.24.2, Python 3.7)
HyukjinKwon commented on issue #24930: [SPARK-28132][PYTHON] Update document type conversion for Pandas UDFs (pyarrow 0.13.0, pandas 0.24.2, Python 3.7) URL: https://github.com/apache/spark/pull/24930#issuecomment-504617839 Thanks @viirya and @BryanCutler ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #24929: [SPARK-28131][PYTHON] Update document type conversion between Python data and SQL types in normal UDFs (Python 3.7)
HyukjinKwon commented on issue #24929: [SPARK-28131][PYTHON] Update document type conversion between Python data and SQL types in normal UDFs (Python 3.7) URL: https://github.com/apache/spark/pull/24929#issuecomment-504617801 Thanks @BryanCutler ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] squito commented on a change in pull request #24935: [SPARK-28005][SCHEDULER] Remove unnecessary log from SparkRackResolver
squito commented on a change in pull request #24935: [SPARK-28005][SCHEDULER]
Remove unnecessary log from SparkRackResolver
URL: https://github.com/apache/spark/pull/24935#discussion_r296430094
##
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/SparkRackResolver.scala
##
@@ -72,8 +72,6 @@ private[spark] class SparkRackResolver(conf: Configuration)
extends Logging {
val rNameList = dnsToSwitchMapping.resolve(hostNames.toList.asJava).asScala
if (rNameList == null || rNameList.isEmpty) {
hostNames.foreach(nodes += new NodeBase(_, NetworkTopology.DEFAULT_RACK))
- logInfo(s"Got an error when resolving hostNames. " +
Review comment:
yeah I don't think we want to delete this log line. In fact I'd keep it at
INFO, but I'd prevent logging this when `hostNames.isEmpty`. (In fact, the
entire function can be a no-op when `hostNames.isEmpty`). Its a useful log msg
when there really is an error -- but when the argument is empty, its logging
even with no error. sorry if my description was unclear.
(also sorry for the late comment -- I thought I posted something along these
lines this morning but guess I forgot to submit it ..)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504617001 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106783/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504616999 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504616999 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504617001 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106783/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504616879 **[Test build #106783 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106783/testReport)** for PR 24865 at commit [`11ab15e`](https://github.com/apache/spark/commit/11ab15e20bf2facbacb048939a6cb6bc9ba7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
SparkQA removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504596718 **[Test build #106783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106783/testReport)** for PR 24865 at commit [`11ab15e`](https://github.com/apache/spark/commit/11ab15e20bf2facbacb048939a6cb6bc9ba7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum closed pull request #24908: [SPARK-28093][SPARK-28109][SQL][2.3] Fix TRIM/LTRIM/RTRIM function parameter order issue
wangyum closed pull request #24908: [SPARK-28093][SPARK-28109][SQL][2.3] Fix TRIM/LTRIM/RTRIM function parameter order issue URL: https://github.com/apache/spark/pull/24908 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #24908: [SPARK-28093][SPARK-28109][SQL][2.3] Fix TRIM/LTRIM/RTRIM function parameter order issue
dongjoon-hyun commented on issue #24908: [SPARK-28093][SPARK-28109][SQL][2.3] Fix TRIM/LTRIM/RTRIM function parameter order issue URL: https://github.com/apache/spark/pull/24908#issuecomment-504616268 Thank you, @wangyum . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs
AmplabJenkins removed a comment on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs URL: https://github.com/apache/spark/pull/20350#issuecomment-504613343 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106782/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs
AmplabJenkins removed a comment on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs URL: https://github.com/apache/spark/pull/20350#issuecomment-504613341 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs
AmplabJenkins commented on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs URL: https://github.com/apache/spark/pull/20350#issuecomment-504613343 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106782/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs
AmplabJenkins commented on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs URL: https://github.com/apache/spark/pull/20350#issuecomment-504613341 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs
SparkQA removed a comment on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs URL: https://github.com/apache/spark/pull/20350#issuecomment-504585374 **[Test build #106782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106782/testReport)** for PR 20350 at commit [`bc25c0d`](https://github.com/apache/spark/commit/bc25c0dbc0e9df3adfc425de09ea1918199eb2e0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs
SparkQA commented on issue #20350: [SPARK-23179][SQL] Support option to throw exception if overflow occurs URL: https://github.com/apache/spark/pull/20350#issuecomment-504613200 **[Test build #106782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106782/testReport)** for PR 20350 at commit [`bc25c0d`](https://github.com/apache/spark/commit/bc25c0dbc0e9df3adfc425de09ea1918199eb2e0). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
AmplabJenkins removed a comment on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504612176 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106786/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
AmplabJenkins removed a comment on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504612173 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
SparkQA removed a comment on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504611755 **[Test build #106786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106786/testReport)** for PR 24936 at commit [`774bcee`](https://github.com/apache/spark/commit/774bcee20a9e94c5ba142ac60d03ec552616ae56). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
AmplabJenkins commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504612173 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
AmplabJenkins commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504612176 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106786/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
SparkQA commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504612169 **[Test build #106786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106786/testReport)** for PR 24936 at commit [`774bcee`](https://github.com/apache/spark/commit/774bcee20a9e94c5ba142ac60d03ec552616ae56). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark
SparkQA commented on issue #24936: [SPARK-24634][SS] Add a new metric regarding number of rows later than watermark URL: https://github.com/apache/spark/pull/24936#issuecomment-504611755 **[Test build #106786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106786/testReport)** for PR 24936 at commit [`774bcee`](https://github.com/apache/spark/commit/774bcee20a9e94c5ba142ac60d03ec552616ae56). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504611586 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504611586 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504611587 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12007/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504611587 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12007/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504611055 **[Test build #106785 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106785/testReport)** for PR 24865 at commit [`1cd73c3`](https://github.com/apache/spark/commit/1cd73c375da4a72822ea3355170fd4af6ff622c5). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504605387 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504605388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106784/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
SparkQA removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504602835 **[Test build #106784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106784/testReport)** for PR 24906 at commit [`b00d4a2`](https://github.com/apache/spark/commit/b00d4a2da2f7c69cc5fc48403d4e07386d85ce16). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result
AmplabJenkins removed a comment on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result URL: https://github.com/apache/spark/pull/21599#issuecomment-504605310 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106781/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504605387 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504605388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106784/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
SparkQA commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504605348 **[Test build #106784 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106784/testReport)** for PR 24906 at commit [`b00d4a2`](https://github.com/apache/spark/commit/b00d4a2da2f7c69cc5fc48403d4e07386d85ce16). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result
AmplabJenkins removed a comment on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result URL: https://github.com/apache/spark/pull/21599#issuecomment-504605306 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result
SparkQA removed a comment on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result URL: https://github.com/apache/spark/pull/21599#issuecomment-504579844 **[Test build #106781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106781/testReport)** for PR 21599 at commit [`77f26f2`](https://github.com/apache/spark/commit/77f26f2c25d3be65a4ae6d0e277acebb1e09d616). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result
AmplabJenkins commented on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result URL: https://github.com/apache/spark/pull/21599#issuecomment-504605310 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106781/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result
AmplabJenkins commented on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result URL: https://github.com/apache/spark/pull/21599#issuecomment-504605306 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result
SparkQA commented on issue #21599: [SPARK-26218][SQL] Overflow on arithmetic operations returns incorrect result URL: https://github.com/apache/spark/pull/21599#issuecomment-504605218 **[Test build #106781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106781/testReport)** for PR 21599 at commit [`77f26f2`](https://github.com/apache/spark/commit/77f26f2c25d3be65a4ae6d0e277acebb1e09d616). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504603581 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12006/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504603578 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504603581 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12006/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504603578 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL]
Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
URL: https://github.com/apache/spark/pull/24865#discussion_r296420442
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -735,4 +735,54 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
df1.groupBy("j").agg(max("k")))
}
}
+
+ // A test with a partition where the number of files in the partition is
+ // large. tests for the condition where the serialization of such a task
may result in a stack
+ // overflow if the files list is stored in a recursive data structure
+ // This test is ignored because it takes long to run (~3 min)
+ ignore("SPARK-27100 stack overflow: read data with large partitions") {
+val nCount = 12000
+// reshuffle data so that many small files are created
+val nShufflePartitions = 6000
+// and with one table partition, should result in 6000 files in one
partition
Review comment:
Do you need to upgrade `nCount` for some reason?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL]
Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
URL: https://github.com/apache/spark/pull/24865#discussion_r296420346
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -735,4 +735,54 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
df1.groupBy("j").agg(max("k")))
}
}
+
+ // A test with a partition where the number of files in the partition is
+ // large. tests for the condition where the serialization of such a task
may result in a stack
+ // overflow if the files list is stored in a recursive data structure
+ // This test is ignored because it takes long to run (~3 min)
+ ignore("SPARK-27100 stack overflow: read data with large partitions") {
+val nCount = 12000
+// reshuffle data so that many small files are created
+val nShufflePartitions = 6000
+// and with one table partition, should result in 6000 files in one
partition
Review comment:
Ur, @parthchandra . ~I gave you two lines including the comment.
Could you fix the comment, too?~ I'll check this again~
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL]
Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
URL: https://github.com/apache/spark/pull/24865#discussion_r296420442
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -735,4 +735,54 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
df1.groupBy("j").agg(max("k")))
}
}
+
+ // A test with a partition where the number of files in the partition is
+ // large. tests for the condition where the serialization of such a task
may result in a stack
+ // overflow if the files list is stored in a recursive data structure
+ // This test is ignored because it takes long to run (~3 min)
+ ignore("SPARK-27100 stack overflow: read data with large partitions") {
+val nCount = 12000
+// reshuffle data so that many small files are created
+val nShufflePartitions = 6000
+// and with one table partition, should result in 6000 files in one
partition
Review comment:
Do you need to upgrade `nCount` for some reason?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL]
Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
URL: https://github.com/apache/spark/pull/24865#discussion_r296420442
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -735,4 +735,54 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
df1.groupBy("j").agg(max("k")))
}
}
+
+ // A test with a partition where the number of files in the partition is
+ // large. tests for the condition where the serialization of such a task
may result in a stack
+ // overflow if the files list is stored in a recursive data structure
+ // This test is ignored because it takes long to run (~3 min)
+ ignore("SPARK-27100 stack overflow: read data with large partitions") {
+val nCount = 12000
+// reshuffle data so that many small files are created
+val nShufflePartitions = 6000
+// and with one table partition, should result in 6000 files in one
partition
Review comment:
BTW, I didn't ask the `nCount`. Do you need to upgrade that for some reason?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
SparkQA commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504602835 **[Test build #106784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106784/testReport)** for PR 24906 at commit [`b00d4a2`](https://github.com/apache/spark/commit/b00d4a2da2f7c69cc5fc48403d4e07386d85ce16). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504602626 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12005/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins removed a comment on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504602622 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504602622 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
dongjoon-hyun commented on a change in pull request #24865: [SPARK-27100][SQL]
Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
URL: https://github.com/apache/spark/pull/24865#discussion_r296420346
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -735,4 +735,54 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
df1.groupBy("j").agg(max("k")))
}
}
+
+ // A test with a partition where the number of files in the partition is
+ // large. tests for the condition where the serialization of such a task
may result in a stack
+ // overflow if the files list is stored in a recursive data structure
+ // This test is ignored because it takes long to run (~3 min)
+ ignore("SPARK-27100 stack overflow: read data with large partitions") {
+val nCount = 12000
+// reshuffle data so that many small files are created
+val nShufflePartitions = 6000
+// and with one table partition, should result in 6000 files in one
partition
Review comment:
Ur, @parthchandra . I gave you two lines including the comment.
Could you fix the comment, too?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
AmplabJenkins commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504602626 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12005/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation
gatorsmile commented on issue #24906: [SPARK-28104][SQL] Implement Spark's own GetColumnsOperation URL: https://github.com/apache/spark/pull/24906#issuecomment-504602446 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
SparkQA commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504596718 **[Test build #106783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106783/testReport)** for PR 24865 at commit [`11ab15e`](https://github.com/apache/spark/commit/11ab15e20bf2facbacb048939a6cb6bc9ba7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
AmplabJenkins removed a comment on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926#issuecomment-504596413 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
AmplabJenkins removed a comment on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926#issuecomment-504596418 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106779/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504596278 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504596285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12004/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
AmplabJenkins commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926#issuecomment-504596413 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
AmplabJenkins commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926#issuecomment-504596418 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106779/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504596278 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504596285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/12004/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
SparkQA removed a comment on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926#issuecomment-504549294 **[Test build #106779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106779/testReport)** for PR 24926 at commit [`9be0110`](https://github.com/apache/spark/commit/9be0110c7b7dee443d30e3188d9f51eaaf7fd3e1). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions
SparkQA commented on issue #24926: [SPARK-28128][PYTHON][SQL] Pandas Grouped UDFs skip empty partitions URL: https://github.com/apache/spark/pull/24926#issuecomment-504596029 **[Test build #106779 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/106779/testReport)** for PR 24926 at commit [`9be0110`](https://github.com/apache/spark/commit/9be0110c7b7dee443d30e3188d9f51eaaf7fd3e1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] parthchandra commented on a change in pull request #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
parthchandra commented on a change in pull request #24865: [SPARK-27100][SQL]
Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
URL: https://github.com/apache/spark/pull/24865#discussion_r296414270
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedReadSuite.scala
##
@@ -735,4 +735,54 @@ abstract class BucketedReadSuite extends QueryTest with
SQLTestUtils {
df1.groupBy("j").agg(max("k")))
}
}
+
+ // A test with a partition where the number of files in the partition is
+ // large. tests for the condition where the serialization of such a task
may result in a stack
+ // overflow if the files list is stored in a recursive data structure
+ // This test is ignored because it takes long to run (~3 min)
+ ignore("SPARK-27100 stack overflow: read data with large partitions") {
+val nCount = 12000
+// reshuffle data so that many small files are created
+val nShufflePartitions = 6000
+// and with one table partition, should result in 6000 files in one
partition
Review comment:
Done. So many things to remember to check :) Thank you for being so careful.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504587567 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins removed a comment on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504587570 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106778/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError `
AmplabJenkins commented on issue #24865: [SPARK-27100][SQL] Use `Array` instead of `Seq` in `FilePartition` to prevent `StackOverflowError ` URL: https://github.com/apache/spark/pull/24865#issuecomment-504587567 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
