[GitHub] [spark] cloud-fan commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables
cloud-fan commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables URL: https://github.com/apache/spark/pull/27391#issuecomment-582285823 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27465: [WIP][K8S] Flaky DepsTestsSuite Test
dongjoon-hyun closed pull request #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
HyukjinKwon edited a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582284510 @shaneknapp to summarize what happen so far: - Seems R version itself bumped up correctly to 3.5.2; however, seems we should reinstall the packages installed previously, including `r-base`. It caused some problems with error messages such as the below (please refer the PR description). ``` * creating vignettes ... ERROR Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics: package ���htmltools��� was installed by an R version with different internals; it needs to be reinstalled for use with this R version ``` Currently, I fixed it by making tests permissive and just skipping CRAN check but we should at least reenable CRAN back. Can we reinstall the packages previously installed for R 3.5.2? Once this is done, I should reenable it [here](https://github.com/apache/spark/pull/27460/files#diff-9e63dce390dd4d80c6abe3413a8b1984R34) per SPARK-30737. It should be able to test via manually calling `./R/check-cran.sh` or reverting the changes made in this PR at `R/run-tests.sh `. - Looks like Arrow R library was unable to find after the upgrade: ``` test_sparkSQL_arrow.R:25: skip: createDataFrame/collect Arrow optimization arrow cannot be loaded ``` This seems causing to skip Arrow related tests in Jenkins. - I think this is minor but seems there are some enviornment issues assuming from: ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time Your system is mis-configured: ���/etc/localtime��� is not a symlink ``` ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time It is strongly recommended to set envionment variable TZ to ���America/Los_Angeles��� (or equivalent) ``` Once this is done, I should remove [this line](https://github.com/apache/spark/pull/27460/files#diff-d4011863c8b176830365b2f224a84bf2R504). But this isn't really a big deal. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables
AmplabJenkins removed a comment on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables URL: https://github.com/apache/spark/pull/27391#issuecomment-582285129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117892/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables
AmplabJenkins commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables URL: https://github.com/apache/spark/pull/27391#issuecomment-582285129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117892/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables
AmplabJenkins removed a comment on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables URL: https://github.com/apache/spark/pull/27391#issuecomment-582285121 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables
AmplabJenkins commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables URL: https://github.com/apache/spark/pull/27391#issuecomment-582285121 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables
SparkQA removed a comment on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables URL: https://github.com/apache/spark/pull/27391#issuecomment-582235096 **[Test build #117892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117892/testReport)** for PR 27391 at commit [`c40895f`](https://github.com/apache/spark/commit/c40895ff5308e3cbd9b35880a988ef9ef61ba578). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
HyukjinKwon edited a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582284510 @shaneknapp to summarize what happen so far: - Seems R version itself bumped up correctly to 3.5.2; however, seems we should reinstall the packages installed previously, including `r-base`. It caused some problems with error messages such as the below (please refer the PR description). ``` * creating vignettes ... ERROR Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics: package ���htmltools��� was installed by an R version with different internals; it needs to be reinstalled for use with this R version ``` Currently, I fixed it by making tests permissive and just skipping CRAN check but we should at least reenable CRAN back. Can we reinstall the packages previously installed for R 3.5.2? Once this is done, I should reenable it [here](https://github.com/apache/spark/pull/27460/files#diff-9e63dce390dd4d80c6abe3413a8b1984R34) per SPARK-30737 - Looks like Arrow R library was unable to find after the upgrade: ``` test_sparkSQL_arrow.R:25: skip: createDataFrame/collect Arrow optimization arrow cannot be loaded ``` This seems causing to skip Arrow related tests in Jenkins. - I think this is minor but seems there are some enviornment issues assuming from: ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time Your system is mis-configured: ���/etc/localtime��� is not a symlink ``` ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time It is strongly recommended to set envionment variable TZ to ���America/Los_Angeles��� (or equivalent) ``` Once this is done, I should remove [this line](https://github.com/apache/spark/pull/27460/files#diff-d4011863c8b176830365b2f224a84bf2R504). But this isn't really a big deal. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables
SparkQA commented on issue #27391: [SPARK-30612][SQL] Resolve qualified column name with v2 tables URL: https://github.com/apache/spark/pull/27391#issuecomment-582284950 **[Test build #117892 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117892/testReport)** for PR 27391 at commit [`c40895f`](https://github.com/apache/spark/commit/c40895ff5308e3cbd9b35880a988ef9ef61ba578). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #27441: [SPARK-30668][SQL] Support `SimpleDateFormat` patterns in parsing timestamps/dates strings
MaxGekk commented on issue #27441: [SPARK-30668][SQL] Support `SimpleDateFormat` patterns in parsing timestamps/dates strings URL: https://github.com/apache/spark/pull/27441#issuecomment-582284712 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
HyukjinKwon edited a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582284510 @shaneknapp to summarize what happen so far: - Seems R version itself bumped up correctly to 3.5.2; however, seems we should reinstall the packages installed previously, including `r-base`. It caused some problems with error messages such as: ``` * creating vignettes ... ERROR Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics: package ���htmltools��� was installed by an R version with different internals; it needs to be reinstalled for use with this R version ``` Currently, I fixed it by making tests permissive and just skipping CRAN check but we should at least reenable CRAN back. Can we reinstall the packages previously installed for R 3.5.2? Once this is done, I should reenable it [here](https://github.com/apache/spark/pull/27460/files#diff-9e63dce390dd4d80c6abe3413a8b1984R34) per SPARK-30737 - Looks like Arrow R library was unable to find after the upgrade: ``` test_sparkSQL_arrow.R:25: skip: createDataFrame/collect Arrow optimization arrow cannot be loaded ``` This seems causing to skip Arrow related tests in Jenkins. - I think this is minor but seems there are some enviornment issues assuming from: ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time Your system is mis-configured: ���/etc/localtime��� is not a symlink ``` ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time It is strongly recommended to set envionment variable TZ to ���America/Los_Angeles��� (or equivalent) ``` Once this is done, I should remove [this line](https://github.com/apache/spark/pull/27460/files#diff-d4011863c8b176830365b2f224a84bf2R504). But this isn't really a big deal. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
HyukjinKwon commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582284510 @shaneknapp to summarize what happen so far: - Seems R version itself bumped up correctly to 3.5.2; however, seems we should reinstall the packages installed previously. It caused some problems with error messages such as: ``` * creating vignettes ... ERROR Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics: package ���htmltools��� was installed by an R version with different internals; it needs to be reinstalled for use with this R version ``` Currently, I fixed it by making tests permissive and just skipping CRAN check but we should at least reenable CRAN back. Can we reinstall the packages previously installed for R 3.5.2? Once this is done, I should reenable it [here](https://github.com/apache/spark/pull/27460/files#diff-9e63dce390dd4d80c6abe3413a8b1984R34) per SPARK-30737 - Looks like Arrow R library was unable to find after the upgrade: ``` test_sparkSQL_arrow.R:25: skip: createDataFrame/collect Arrow optimization arrow cannot be loaded ``` This seems causing to skip Arrow related tests in Jenkins. - I think this is minor but seems there are some enviornment issues assuming from: ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time Your system is mis-configured: ���/etc/localtime��� is not a symlink ``` ``` test_sparkSQL.R:499: warning: SPARK-17811: can create DataFrame containing NA as date and time It is strongly recommended to set envionment variable TZ to ���America/Los_Angeles��� (or equivalent) ``` Once this is done, I should remove [this line](https://github.com/apache/spark/pull/27460/files#diff-d4011863c8b176830365b2f224a84bf2R504). But this isn't really a big deal. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582284294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22670/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582284289 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582284294 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22670/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582284289 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
SparkQA commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582283885 **[Test build #117908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117908/testReport)** for PR 27439 at commit [`5929571`](https://github.com/apache/spark/commit/5929571c48c5f39eb3738428ef602a1e75a36888). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582282110 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117876/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582283346 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117890/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582283336 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582283336 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582283346 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117890/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
SparkQA removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582235060 **[Test build #117890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117890/testReport)** for PR 27463 at commit [`4201a93`](https://github.com/apache/spark/commit/4201a9352419850508a6183f9dcf607f71d9af29). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
SparkQA commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582282764 **[Test build #117890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117890/testReport)** for PR 27463 at commit [`4201a93`](https://github.com/apache/spark/commit/4201a9352419850508a6183f9dcf607f71d9af29). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582282101 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582282101 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
HyukjinKwon commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582282056 Merged to master, branch-3.0, and branch-2.4. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27306: [SPARK-30594][CORE] Do not post SparkListenerBlockUpdated when updateBlockInfo returns false
Ngone51 commented on a change in pull request #27306: [SPARK-30594][CORE] Do
not post SparkListenerBlockUpdated when updateBlockInfo returns false
URL: https://github.com/apache/spark/pull/27306#discussion_r375100365
##
File path: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala
##
@@ -1693,6 +1696,16 @@ class BlockManagerSuite extends SparkFunSuite with
Matchers with BeforeAndAfterE
assert(locs(blockIds(0)) == expectedLocs)
}
+ test("SPARK-30594: Do not post SparkListenerBlockUpdated when
updateBlockInfo returns false") {
+// update block info for non-existent block manager
+val updateInfo = UpdateBlockInfo(BlockManagerId("1", "host1", 100),
+ BlockId("test_1"), StorageLevel.MEMORY_ONLY, 1, 1)
+val result = master.driverEndpoint.askSync[Boolean](updateInfo)
Review comment:
Because the `BlockManagerId("1", "host1", 100)` hasn't registered at driver.
And driver will response `false` for the block from a non-existent block
manager.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
AmplabJenkins commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582282110 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117876/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
SparkQA removed a comment on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582221476 **[Test build #117876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117876/testReport)** for PR 27439 at commit [`50d713e`](https://github.com/apache/spark/commit/50d713e9293aed183f496a29ad300049fed4a84d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
maropu commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582281756 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
HyukjinKwon closed pull request #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
HyukjinKwon commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582281488 I am going to merge to unblock other PRs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2)
SparkQA commented on issue #27439: [SPARK-29108][SQL][TESTS][FOLLOWUP] Comment out no use and insert into statement of window.sql (Part 2) URL: https://github.com/apache/spark/pull/27439#issuecomment-582281507 **[Test build #117876 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117876/testReport)** for PR 27439 at commit [`50d713e`](https://github.com/apache/spark/commit/50d713e9293aed183f496a29ad300049fed4a84d). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character.
maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL
fails to parse when comment contains an unmatched quote character.
URL: https://github.com/apache/spark/pull/27321#discussion_r375097596
##
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
##
@@ -400,4 +400,21 @@ class CliSuite extends SparkFunSuite with
BeforeAndAfterAll with Logging {
-> "1.00"
)
}
+
+ test("SPARK-30049 Should not complaint for quotes in commented lines") {
+runCliWithin(1.minute)(
+ """SELECT concat('test', 'comment') -- someone's comment here
+|;""".stripMargin -> "testcomment"
+)
+ }
+
+ test("SPARK-30049 Should not complaint for quotes in commented with
multi-lines") {
+runCliWithin(1.minute)(
+ """SELECT concat('test', 'comment') -- someone's comment here \\
+|comment continues here with single ' quote
+| extra '
+|;""".stripMargin -> "testcomment"
+)
+ }
+
Review comment:
nit: Can you avoid this unnecessary blank?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character.
maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL
fails to parse when comment contains an unmatched quote character.
URL: https://github.com/apache/spark/pull/27321#discussion_r375098451
##
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##
@@ -509,24 +509,40 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
private def splitSemiColon(line: String): JList[String] = {
var insideSingleQuote = false
var insideDoubleQuote = false
+var insideComment = false
var escape = false
var beginIndex = 0
+var endIndex = line.length
val ret = new JArrayList[String]
+
for (index <- 0 until line.length) {
- if (line.charAt(index) == '\'') {
+ if (line.charAt(index) == '\'' && !insideComment) {
// take a look to see if it is escaped
if (!escape) {
// flip the boolean variable
insideSingleQuote = !insideSingleQuote
}
- } else if (line.charAt(index) == '\"') {
+ } else if (line.charAt(index) == '\"' && !insideComment) {
// take a look to see if it is escaped
if (!escape) {
// flip the boolean variable
insideDoubleQuote = !insideDoubleQuote
}
+ } else if (line.charAt(index) == '-') {
+val hasNext: Boolean = index + 1 < line.length
+if (insideDoubleQuote || insideSingleQuote || insideComment) {
+ // ignore
Review comment:
nit: Can you add more comments here about which case (with example queries)
we can ignore?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character.
maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL
fails to parse when comment contains an unmatched quote character.
URL: https://github.com/apache/spark/pull/27321#discussion_r375098451
##
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##
@@ -509,24 +509,40 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
private def splitSemiColon(line: String): JList[String] = {
var insideSingleQuote = false
var insideDoubleQuote = false
+var insideComment = false
var escape = false
var beginIndex = 0
+var endIndex = line.length
val ret = new JArrayList[String]
+
for (index <- 0 until line.length) {
- if (line.charAt(index) == '\'') {
+ if (line.charAt(index) == '\'' && !insideComment) {
// take a look to see if it is escaped
if (!escape) {
// flip the boolean variable
insideSingleQuote = !insideSingleQuote
}
- } else if (line.charAt(index) == '\"') {
+ } else if (line.charAt(index) == '\"' && !insideComment) {
// take a look to see if it is escaped
if (!escape) {
// flip the boolean variable
insideDoubleQuote = !insideDoubleQuote
}
+ } else if (line.charAt(index) == '-') {
+val hasNext: Boolean = index + 1 < line.length
+if (insideDoubleQuote || insideSingleQuote || insideComment) {
+ // ignore
Review comment:
Can you add more comments here about which case we can ignore?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
AmplabJenkins commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582279762 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
SparkQA commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582279666 **[Test build #117904 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117904/testReport)** for PR 27460 at commit [`b5c0e08`](https://github.com/apache/spark/commit/b5c0e0883f52b7ff78ca75324a6b18bdad74d390). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
AmplabJenkins commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582279773 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117904/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
SparkQA removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582270614 **[Test build #117904 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117904/testReport)** for PR 27460 at commit [`b5c0e08`](https://github.com/apache/spark/commit/b5c0e0883f52b7ff78ca75324a6b18bdad74d390). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character.
maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL
fails to parse when comment contains an unmatched quote character.
URL: https://github.com/apache/spark/pull/27321#discussion_r375097827
##
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
##
@@ -400,4 +400,21 @@ class CliSuite extends SparkFunSuite with
BeforeAndAfterAll with Logging {
-> "1.00"
)
}
+
+ test("SPARK-30049 Should not complaint for quotes in commented lines") {
+runCliWithin(1.minute)(
+ """SELECT concat('test', 'comment') -- someone's comment here
+|;""".stripMargin -> "testcomment"
+)
+ }
+
+ test("SPARK-30049 Should not complaint for quotes in commented with
multi-lines") {
+runCliWithin(1.minute)(
+ """SELECT concat('test', 'comment') -- someone's comment here \\
+|comment continues here with single ' quote
+| extra '
+|;""".stripMargin -> "testcomment"
Review comment:
Can you add a test case having spaces in the head, e.g., `|
;""".stripMargin`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
AmplabJenkins removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582279762 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
AmplabJenkins removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582279773 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117904/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources
AmplabJenkins commented on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources URL: https://github.com/apache/spark/pull/27455#issuecomment-582279450 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582279435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22669/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character.
maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL
fails to parse when comment contains an unmatched quote character.
URL: https://github.com/apache/spark/pull/27321#discussion_r375097511
##
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##
@@ -509,24 +509,40 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
private def splitSemiColon(line: String): JList[String] = {
var insideSingleQuote = false
var insideDoubleQuote = false
+var insideComment = false
var escape = false
var beginIndex = 0
+var endIndex = line.length
val ret = new JArrayList[String]
+
for (index <- 0 until line.length) {
- if (line.charAt(index) == '\'') {
+ if (line.charAt(index) == '\'' && !insideComment) {
// take a look to see if it is escaped
if (!escape) {
// flip the boolean variable
insideSingleQuote = !insideSingleQuote
}
- } else if (line.charAt(index) == '\"') {
+ } else if (line.charAt(index) == '\"' && !insideComment) {
// take a look to see if it is escaped
if (!escape) {
// flip the boolean variable
insideDoubleQuote = !insideDoubleQuote
}
+ } else if (line.charAt(index) == '-') {
+val hasNext: Boolean = index + 1 < line.length
Review comment:
nit: `val hasNext = index + 1 < line.length`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources
AmplabJenkins removed a comment on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources URL: https://github.com/apache/spark/pull/27455#issuecomment-582279450 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character.
maropu commented on a change in pull request #27321: [SPARK-30049][SQL] SQL
fails to parse when comment contains an unmatched quote character.
URL: https://github.com/apache/spark/pull/27321#discussion_r375097596
##
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
##
@@ -400,4 +400,21 @@ class CliSuite extends SparkFunSuite with
BeforeAndAfterAll with Logging {
-> "1.00"
)
}
+
+ test("SPARK-30049 Should not complaint for quotes in commented lines") {
+runCliWithin(1.minute)(
+ """SELECT concat('test', 'comment') -- someone's comment here
+|;""".stripMargin -> "testcomment"
+)
+ }
+
+ test("SPARK-30049 Should not complaint for quotes in commented with
multi-lines") {
+runCliWithin(1.minute)(
+ """SELECT concat('test', 'comment') -- someone's comment here \\
+|comment continues here with single ' quote
+| extra '
+|;""".stripMargin -> "testcomment"
+)
+ }
+
Review comment:
nit: remove this unnecessary blank.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582279435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22669/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources
AmplabJenkins removed a comment on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources URL: https://github.com/apache/spark/pull/27455#issuecomment-582279458 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22668/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources
AmplabJenkins commented on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources URL: https://github.com/apache/spark/pull/27455#issuecomment-582279458 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22668/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582279427 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582279427 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27302: [SPARK-30506][SQL][DOC] Document for generic file source options/configs
cloud-fan commented on issue #27302: [SPARK-30506][SQL][DOC] Document for generic file source options/configs URL: https://github.com/apache/spark/pull/27302#issuecomment-582279085 LGTM, can you fix the python style issue? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
SparkQA commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582279002 **[Test build #117907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117907/testReport)** for PR 27429 at commit [`484e86a`](https://github.com/apache/spark/commit/484e86a3825da89397729c6a3b6d9806a0737fc4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources
SparkQA commented on issue #27455: [SPARK-30729][CORE] Eagerly filter out zombie TaskSetManager before offering resources URL: https://github.com/apache/spark/pull/27455#issuecomment-582278981 **[Test build #117906 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117906/testReport)** for PR 27455 at commit [`857113f`](https://github.com/apache/spark/commit/857113f3be174d7dda6f393de7b72ba40b454581). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582277140 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins removed a comment on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582277145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22667/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character.
maropu commented on issue #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character. URL: https://github.com/apache/spark/pull/27321#issuecomment-582277756 yea, right. Spark doesnt support multi-line queries in `spark.sql`: (related jira: https://issues.apache.org/jira/browse/SPARK-24260) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582277145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22667/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
AmplabJenkins commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582277140 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
SparkQA commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582276819 **[Test build #117905 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117905/testReport)** for PR 27429 at commit [`8bcee2f`](https://github.com/apache/spark/commit/8bcee2fbdd4b92f5d71bc49e30004a3d1e28f3ef). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27151: [SPARK-26218][SQL][Follow up] Fix the corner case when casting float to Integer.
cloud-fan commented on a change in pull request #27151:
[SPARK-26218][SQL][Follow up] Fix the corner case when casting float to Integer.
URL: https://github.com/apache/spark/pull/27151#discussion_r375093357
##
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/float4.sql
##
@@ -106,7 +106,7 @@ SELECT smallint(float('32767.6'));
SELECT smallint(float('-32768.4'));
SELECT smallint(float('-32768.6'));
SELECT int(float('2147483520'));
-SELECT int(float('2147483647'));
+SELECT int(float('2147483392'));
Review comment:
let's NOT change the pgsql tests. They are used to verify the difference
between Spark and pgsql. We should respect the test result, whatever it is.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27465: [WIP][K8S] Flaky DepsTestsSuite Test
dongjoon-hyun closed pull request #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun opened a new pull request #27465: [WIP][K8S] Flaky DepsTestsSuite Test
dongjoon-hyun opened a new pull request #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce any user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression
beliefer commented on issue #27429: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/27429#issuecomment-582275182 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
cloud-fan commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375092341
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3551,6 +3609,12 @@ object functions {
/**
* Returns a map whose key-value pairs satisfy a predicate.
+ * {{{
+ * df.select(map_filter(col("m"), (k, v) => k * 10 === v))
+ * }}}
+ *
+ * @param expr the input map column
+ * @param f (k, v) => predicate, the Boolean predicate to filter the input
map column
Review comment:
`(key, value) => predicate`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
cloud-fan commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375092574
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3561,6 +3625,14 @@ object functions {
/**
* Merge two given maps, key-wise into a single map using a function.
+ * {{{
+ * df.select(map_zip_with(df("m1"), df("m2"), (k, v1, v2) => k === v1 +
v2))
+ * }}}
+ *
+ * @param left the left input map column
+ * @param right the right input map column
+ * @param f (k, v1, v2) => v, the lambda function to merge two values of
input map columns into
Review comment:
`(key, value1, value2) => new_value`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582274593 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117903/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
cloud-fan commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375092082
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3530,6 +3576,12 @@ object functions {
/**
* Applies a function to every key-value pair in a map and returns
* a map with the results of those applications as the new keys for the
pairs.
+ * {{{
+ * df.select(transform_keys(col("i"), (k, v) => k + v))
+ * }}}
+ *
+ * @param expr the input map column
+ * @param f col => k_transformed_col, the lambda function to transform the
key of input column
Review comment:
`(key, value) => new_key`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
cloud-fan commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375092137
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3541,6 +3593,12 @@ object functions {
/**
* Applies a function to every key-value pair in a map and returns
* a map with the results of those applications as the new values for the
pairs.
+ * {{{
+ * df.select(transform_values(col("i"), (k, v) => k + v))
+ * }}}
+ *
+ * @param expr the input map column
+ * @param f col => v_transformed_col, the lambda function to transform the
value of input column
Review comment:
ditto
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582274276 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117886/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
SparkQA removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582270622 **[Test build #117903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117903/testReport)** for PR 27461 at commit [`b8203ed`](https://github.com/apache/spark/commit/b8203ed695a1777b41d2541345efb0eae3325f73). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582274577 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582274275 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #27150: [SPARK-30471][SQL] Fix issue when comparing String and IntegerType
maropu commented on issue #27150: [SPARK-30471][SQL] Fix issue when comparing String and IntegerType URL: https://github.com/apache/spark/pull/27150#issuecomment-582274362 PgSQL has the weird casting rule for string `literals`, so I think its ok to ignore the behaivour. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582274577 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
SparkQA commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582274548 **[Test build #117903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117903/testReport)** for PR 27461 at commit [`b8203ed`](https://github.com/apache/spark/commit/b8203ed695a1777b41d2541345efb0eae3325f73). * This patch **fails to generate documentation**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins commented on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582274593 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117903/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582274276 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117886/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
cloud-fan commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375091273
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3488,6 +3511,15 @@ object functions {
* Applies a binary operator to an initial state and all elements in the
array,
* and reduces this to a single state. The final state is converted into the
final result
* by applying a finish function.
+ * {{{
+ * df.select(aggregate(col("i"), lit(0), (acc, x) => acc + x, _ * 10))
+ * }}}
+ *
+ * @param expr the input array column
+ * @param zero the initial value
+ * @param merge (c, v) => c, the merge function to reduce all elements into
a single state
Review comment:
`c`, `v` is too vague. How about `(combined_value, input_value) =>
combined_value, the merge function to merge an input value to the
combined_value`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
AmplabJenkins commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582274275 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
cloud-fan commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375091349
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3508,6 +3540,13 @@ object functions {
/**
* Applies a binary operator to an initial state and all elements in the
array,
* and reduces this to a single state.
+ * {{{
+ * df.select(aggregate(col("i"), lit(0), (acc, x) => acc + x))
+ * }}}
+ *
+ * @param expr the input array column
+ * @param zero the initial value
+ * @param merge (c, v) => c, the merge function to reduce all elements into
a single state
Review comment:
ditto
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
cloud-fan commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375091273
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3488,6 +3511,15 @@ object functions {
* Applies a binary operator to an initial state and all elements in the
array,
* and reduces this to a single state. The final state is converted into the
final result
* by applying a finish function.
+ * {{{
+ * df.select(aggregate(col("i"), lit(0), (acc, x) => acc + x, _ * 10))
+ * }}}
+ *
+ * @param expr the input array column
+ * @param zero the initial value
+ * @param merge (c, v) => c, the merge function to reduce all elements into
a single state
Review comment:
`c`, `v` is too vague. How about `(combined_value, input_value) =>
combined_value`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
SparkQA removed a comment on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582232009 **[Test build #117886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117886/testReport)** for PR 27463 at commit [`1939c3f`](https://github.com/apache/spark/commit/1939c3fe999df2f5b0a461f8c6b330f547e7e302). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem`
SparkQA commented on issue #27463: [SPARK-26700][Core][FOLLOWUP] Add config `spark.network.maxRemoteBlockSizeFetchToMem` URL: https://github.com/apache/spark/pull/27463#issuecomment-582273551 **[Test build #117886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117886/testReport)** for PR 27463 at commit [`1939c3f`](https://github.com/apache/spark/commit/1939c3fe999df2f5b0a461f8c6b330f547e7e302). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test
SparkQA removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465#issuecomment-582266523 **[Test build #117902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117902/testReport)** for PR 27465 at commit [`de0cf3e`](https://github.com/apache/spark/commit/de0cf3e8bc5bc677a2b2c2bd40cd9a14aae091d8). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test
AmplabJenkins removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465#issuecomment-582271146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117902/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test
AmplabJenkins removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465#issuecomment-582268636 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22664/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test
AmplabJenkins removed a comment on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465#issuecomment-582271142 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test
AmplabJenkins commented on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465#issuecomment-582271142 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test
AmplabJenkins commented on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465#issuecomment-582271146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117902/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582270994 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test
SparkQA commented on issue #27465: [WIP][K8S] Flaky DepsTestsSuite Test URL: https://github.com/apache/spark/pull/27465#issuecomment-582271041 **[Test build #117902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/117902/testReport)** for PR 27465 at commit [`de0cf3e`](https://github.com/apache/spark/commit/de0cf3e8bc5bc677a2b2c2bd40cd9a14aae091d8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
AmplabJenkins commented on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582270991 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22666/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest
AmplabJenkins removed a comment on issue #27461: [SPARK-30736][ML] One-Pass ChiSquareTest URL: https://github.com/apache/spark/pull/27461#issuecomment-582271003 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22665/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27449: [SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
HyukjinKwon commented on a change in pull request #27449:
[SPARK-27297][DOC][FOLLOW-UP] Improve documentation for various Scala functions
URL: https://github.com/apache/spark/pull/27449#discussion_r375088061
##
File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala
##
@@ -3431,6 +3444,12 @@ object functions {
/**
* Returns whether a predicate holds for one or more elements in the array.
+ * {{{
+ * df.select(exists(col("i"), _ % 2 === 0))
+ * }}}
+ *
+ * @param column the input array column
+ * @param f col => predicate, the Boolean predicate to check the input column
Review comment:
okie. that's fine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN
AmplabJenkins removed a comment on issue #27460: [SPARK-30733][R][HOTFIX] Fix SparkR tests per testthat and R version upgrade, and disable CRAN URL: https://github.com/apache/spark/pull/27460#issuecomment-582270982 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
