[GitHub] [spark] SparkQA commented on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
SparkQA commented on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622681309 **[Test build #122198 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122198/testReport)** for PR 28208 at commit [`295d727`](https://github.com/apache/spark/commit/295d7278324eeb5b1a356baa699b244e8de94eab). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28432: [MINOR][SQL][TESTS] Disable UI in SQL benchmarks by default
AmplabJenkins removed a comment on pull request #28432: URL: https://github.com/apache/spark/pull/28432#issuecomment-622673469 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on pull request #28431: [SPARK-31623][SQL][TESTS] Benchmark rebasing of INT96 and TIMESTAMP_MILLIS timestamps in read/write
MaxGekk commented on pull request #28431: URL: https://github.com/apache/spark/pull/28431#issuecomment-622673487 @cloud-fan @HyukjinKwon Please, take a look at this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28432: [MINOR][SQL][TESTS] Disable UI in SQL benchmarks by default
AmplabJenkins commented on pull request #28432: URL: https://github.com/apache/spark/pull/28432#issuecomment-622673469 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28432: [MINOR][SQL][TESTS] Disable UI in SQL benchmarks by default
SparkQA commented on pull request #28432: URL: https://github.com/apache/spark/pull/28432#issuecomment-622673353 **[Test build #122197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122197/testReport)** for PR 28432 at commit [`98d2b1f`](https://github.com/apache/spark/commit/98d2b1ffd4aabd292c27083a669daba3f336f928). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622672850 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622672850 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
SparkQA commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622672737 **[Test build #122196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122196/testReport)** for PR 28435 at commit [`babab25`](https://github.com/apache/spark/commit/babab25ec0deec09d4e97be74f9fa75890496292). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
AmplabJenkins removed a comment on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622671028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122195/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
AmplabJenkins removed a comment on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622671024 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
SparkQA removed a comment on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622670802 **[Test build #122195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122195/testReport)** for PR 28208 at commit [`b0a9149`](https://github.com/apache/spark/commit/b0a9149680addf6630af6ee2e99cfb0036b3d7b0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
AmplabJenkins removed a comment on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622670833 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122185/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
SparkQA commented on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622671019 **[Test build #122195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122195/testReport)** for PR 28208 at commit [`b0a9149`](https://github.com/apache/spark/commit/b0a9149680addf6630af6ee2e99cfb0036b3d7b0). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class SparkPlanGraphEdge(fromId: Long, toId: Long) ` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
AmplabJenkins commented on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622671024 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
AmplabJenkins removed a comment on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622670831 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
AmplabJenkins commented on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622670831 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
SparkQA commented on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622670802 **[Test build #122195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122195/testReport)** for PR 28208 at commit [`b0a9149`](https://github.com/apache/spark/commit/b0a9149680addf6630af6ee2e99cfb0036b3d7b0). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
SparkQA removed a comment on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622649684 **[Test build #122185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122185/testReport)** for PR 28434 at commit [`d32a6c4`](https://github.com/apache/spark/commit/d32a6c4fd03a6aa1f7e27f7dd9f60f7da2b4583a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
SparkQA commented on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622670731 **[Test build #122185 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122185/testReport)** for PR 28434 at commit [`d32a6c4`](https://github.com/apache/spark/commit/d32a6c4fd03a6aa1f7e27f7dd9f60f7da2b4583a). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
AmplabJenkins commented on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622670252 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
AmplabJenkins removed a comment on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622670252 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622669326 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622669326 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
SparkQA removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622667725 **[Test build #122192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122192/testReport)** for PR 28435 at commit [`d1d5323`](https://github.com/apache/spark/commit/d1d53239fb24c33ecf23822968968f138f097dcc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
AmplabJenkins removed a comment on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622669211 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
SparkQA commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622669291 **[Test build #122192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122192/testReport)** for PR 28435 at commit [`d1d5323`](https://github.com/apache/spark/commit/d1d53239fb24c33ecf23822968968f138f097dcc). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
AmplabJenkins commented on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622669211 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
SparkQA removed a comment on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622632764 **[Test build #122184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122184/testReport)** for PR 28208 at commit [`817ebab`](https://github.com/apache/spark/commit/817ebab256f819bc157ff36c2f25909625d4b9ec). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #27983: [SPARK-15694][SQL]Implement ScriptTransformation in sql/core
AmplabJenkins removed a comment on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-622669015 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28208: [SPARK-31440][SQL] Improve SQL Rest API
SparkQA commented on pull request #28208: URL: https://github.com/apache/spark/pull/28208#issuecomment-622669020 **[Test build #122184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122184/testReport)** for PR 28208 at commit [`817ebab`](https://github.com/apache/spark/commit/817ebab256f819bc157ff36c2f25909625d4b9ec). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #27983: [SPARK-15694][SQL]Implement ScriptTransformation in sql/core
AmplabJenkins commented on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-622669015 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #27983: [SPARK-15694][SQL]Implement ScriptTransformation in sql/core
SparkQA commented on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-622668933 **[Test build #122194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122194/testReport)** for PR 27983 at commit [`6fcb1af`](https://github.com/apache/spark/commit/6fcb1af04f3ea3d2a07fea253670a8d24c509869). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins removed a comment on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622668665 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622668665 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
SparkQA removed a comment on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622667726 **[Test build #122193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122193/testReport)** for PR 28433 at commit [`15da96c`](https://github.com/apache/spark/commit/15da96c4db4acb858e6a94e1d707846dbca291a3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
SparkQA commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622668635 **[Test build #122193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122193/testReport)** for PR 28433 at commit [`15da96c`](https://github.com/apache/spark/commit/15da96c4db4acb858e6a94e1d707846dbca291a3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins removed a comment on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622667842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622667842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622667834 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622667834 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
SparkQA commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622667726 **[Test build #122193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122193/testReport)** for PR 28433 at commit [`15da96c`](https://github.com/apache/spark/commit/15da96c4db4acb858e6a94e1d707846dbca291a3). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28435: [WIP][SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
SparkQA commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622667725 **[Test build #122192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122192/testReport)** for PR 28435 at commit [`d1d5323`](https://github.com/apache/spark/commit/d1d53239fb24c33ecf23822968968f138f097dcc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
dilipbiswal commented on a change in pull request #28433:
URL: https://github.com/apache/spark/pull/28433#discussion_r418887456
##
File path: docs/sql-ref-datatypes.md
##
@@ -824,4 +312,4 @@ SELECT COUNT(*), c2 FROM test GROUP BY c2;
|2|-Infinity|
|3| Infinity|
+-+-+
-{% endhighlight %}
\ No newline at end of file
+{% endhighlight %}
Review comment:
@maropu Fixed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
dilipbiswal commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622667364 @maropu sorry, i missed sql-ref-null-semantics.md. I have added it now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
dilipbiswal commented on a change in pull request #28433: URL: https://github.com/apache/spark/pull/28433#discussion_r418887140 ## File path: docs/sql-ref-datetime-pattern.md ## @@ -58,8 +58,8 @@ Spark uses pattern letters in the following table for date and timestamp parsing |**Z**|zone-offset|offset-Z|+; -0800; -08:00;| |**'**|escape for text|delimiter|| |**''**|single quote|literal|'| -|**[**|optional section start||| -|**]**|optional section end||| +|**[**|optional section start| | | Review comment: @maropu @gatorsmile OK. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
AmplabJenkins removed a comment on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-622666121 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
AmplabJenkins commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-622666121 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
SparkQA commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-622666026 **[Test build #122191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122191/testReport)** for PR 28425 at commit [`02f230b`](https://github.com/apache/spark/commit/02f230b4747ce396a5bfe75d6687c6b314a30150). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
dilipbiswal commented on a change in pull request #28425:
URL: https://github.com/apache/spark/pull/28425#discussion_r418880068
##
File path:
external/avro/src/main/scala/org/apache/spark/sql/v2/avro/AvroScan.scala
##
@@ -65,4 +65,8 @@ case class AvroScan(
}
override def hashCode(): Int = super.hashCode()
+
+ override def getMetaData(): Map[String, String] = {
+super.metaData ++ Map("Format" -> "avro")
Review comment:
@maropu OK.. added a test.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
gatorsmile commented on a change in pull request #28433: URL: https://github.com/apache/spark/pull/28433#discussion_r418870517 ## File path: docs/sql-ref-datetime-pattern.md ## @@ -58,8 +58,8 @@ Spark uses pattern letters in the following table for date and timestamp parsing |**Z**|zone-offset|offset-Z|+; -0800; -08:00;| |**'**|escape for text|delimiter|| |**''**|single quote|literal|'| -|**[**|optional section start||| -|**]**|optional section end||| +|**[**|optional section start| | | Review comment: it would be nicer if we have an extra space between | and | Yes. Line 59 is needed too This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should filter out all the unevaluable expr
AmplabJenkins removed a comment on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-622663517 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should filter out all the unevaluable expr
AmplabJenkins commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-622663517 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should filter out all the unevaluable expr
SparkQA commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-622663422 **[Test build #122190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122190/testReport)** for PR 28383 at commit [`a7638d6`](https://github.com/apache/spark/commit/a7638d602fca15911c99fd14703f144902f68f47). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] cxzl25 commented on a change in pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should filter out all the unevaluable expr
cxzl25 commented on a change in pull request #28383:
URL: https://github.com/apache/spark/pull/28383#discussion_r418869108
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuerySuite.scala
##
@@ -103,6 +103,20 @@ class OptimizeMetadataOnlyQuerySuite extends QueryTest
with SharedSparkSession {
"select partcol2, min(partcol1) from srcpart where partcol1 = 0 group by
partcol2",
"select max(c1) from (select partcol1 + 1 as c1 from srcpart where
partcol1 = 0) t")
+ testMetadataOnly(
+"SPARK-31590 The filter used by Metadata-only queries should not have
Unevaluable",
+"""
+ |SELECT partcol1, MAX(partcol2) AS partcol2
+ |FROM srcpart
+ |WHERE partcol1 = (
+ | SELECT MAX(partcol1)
+ | FROM srcpart
+ |)
+ |AND partcol2= 'event'
+ |GROUP BY partcol1
+ |""".stripMargin
+ )
Review comment:
Sorry, I typed an extra letter t, it should be the existing partition
value to filter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622662391 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122188/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622662388 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
SparkQA removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622659533 **[Test build #122188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122188/testReport)** for PR 28435 at commit [`bf11d47`](https://github.com/apache/spark/commit/bf11d47ee59a9a20f797e32855fa304caa80b373). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
SparkQA commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622662383 **[Test build #122188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122188/testReport)** for PR 28435 at commit [`bf11d47`](https://github.com/apache/spark/commit/bf11d47ee59a9a20f797e32855fa304caa80b373). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622662388 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622661909 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622661909 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should filter out all the unevaluable expr
AmplabJenkins removed a comment on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-622661681 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622631568 **[Test build #122183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122183/testReport)** for PR 28331 at commit [`be2a5e7`](https://github.com/apache/spark/commit/be2a5e736e051ca0497906b2a2e904c7b4033596). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should filter out all the unevaluable expr
AmplabJenkins commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-622661681 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622661695 **[Test build #122183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122183/testReport)** for PR 28331 at commit [`be2a5e7`](https://github.com/apache/spark/commit/be2a5e736e051ca0497906b2a2e904c7b4033596). * This patch passes all tests. * This patch **does not merge cleanly**. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should filter out all the unevaluable expr
SparkQA commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-622661598 **[Test build #122189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122189/testReport)** for PR 28383 at commit [`7046db8`](https://github.com/apache/spark/commit/7046db81d1a830c97a20f8e690cfb507c712da82). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should not have Unevaluable
HyukjinKwon commented on a change in pull request #28383: URL: https://github.com/apache/spark/pull/28383#discussion_r418857861 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/OptimizeMetadataOnlyQuery.scala ## @@ -117,7 +117,7 @@ case class OptimizeMetadataOnlyQuery(catalog: SessionCatalog) extends Rule[Logic case a: AttributeReference => a.withName(relation.output.find(_.semanticEquals(a)).get.name) } -} +}.filterNot(SubqueryExpression.hasSubquery) Review comment: Yeah let's keep the PR title.and description matched .. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #28383: [SPARK-31590][SQL] The filter used by Metadata-only queries should not have Unevaluable
HyukjinKwon commented on pull request #28383: URL: https://github.com/apache/spark/pull/28383#issuecomment-622660745 @cxzl25 can you revert the test back to the original one and focus on the cleanup? The case before was a valid, and failed in the master. The fix itself seems right too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622659650 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
AmplabJenkins removed a comment on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622659650 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
SparkQA commented on pull request #28435: URL: https://github.com/apache/spark/pull/28435#issuecomment-622659533 **[Test build #122188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122188/testReport)** for PR 28435 at commit [`bf11d47`](https://github.com/apache/spark/commit/bf11d47ee59a9a20f797e32855fa304caa80b373). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 opened a new pull request #28435: [SPARK-31625][YARN] Unregister application from YARN RM outside the shutdown hook if it succeeds
imback82 opened a new pull request #28435: URL: https://github.com/apache/spark/pull/28435 ### What changes were proposed in this pull request? Currently, an application is unregistered from YARN resource manager as a shutdown hook. In the scenario where the shutdown hook does not run (e.g., timeouts, etc.), the application is not unregistered, resulting in YARN resubmitting the application even if it succeeded. For example, you could see the following on the driver log: ``` 20/04/30 06:20:29 INFO SparkContext: Successfully stopped SparkContext 20/04/30 06:20:29 INFO ApplicationMaster: Final app status: SUCCEEDED, exitCode: 0 20/04/30 06:20:59 WARN ShutdownHookManager: ShutdownHook '$anon$2' timeout, java.util.concurrent.TimeoutException java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:205) at org.apache.hadoop.util.ShutdownHookManager.executeShutdown(ShutdownHookManager.java:124) at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:95) ``` You can see that the final app status is `SUCCEEDED`. However, on the YARN RM side: ``` 020-04-30 06:21:25,083 INFO RMContainerImpl: container_1588227360159_0001_01_01 Container Transitioned from RUNNING to COMPLETED 2020-04-30 06:21:25,085 INFO RMAppAttemptImpl: Updating application attempt appattempt_1588227360159_0001_01 with final state: FAILED, and exit status: 0 2020-04-30 06:21:25,085 INFO RMAppAttemptImpl: appattempt_1588227360159_0001_01 State change from RUNNING to FINAL_SAVING on event = CONTAINER_FINISHED ``` You see that the final state of the application becomes `FAILED` since the container is finished before the application is unregistered. This could be misleading and this PR proposes to unregister the application right away if it succeeds. ### Why are the changes needed? To fix a bug where application is not unregistered even if it succeeds. ### Does this PR introduce _any_ user-facing change? Yes. If the application succeeds, it will be unregistered even if the shutdown hook doesn't get run. The following is YARN RM log when there was a timeout on the shutdown hook after the fix: ``` 2020-05-02 02:13:02,222 INFO RMContainerImpl: container_1588385278256_0001_01_01 Container Transitioned from RUNNING to COMPLETED 2020-05-02 02:13:02,222 INFO ApplicationMasterService: Unregistering app attempt : appattempt_1588385278256_0001_01 2020-05-02 02:13:02,224 INFO RMAppAttemptImpl: appattempt_1588385278256_0001_01 State change from FINISHING to FINISHED on event = CONTAINER_FINISHED 2020-05-02 02:13:02,226 INFO RMAppImpl: application_1588385278256_0001 State change from FINISHING to FINISHED on event = ATTEMPT_FINISHED ``` ### How was this patch tested? Since this behavior is checking against the scenario where shutdown hook is involved, no test has been added, but manually validated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
AmplabJenkins removed a comment on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-622657444 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
AmplabJenkins commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-622657444 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
SparkQA commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-622657325 **[Test build #122187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122187/testReport)** for PR 28425 at commit [`3d6040a`](https://github.com/apache/spark/commit/3d6040a4e48850278714b91b2db4b27c0410994d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
maropu commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622655461 > Replace HTML table by MD table All the fixes in `docs/sql-ref-*` addressed in this PR? I found the tag in `sql-ref-null-semantics.md`, too; ``` $ls sql-ref* | xargs grep " sql-ref-ansi-compliance.md: sql-ref-datatypes.md: sql-ref-datatypes.md: sql-ref-datatypes.md: sql-ref-datatypes.md: sql-ref-datatypes.md: sql-ref-datetime-pattern.md: sql-ref-null-semantics.md: sql-ref-null-semantics.md: sql-ref-null-semantics.md: sql-ref-null-semantics.md: sql-ref-syntax-qry-select-tvf.md: ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
maropu commented on a change in pull request #28433: URL: https://github.com/apache/spark/pull/28433#discussion_r418834997 ## File path: docs/sql-ref-datetime-pattern.md ## @@ -58,8 +58,8 @@ Spark uses pattern letters in the following table for date and timestamp parsing |**Z**|zone-offset|offset-Z|+; -0800; -08:00;| |**'**|escape for text|delimiter|| |**''**|single quote|literal|'| -|**[**|optional section start||| -|**]**|optional section end||| +|**[**|optional section start| | | Review comment: Need the space between `|`s? But, no space between `|`s in line 59? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
maropu commented on a change in pull request #28433:
URL: https://github.com/apache/spark/pull/28433#discussion_r418834443
##
File path: docs/sql-ref-datatypes.md
##
@@ -824,4 +312,4 @@ SELECT COUNT(*), c2 FROM test GROUP BY c2;
|2|-Infinity|
|3| Infinity|
+-+-+
-{% endhighlight %}
\ No newline at end of file
+{% endhighlight %}
Review comment:
What's a difference here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
maropu commented on a change in pull request #28425:
URL: https://github.com/apache/spark/pull/28425#discussion_r418829916
##
File path:
external/avro/src/main/scala/org/apache/spark/sql/v2/avro/AvroScan.scala
##
@@ -65,4 +65,8 @@ case class AvroScan(
}
override def hashCode(): Int = super.hashCode()
+
+ override def getMetaData(): Map[String, String] = {
+super.metaData ++ Map("Format" -> "avro")
Review comment:
Yea if we don't have any other suitable place for adding the test. At
least, I think its better to add tests for it somewhere.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
maropu commented on a change in pull request #28425:
URL: https://github.com/apache/spark/pull/28425#discussion_r418829024
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/connector/SupportsMetadata.scala
##
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.internal.connector
+
+/**
+ * A mix in interface for {@link FileScan}. This can be used to report metadata
+ * for a file based scan operator. This is currently used for supporting
formatted
+ * explain.
+ */
Review comment:
Not sure, but if we expose this, developers could improve explain output
for their custom scan? cc: @cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
SparkQA removed a comment on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622649687 **[Test build #122186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122186/testReport)** for PR 28433 at commit [`c419b93`](https://github.com/apache/spark/commit/c419b931b96e00f6787137032fde8b211fb39def). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins removed a comment on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622651089 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622651089 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
SparkQA commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622651055 **[Test build #122186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122186/testReport)** for PR 28433 at commit [`c419b93`](https://github.com/apache/spark/commit/c419b931b96e00f6787137032fde8b211fb39def). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node
dilipbiswal commented on a change in pull request #28425:
URL: https://github.com/apache/spark/pull/28425#discussion_r418820695
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/connector/SupportsMetadata.scala
##
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.spark.sql.internal.connector
+
+/**
+ * A mix in interface for {@link FileScan}. This can be used to report metadata
+ * for a file based scan operator. This is currently used for supporting
formatted
+ * explain.
+ */
Review comment:
@maropu On second thought, this is not an external interface, right ? So
don't think we need any annotations here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622650027 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122181/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622650024 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
AmplabJenkins removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622650024 Build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
AmplabJenkins removed a comment on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622649815 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA removed a comment on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622607854 **[Test build #122181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122181/testReport)** for PR 28331 at commit [`adb03db`](https://github.com/apache/spark/commit/adb03dbf44f10e4e254e8a5c3957a058a0dd8de7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins removed a comment on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622649821 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28331: [WIP][SPARK-20629][CORE] Copy shuffle data when nodes are being shutdown
SparkQA commented on pull request #28331: URL: https://github.com/apache/spark/pull/28331#issuecomment-622649891 **[Test build #122181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122181/testReport)** for PR 28331 at commit [`adb03db`](https://github.com/apache/spark/commit/adb03dbf44f10e4e254e8a5c3957a058a0dd8de7). * This patch **fails Spark unit tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
AmplabJenkins commented on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622649815 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
AmplabJenkins commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622649821 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
SparkQA commented on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622649684 **[Test build #122185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122185/testReport)** for PR 28434 at commit [`d32a6c4`](https://github.com/apache/spark/commit/d32a6c4fd03a6aa1f7e27f7dd9f60f7da2b4583a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
SparkQA commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622649687 **[Test build #122186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122186/testReport)** for PR 28433 at commit [`c419b93`](https://github.com/apache/spark/commit/c419b931b96e00f6787137032fde8b211fb39def). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on a change in pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
dilipbiswal commented on a change in pull request #28433: URL: https://github.com/apache/spark/pull/28433#discussion_r418816535 ## File path: docs/sql-ref-ansi-compliance.md ## @@ -27,35 +27,10 @@ The casting behaviours are defined as store assignment rules in the standard. When `spark.sql.storeAssignmentPolicy` is set to `ANSI`, Spark SQL complies with the ANSI store assignment rules. This is a separate configuration because its default value is `ANSI`, while the configuration `spark.sql.ansi.enabled` is disabled by default. - -Property NameDefaultMeaningSince Version - - spark.sql.ansi.enabled - false - -(Experimental) When true, Spark tries to conform to the ANSI SQL specification: -1. Spark will throw a runtime exception if an overflow occurs in any operation on integral/decimal field. -2. Spark will forbid using the reserved keywords of ANSI SQL as identifiers in the SQL parser. - - 3.0.0 - - - spark.sql.storeAssignmentPolicy - ANSI - -(Experimental) When inserting a value into a column with different data type, Spark will perform type coercion. -Currently, we support 3 policies for the type coercion rules: ANSI, legacy and strict. With ANSI policy, -Spark performs the type coercion as per ANSI SQL. In practice, the behavior is mostly the same as PostgreSQL. -It disallows certain unreasonable type conversions such as converting string to int or double to boolean. -With legacy policy, Spark allows the type coercion as long as it is a valid Cast, which is very loose. -e.g. converting string to int or double to boolean is allowed. -It is also the only behavior in Spark 2.x and it is compatible with Hive. -With strict policy, Spark doesn't allow any possible precision loss or data truncation in type coercion, -e.g. converting double to int or decimal to double is not allowed. - - 3.0.0 - - +|Property Name|Default|Meaning|Since Version| +|--- |--- |--- |--- | Review comment: @huaxingao Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dilipbiswal commented on pull request #28433: [SPARK-31030] [DOCS] [FOLLOWUP] Replace HTML Table by Markdown Table
dilipbiswal commented on pull request #28433: URL: https://github.com/apache/spark/pull/28433#issuecomment-622649549 cc @maropu This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] brkyvz opened a new pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
brkyvz opened a new pull request #28434: URL: https://github.com/apache/spark/pull/28434 ## What changes were proposed in this pull request? SHOW TBLPROPERTIES does not get the correct table properties for tables using the Session Catalog. This PR fixes that, by explicitly falling back to the V1 implementation if the table is in fact a V1 table. ### Why are the changes needed? Shows the incorrect table properties, i.e. only what exists in the Hive MetaStore for V2 tables that may have table properties outside of the MetaStore. ### Does this PR introduce _any_ user-facing change? Fixes a bug ### How was this patch tested? Regression test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] brkyvz commented on pull request #28434: [SPARK-31624] Fix SHOW TBLPROPERTIES for V2 tables that leverage the session catalog
brkyvz commented on pull request #28434: URL: https://github.com/apache/spark/pull/28434#issuecomment-622649369 It would be great to merge this into Spark 3.0. cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
