[GitHub] [spark] venkata91 commented on a change in pull request #26339: [SPARK-27194][SPARK-29302][SQL] Fix the issue that for dynamic partition overwrite a task would conflict with its speculative t
venkata91 commented on a change in pull request #26339:
[SPARK-27194][SPARK-29302][SQL] Fix the issue that for dynamic partition
overwrite a task would conflict with its speculative task
URL: https://github.com/apache/spark/pull/26339#discussion_r410825389
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/sources/PartitionedWriteSuite.scala
##
@@ -157,3 +161,49 @@ class PartitionedWriteSuite extends QueryTest with
SharedSparkSession {
}
}
}
+
+private class DetectDynamicSpeculationCommitProtocol(
+jobId: String,
+path: String,
+dynamicPartitionOverwrite: Boolean)
+ extends HadoopMapReduceCommitProtocol(jobId, path,
dynamicPartitionOverwrite) {
+
+ override def commitTask(taskContext: TaskAttemptContext): TaskCommitMessage
= {
+if (dynamicPartitionOverwrite) {
+ val partitionPathSet = dynamicStagingTaskFiles
+.map(taskFile => getDynamicPartitionPath(taskFile, taskContext))
+.map(_.toUri.getPath.stripPrefix(stagingDir.toUri.getPath +
+ Path.SEPARATOR))
+ assert(partitionPathSet.equals(partitionPaths))
+}
+super.commitTask(taskContext)
+ }
+}
+
+class PartitionedSpeculateWriteSuite extends QueryTest with SharedSparkSession
{
+ import testImplicits._
+
+ override def sparkConf(): SparkConf = {
+super.sparkConf
+ .set(config.SPECULATION_MULTIPLIER, 0.0)
+ .set(config.SPECULATION_QUANTILE, 0.5)
+ .set(config.SPECULATION_ENABLED, true)
+ }
+
+ test("SPARK-27194 SPARK-29302: Fix the issue that for dynamic partition
overwrite, a " +
+"task would conflict with its speculative task") {
+withSQLConf(SQLConf.PARTITION_OVERWRITE_MODE.key ->
PartitionOverwriteMode.DYNAMIC.toString,
+ SQLConf.FILE_COMMIT_PROTOCOL_CLASS.key ->
+classOf[DetectDynamicSpeculationCommitProtocol].getName) {
+ withTable("t") {
Review comment:
I was testing it locally, without the fix this test still passes. Can you
please check once?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on issue #28197: [SPARK-31431][SQL] Add CalendarInterval encoder support
yaooqinn commented on issue #28197: [SPARK-31431][SQL] Add CalendarInterval encoder support URL: https://github.com/apache/spark/pull/28197#issuecomment-616039148 It will be public after 3.0.0 release @viirya This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616037462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26155/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
viirya commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616037411 also cc @cloud-fan @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616037456 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616037456 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616037462 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26155/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
SparkQA commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616037142 **[Test build #121471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121471/testReport)** for PR 28255 at commit [`f402180`](https://github.com/apache/spark/commit/f402180974408bf3a22e80081eeed57793ad4830). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616033897 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616033907 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121465/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616033897 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616033907 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121465/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
SparkQA removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616009863 **[Test build #121465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121465/testReport)** for PR 28255 at commit [`da210ea`](https://github.com/apache/spark/commit/da210ea809c3e37e51ef1cc1bafe5e7f9fe2c2ca). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
SparkQA commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616033385 **[Test build #121465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121465/testReport)** for PR 28255 at commit [`da210ea`](https://github.com/apache/spark/commit/da210ea809c3e37e51ef1cc1bafe5e7f9fe2c2ca). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616032050 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616032052 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121466/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616032052 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121466/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616032050 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616031847 **[Test build #121466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121466/testReport)** for PR 28239 at commit [`d699f58`](https://github.com/apache/spark/commit/d699f58b4eca6f8fe7ae1fb9b2ff0e92e0e4f013). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
SparkQA removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616011120 **[Test build #121466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121466/testReport)** for PR 28239 at commit [`d699f58`](https://github.com/apache/spark/commit/d699f58b4eca6f8fe7ae1fb9b2ff0e92e0e4f013). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei edited a comment on issue #26339: [SPARK-27194][SPARK-29302][SQL] Fix the issue that for dynamic partition overwrite a task would conflict with its speculative task
turboFei edited a comment on issue #26339: [SPARK-27194][SPARK-29302][SQL] Fix the issue that for dynamic partition overwrite a task would conflict with its speculative task URL: https://github.com/apache/spark/pull/26339#issuecomment-613445024 Revert to origin code. It seems that let all tasks, which have same stageId and taskId, to rename their dynamic staging task files is the best plan. Otherwise, it is too complex, and may cause data loss. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616029885 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616029887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26154/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616029887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26154/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616029885 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616029708 **[Test build #121470 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121470/testReport)** for PR 28239 at commit [`bb52763`](https://github.com/apache/spark/commit/bb52763b6e3ed55285e25811bf23b92b4d8c4394). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
turboFei commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616029679 I have used `grep` command to find all tests, which does clean up their used views, and refactor them. cc @maropu This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616028195 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121469/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616028195 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121469/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616028189 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616028189 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
SparkQA commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616028087 **[Test build #121469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121469/testReport)** for PR 28237 at commit [`0121faf`](https://github.com/apache/spark/commit/0121faf90018a7b15bcc303298576d442412ef1a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
SparkQA removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616025906 **[Test build #121469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121469/testReport)** for PR 28237 at commit [`0121faf`](https://github.com/apache/spark/commit/0121faf90018a7b15bcc303298576d442412ef1a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
SparkQA commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616025906 **[Test build #121469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121469/testReport)** for PR 28237 at commit [`0121faf`](https://github.com/apache/spark/commit/0121faf90018a7b15bcc303298576d442412ef1a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616024663 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins commented on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616024667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26153/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616024667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26153/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
AmplabJenkins removed a comment on issue #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#issuecomment-616024663 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
huaxingao commented on a change in pull request #28237:
[SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
URL: https://github.com/apache/spark/pull/28237#discussion_r410809649
##
File path: docs/sql-ref-literals.md
##
@@ -0,0 +1,385 @@
+---
+layout: global
+title: Literals
+displayTitle: Literals
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+A literal (also known as a constant) represents a fixed data value. Spark SQL
supports the following literals:
+
+ * [String Literal](#string-literal)
+ * [Null Literal](#null-literal)
+ * [Boolean Literal](#boolean-literal)
+ * [Numeric Literal](#numeric-literal)
+ * [Datetime Literal](#datetime-literal)
+ * [Interval Literal](#interval-literal)
+
+### String Literal
+
+A string literal is used to specify a character string value.
+
+ Syntax
+
+{% highlight sql %}
+'...' | "..."
+{% endhighlight %}
+...: indicates any string.
+
+ Examples
+
+{% highlight sql %}
+SELECT 'Hello, World!' AS col;
+ +-+
+ | col|
+ +-+
+ |Hello, World!|
+ +-+
+
+SELECT "SPARK SQL" AS col;
+ +-+
+ | col|
+ +-+
+ |Spark SQL|
+ +-+
+
+SELECT SELECT 'it\'s $10.' AS col;
+ +-+
+ | col|
+ +-+
+ |It's $10.|
+ +--+
+{% endhighlight %}
+
+### Null Literal
+
+A null literal is used to specify a null value.
+
+ Syntax
+
+{% highlight sql %}
+NULL
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT NULL AS col;
+ ++
+ | col|
+ ++
+ |NULL|
+ ++
+{% endhighlight %}
+
+### Boolean Literal
+
+A boolean literal is used to specify a boolean value.
+
+ Syntax
+
+{% highlight sql %}
+TRUE | FALSE
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT TRUE AS col;
+ ++
+ | col|
+ ++
+ |true|
+ ++
+{% endhighlight %}
+
+### Numeric Literal
+
+A numeric literal is used to specify a fixed or floating-point number.
+
+ Integer Literal
+
+ Syntax
+
+{% highlight sql %}
+[ + | - ] digit [ ... ] [ L | S | Y ]
+{% endhighlight %}
+digit: one of 0, 1, 2, 3, 4, 5, 6, 7, 8 or 9.
+default (no postfix): indicates a 4-byte signed integer number.
+L: case insensitive, indicates BIGINT, which is a 8-byte signed
integer number.
+S: case insensitive, indicates SMALLINT, which is a 2-byte signed
integer number.
+Y: case insensitive, indicates TINYINT, which is a 1-byte signed
integer number.
+ Examples
+
+{% highlight sql %}
+SELECT -2147483648 AS col;
+ +---+
+ |col|
+ +---+
+ |-2147483648|
+ +---+
+
+SELECT 9223372036854775807l AS col;
+ +---+
+ |col|
+ +---+
+ |9223372036854775807|
+ +---+
+
+SELECT -32Y AS col;
+ +---+
+ |col|
+ +---+
+ |-32|
+ +---+
+
+SELECT 482S AS col;
+ +---+
+ |col|
+ +---+
+ |482|
+ +---+
+{% endhighlight %}
+
+ Decimal Literal
+
+ Syntax
+
+{% highlight sql %}
+ [ + | - ] { digit [ ... ] . [ digit [ ... ] ] | . digit [ ... ] }
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT 12.578 AS col;
+ +--+
+ | col|
+ +--+
+ |12.578|
+ +--+
+
+SELECT -0.1234567 AS col;
+ +--+
+ | col|
+ +--+
+ |-0.1234567|
+ +--+
+
+SELECT -.1234567 AS col;
+ +--+
+ | col|
+ +--+
+ |-0.1234567|
+ +--+
+{% endhighlight %}
+
+ Floating Point and BigDecimal Literals
+
+ Syntax
+
+{% highlight sql %}
+ [ + | - ] { digit [ ... ] [ E [ + | - ] digit [ ... ] ] [ D | BD ]
+ | digit [ ... ] . [ digit [ ... ] ] [ E [ + | - ] digit [ ... ] ]
[ D | BD ]
+ | . digit [ ... ] [ E [ + | - ] digit [ ... ] ] [ D | BD ] }
+{% endhighlight %}
+digit: one of 0, 1, 2, 3, 4, 5, 6, 7, 8 or 9.
+default (no postfix): indicate a 4-byte single-precision floating point
number.
+D: case insensitive, indicates DOUBLE, which is a 8-byte
double-precision floating point number.
+BD: case insensitive, indicates BIGDECIMAL, which is an
arbitrary-precision signed decimal number.
Review comment:
I think strictly speaking, infinity, nan are not constants, because
```SELECT infinity``` doesn't work,
[GitHub] [spark] huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
huaxingao commented on a change in pull request #28237:
[SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
URL: https://github.com/apache/spark/pull/28237#discussion_r410809668
##
File path: docs/sql-ref-literals.md
##
@@ -0,0 +1,385 @@
+---
+layout: global
+title: Literals
+displayTitle: Literals
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+A literal (also known as a constant) represents a fixed data value. Spark SQL
supports the following literals:
+
+ * [String Literal](#string-literal)
+ * [Null Literal](#null-literal)
+ * [Boolean Literal](#boolean-literal)
+ * [Numeric Literal](#numeric-literal)
+ * [Datetime Literal](#datetime-literal)
+ * [Interval Literal](#interval-literal)
+
+### String Literal
+
+A string literal is used to specify a character string value.
+
+ Syntax
+
+{% highlight sql %}
+'...' | "..."
+{% endhighlight %}
+...: indicates any string.
+
+ Examples
+
+{% highlight sql %}
+SELECT 'Hello, World!' AS col;
+ +-+
+ | col|
+ +-+
+ |Hello, World!|
+ +-+
+
+SELECT "SPARK SQL" AS col;
+ +-+
+ | col|
+ +-+
+ |Spark SQL|
+ +-+
+
+SELECT SELECT 'it\'s $10.' AS col;
+ +-+
+ | col|
+ +-+
+ |It's $10.|
+ +--+
+{% endhighlight %}
+
+### Null Literal
+
+A null literal is used to specify a null value.
+
+ Syntax
+
+{% highlight sql %}
+NULL
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT NULL AS col;
+ ++
+ | col|
+ ++
+ |NULL|
+ ++
+{% endhighlight %}
+
+### Boolean Literal
+
+A boolean literal is used to specify a boolean value.
+
+ Syntax
+
+{% highlight sql %}
+TRUE | FALSE
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT TRUE AS col;
+ ++
+ | col|
+ ++
+ |true|
+ ++
+{% endhighlight %}
+
+### Numeric Literal
+
+A numeric literal is used to specify a fixed or floating-point number.
+
+ Integer Literal
+
+ Syntax
+
+{% highlight sql %}
+[ + | - ] digit [ ... ] [ L | S | Y ]
+{% endhighlight %}
+digit: one of 0, 1, 2, 3, 4, 5, 6, 7, 8 or 9.
+default (no postfix): indicates a 4-byte signed integer number.
+L: case insensitive, indicates BIGINT, which is a 8-byte signed
integer number.
+S: case insensitive, indicates SMALLINT, which is a 2-byte signed
integer number.
+Y: case insensitive, indicates TINYINT, which is a 1-byte signed
integer number.
+ Examples
+
+{% highlight sql %}
+SELECT -2147483648 AS col;
+ +---+
+ |col|
+ +---+
+ |-2147483648|
+ +---+
+
+SELECT 9223372036854775807l AS col;
+ +---+
+ |col|
+ +---+
+ |9223372036854775807|
+ +---+
+
+SELECT -32Y AS col;
+ +---+
+ |col|
+ +---+
+ |-32|
+ +---+
+
+SELECT 482S AS col;
+ +---+
+ |col|
+ +---+
+ |482|
+ +---+
+{% endhighlight %}
+
+ Decimal Literal
+
+ Syntax
+
+{% highlight sql %}
+ [ + | - ] { digit [ ... ] . [ digit [ ... ] ] | . digit [ ... ] }
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT 12.578 AS col;
+ +--+
+ | col|
+ +--+
+ |12.578|
+ +--+
+
+SELECT -0.1234567 AS col;
+ +--+
+ | col|
+ +--+
+ |-0.1234567|
+ +--+
+
+SELECT -.1234567 AS col;
+ +--+
+ | col|
+ +--+
+ |-0.1234567|
+ +--+
+{% endhighlight %}
+
+ Floating Point and BigDecimal Literals
+
+ Syntax
+
+{% highlight sql %}
+ [ + | - ] { digit [ ... ] [ E [ + | - ] digit [ ... ] ] [ D | BD ]
+ | digit [ ... ] . [ digit [ ... ] ] [ E [ + | - ] digit [ ... ] ]
[ D | BD ]
Review comment:
I guess it might be better to align the options?
```
[ + | - ] { digit [ ... ] [ E [ + | - ] digit [ ... ] ] [ D | BD ] |
digit [ ... ] . [ digit [ ... ] ] [ E [ + | - ] digit [ ... ] ]
[ D | BD ] |
. digit [ ... ] [ E [ + | - ] digit [ ... ] ] [ D | BD ] }
```
```
DATE { '' |
'-[m]m' |
'-[m]m-[d]d' |
'-[m]m-[d]d ' |
'-[m]m-[d]d[T]c[...]' }
```
[GitHub] [spark] huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
huaxingao commented on a change in pull request #28237:
[SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
URL: https://github.com/apache/spark/pull/28237#discussion_r410809664
##
File path: docs/sql-ref-literals.md
##
@@ -0,0 +1,385 @@
+---
+layout: global
+title: Literals
+displayTitle: Literals
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+A literal (also known as a constant) represents a fixed data value. Spark SQL
supports the following literals:
+
+ * [String Literal](#string-literal)
+ * [Null Literal](#null-literal)
+ * [Boolean Literal](#boolean-literal)
+ * [Numeric Literal](#numeric-literal)
+ * [Datetime Literal](#datetime-literal)
+ * [Interval Literal](#interval-literal)
+
+### String Literal
+
+A string literal is used to specify a character string value.
+
+ Syntax
+
+{% highlight sql %}
+'...' | "..."
+{% endhighlight %}
+...: indicates any string.
Review comment:
I added the escape description, but I reverted to the original syntax for
string, because we always use ```...``` to mean ```the preceding element can be
repeated```.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
huaxingao commented on a change in pull request #28237:
[SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
URL: https://github.com/apache/spark/pull/28237#discussion_r410809690
##
File path: docs/sql-ref-literals.md
##
@@ -0,0 +1,385 @@
+---
+layout: global
+title: Literals
+displayTitle: Literals
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+A literal (also known as a constant) represents a fixed data value. Spark SQL
supports the following literals:
+
+ * [String Literal](#string-literal)
+ * [Null Literal](#null-literal)
+ * [Boolean Literal](#boolean-literal)
+ * [Numeric Literal](#numeric-literal)
+ * [Datetime Literal](#datetime-literal)
+ * [Interval Literal](#interval-literal)
+
+### String Literal
+
+A string literal is used to specify a character string value.
+
+ Syntax
+
+{% highlight sql %}
+'...' | "..."
+{% endhighlight %}
+...: indicates any string.
+
+ Examples
+
+{% highlight sql %}
+SELECT 'Hello, World!' AS col;
+ +-+
+ | col|
+ +-+
+ |Hello, World!|
+ +-+
+
+SELECT "SPARK SQL" AS col;
+ +-+
+ | col|
+ +-+
+ |Spark SQL|
+ +-+
+
+SELECT SELECT 'it\'s $10.' AS col;
+ +-+
+ | col|
+ +-+
+ |It's $10.|
+ +--+
+{% endhighlight %}
+
+### Null Literal
+
+A null literal is used to specify a null value.
+
+ Syntax
+
+{% highlight sql %}
+NULL
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT NULL AS col;
+ ++
+ | col|
+ ++
+ |NULL|
+ ++
+{% endhighlight %}
+
+### Boolean Literal
+
+A boolean literal is used to specify a boolean value.
+
+ Syntax
+
+{% highlight sql %}
+TRUE | FALSE
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT TRUE AS col;
+ ++
+ | col|
+ ++
+ |true|
+ ++
+{% endhighlight %}
+
+### Numeric Literal
+
+A numeric literal is used to specify a fixed or floating-point number.
+
+ Integer Literal
+
+ Syntax
+
+{% highlight sql %}
+[ + | - ] digit [ ... ] [ L | S | Y ]
+{% endhighlight %}
+digit: one of 0, 1, 2, 3, 4, 5, 6, 7, 8 or 9.
+default (no postfix): indicates a 4-byte signed integer number.
+L: case insensitive, indicates BIGINT, which is a 8-byte signed
integer number.
Review comment:
I think we need this to differentiate it from the default case, which is
described as a 4-byte signed integer number.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
huaxingao commented on a change in pull request #28237:
[SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
URL: https://github.com/apache/spark/pull/28237#discussion_r410809710
##
File path: docs/sql-ref-literals.md
##
@@ -0,0 +1,385 @@
+---
+layout: global
+title: Literals
+displayTitle: Literals
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+A literal (also known as a constant) represents a fixed data value. Spark SQL
supports the following literals:
+
+ * [String Literal](#string-literal)
+ * [Null Literal](#null-literal)
+ * [Boolean Literal](#boolean-literal)
+ * [Numeric Literal](#numeric-literal)
+ * [Datetime Literal](#datetime-literal)
+ * [Interval Literal](#interval-literal)
+
+### String Literal
+
+A string literal is used to specify a character string value.
+
+ Syntax
+
+{% highlight sql %}
+'...' | "..."
+{% endhighlight %}
+...: indicates any string.
+
+ Examples
+
+{% highlight sql %}
+SELECT 'Hello, World!' AS col;
+ +-+
+ | col|
+ +-+
+ |Hello, World!|
+ +-+
+
+SELECT "SPARK SQL" AS col;
+ +-+
+ | col|
+ +-+
+ |Spark SQL|
+ +-+
+
+SELECT SELECT 'it\'s $10.' AS col;
+ +-+
+ | col|
+ +-+
+ |It's $10.|
+ +--+
+{% endhighlight %}
+
+### Null Literal
+
+A null literal is used to specify a null value.
+
+ Syntax
+
+{% highlight sql %}
+NULL
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT NULL AS col;
+ ++
+ | col|
+ ++
+ |NULL|
+ ++
+{% endhighlight %}
+
+### Boolean Literal
+
+A boolean literal is used to specify a boolean value.
+
+ Syntax
+
+{% highlight sql %}
+TRUE | FALSE
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT TRUE AS col;
+ ++
+ | col|
+ ++
+ |true|
+ ++
+{% endhighlight %}
+
+### Numeric Literal
+
+A numeric literal is used to specify a fixed or floating-point number.
+
+ Integer Literal
+
+ Syntax
+
+{% highlight sql %}
+[ + | - ] digit [ ... ] [ L | S | Y ]
+{% endhighlight %}
+digit: one of 0, 1, 2, 3, 4, 5, 6, 7, 8 or 9.
+default (no postfix): indicates a 4-byte signed integer number.
+L: case insensitive, indicates BIGINT, which is a 8-byte signed
integer number.
+S: case insensitive, indicates SMALLINT, which is a 2-byte signed
integer number.
+Y: case insensitive, indicates TINYINT, which is a 1-byte signed
integer number.
+ Examples
+
+{% highlight sql %}
+SELECT -2147483648 AS col;
+ +---+
+ |col|
+ +---+
+ |-2147483648|
+ +---+
+
+SELECT 9223372036854775807l AS col;
+ +---+
+ |col|
+ +---+
+ |9223372036854775807|
+ +---+
+
+SELECT -32Y AS col;
+ +---+
+ |col|
+ +---+
+ |-32|
+ +---+
+
+SELECT 482S AS col;
+ +---+
+ |col|
+ +---+
+ |482|
+ +---+
+{% endhighlight %}
+
+ Decimal Literal
+
+ Syntax
+
+{% highlight sql %}
+ [ + | - ] { digit [ ... ] . [ digit [ ... ] ] | . digit [ ... ] }
+{% endhighlight %}
+
+ Examples
+
+{% highlight sql %}
+SELECT 12.578 AS col;
+ +--+
+ | col|
+ +--+
+ |12.578|
+ +--+
+
+SELECT -0.1234567 AS col;
+ +--+
+ | col|
+ +--+
+ |-0.1234567|
+ +--+
+
+SELECT -.1234567 AS col;
+ +--+
+ | col|
+ +--+
+ |-0.1234567|
+ +--+
+{% endhighlight %}
+
+ Floating Point and BigDecimal Literals
+
+ Syntax
+
+{% highlight sql %}
+ [ + | - ] { digit [ ... ] [ E [ + | - ] digit [ ... ] ] [ D | BD ]
+ | digit [ ... ] . [ digit [ ... ] ] [ E [ + | - ] digit [ ... ] ]
[ D | BD ]
+ | . digit [ ... ] [ E [ + | - ] digit [ ... ] ] [ D | BD ] }
+{% endhighlight %}
+digit: one of 0, 1, 2, 3, 4, 5, 6, 7, 8 or 9.
+default (no postfix): indicate a 4-byte single-precision floating point
number.
+D: case insensitive, indicates DOUBLE, which is a 8-byte
double-precision floating point number.
+BD: case insensitive, indicates BIGDECIMAL, which is an
arbitrary-precision signed decimal number.
+ Examples
+
+{% highlight sql %}
+SELECT 5E2 AS col;
+ +-+
+ | col|
+ +-+
+ |500.0|
+ +-+
+
+SELECT 5D
[GitHub] [spark] huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#discussion_r410809705 ## File path: docs/sql-ref-literals.md ## @@ -0,0 +1,385 @@ +--- +layout: global +title: Literals +displayTitle: Literals +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +A literal (also known as a constant) represents a fixed data value. Spark SQL supports the following literals: + + * [String Literal](#string-literal) + * [Null Literal](#null-literal) + * [Boolean Literal](#boolean-literal) + * [Numeric Literal](#numeric-literal) + * [Datetime Literal](#datetime-literal) + * [Interval Literal](#interval-literal) Review comment: I think array and map are functions instead of constants, so no need to document these here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference
huaxingao commented on a change in pull request #28237: [SPARK-31465][SQL][DOCS] Document Literal in SQL Reference URL: https://github.com/apache/spark/pull/28237#discussion_r410809622 ## File path: docs/sql-ref-literals.md ## @@ -0,0 +1,378 @@ +--- +layout: global +title: Literals +displayTitle: Literals +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + +A literal (also known as a constant) represents a fixed data value. Spark SQL supports the following literals: + + * [String Literal](#string-literal) + * [Null Literal](#null-literal) + * [Boolean Literal](#boolean-literal) + * [Numeric Literal](#numeric-literal) + * [Datetime Literal](#datetime-literal) + * [Interval Literal](#interval-literal) + +### String Literal + +A string literal is used to specify a character string value. + + Syntax Review comment: OK I will remove This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
turboFei commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616017124 Will try to refactor other suites. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616015377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26152/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616015377 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26152/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616015376 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616015376 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException
SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the sql tests to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616015147 **[Test build #121468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121468/testReport)** for PR 28239 at commit [`babffc7`](https://github.com/apache/spark/commit/babffc701603d7b1243d739f26794b1eb4f09a74). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121467/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014474 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121467/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
SparkQA removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014236 **[Test build #121467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121467/testReport)** for PR 28239 at commit [`565631b`](https://github.com/apache/spark/commit/565631b3b97ede0ca81d3141ca771842322702cb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014471 **[Test build #121467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121467/testReport)** for PR 28239 at commit [`565631b`](https://github.com/apache/spark/commit/565631b3b97ede0ca81d3141ca771842322702cb). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014474 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014349 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014349 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26151/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26151/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014236 **[Test build #121467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121467/testReport)** for PR 28239 at commit [`565631b`](https://github.com/apache/spark/commit/565631b3b97ede0ca81d3141ca771842322702cb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
turboFei commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014159 > I think the change looks reasonable. Did you check them only in `SQLQuerySuite`? I mean, have you checked if the logs of the other test suites have no `TableAlreadyExistsException`? Hi, I found this issue when adding UT into hive/SQLQuerySuite. There is no `TableAlreadyExistsException` in the logs before add UT. I just checked the sql/SQLQuerySuite and refactor it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei edited a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
turboFei edited a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616014159 > I think the change looks reasonable. Did you check them only in `SQLQuerySuite`? I mean, have you checked if the logs of the other test suites have no `TableAlreadyExistsException`? Hi, I found this issue when adding UT into hive/SQLQuerySuite. There is no `TableAlreadyExistsException` in the logs before add UT. I just check the sql/SQLQuerySuite and refactor it too. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616011288 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26150/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616011288 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26150/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616011284 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616011284 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-616011120 **[Test build #121466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121466/testReport)** for PR 28239 at commit [`d699f58`](https://github.com/apache/spark/commit/d699f58b4eca6f8fe7ae1fb9b2ff0e92e0e4f013). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei commented on a change in pull request #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
turboFei commented on a change in pull request #28239: [SPARK-31467][SQL][TEST]
Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
URL: https://github.com/apache/spark/pull/28239#discussion_r410795576
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##
@@ -898,33 +913,39 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils
with TestHiveSingleton {
}
test("resolve udtf in projection #1") {
-val ds = (1 to 5).map(i => s"""{"a":[$i, ${i + 1}]}""").toDS()
-read.json(ds).createOrReplaceTempView("data")
-val df = sql("SELECT explode(a) AS val FROM data")
-val col = df("val")
+withTempView("data") {
+ val ds = (1 to 5).map(i => s"""{"a":[$i, ${i + 1}]}""").toDS()
+ read.json(ds).createOrReplaceTempView("data")
+ val df = sql("SELECT explode(a) AS val FROM data")
+ val col = df("val")
Review comment:
removed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616010038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26149/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616010033 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei commented on a change in pull request #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
turboFei commented on a change in pull request #28239: [SPARK-31467][SQL][TEST]
Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
URL: https://github.com/apache/spark/pull/28239#discussion_r410795468
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##
@@ -974,47 +995,57 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils
with TestHiveSingleton {
}
test("SPARK-5203 union with different decimal precision") {
-Seq.empty[(java.math.BigDecimal, java.math.BigDecimal)]
- .toDF("d1", "d2")
- .select($"d1".cast(DecimalType(10, 5)).as("d"))
- .createOrReplaceTempView("dn")
+withTempView("dn") {
+ Seq.empty[(java.math.BigDecimal, java.math.BigDecimal)]
+.toDF("d1", "d2")
+.select($"d1".cast(DecimalType(10, 5)).as("d"))
+.createOrReplaceTempView("dn")
-sql("select d from dn union all select d * 2 from dn")
- .queryExecution.analyzed
+ sql("select d from dn union all select d * 2 from dn")
+.queryExecution.analyzed
+}
}
test("Star Expansion - script transform") {
-assume(TestUtils.testCommandAvailable("/bin/bash"))
-val data = (1 to 10).map { i => (i, i, i) }
-data.toDF("d1", "d2", "d3").createOrReplaceTempView("script_trans")
-assert(10 === sql("SELECT TRANSFORM (*) USING 'cat' FROM
script_trans").count())
+withTempView("script_trans") {
+ assume(TestUtils.testCommandAvailable("/bin/bash"))
+ val data = (1 to 10).map { i => (i, i, i) }
+ data.toDF("d1", "d2", "d3").createOrReplaceTempView("script_trans")
+ assert(10 === sql("SELECT TRANSFORM (*) USING 'cat' FROM
script_trans").count())
+}
}
test("test script transform for stdout") {
-assume(TestUtils.testCommandAvailable("/bin/bash"))
-val data = (1 to 10).map { i => (i, i, i) }
-data.toDF("d1", "d2", "d3").createOrReplaceTempView("script_trans")
-assert(10 ===
- sql("SELECT TRANSFORM (d1, d2, d3) USING 'cat' AS (a,b,c) FROM
script_trans").count())
+withTempView("script_trans") {
+ assume(TestUtils.testCommandAvailable("/bin/bash"))
+ val data = (1 to 10).map { i => (i, i, i) }
+ data.toDF("d1", "d2", "d3").createOrReplaceTempView("script_trans")
+ assert(10 ===
+sql("SELECT TRANSFORM (d1, d2, d3) USING 'cat' AS (a,b,c) FROM
script_trans").count())
+}
}
test("test script transform for stderr") {
-assume(TestUtils.testCommandAvailable("/bin/bash"))
-val data = (1 to 10).map { i => (i, i, i) }
-data.toDF("d1", "d2", "d3").createOrReplaceTempView("script_trans")
-assert(0 ===
- sql("SELECT TRANSFORM (d1, d2, d3) USING 'cat 1>&2' AS (a,b,c) FROM
script_trans").count())
+withTempView("script_trans") {
+ assume(TestUtils.testCommandAvailable("/bin/bash"))
+ val data = (1 to 10).map { i => (i, i, i) }
+ data.toDF("d1", "d2", "d3").createOrReplaceTempView("script_trans")
+ assert(0 ===
+sql("SELECT TRANSFORM (d1, d2, d3) USING 'cat 1>&2' AS (a,b,c) FROM
script_trans").count())
+}
}
test("test script transform data type") {
-assume(TestUtils.testCommandAvailable("/bin/bash"))
-val data = (1 to 5).map { i => (i, i) }
-data.toDF("key", "value").createOrReplaceTempView("test")
-checkAnswer(
- sql("""FROM
- |(FROM test SELECT TRANSFORM(key, value) USING 'cat' AS (`thing1`
int, thing2 string)) t
- |SELECT thing1 + 1
+withTempView("test") {
+ assume(TestUtils.testCommandAvailable("/bin/bash"))
+ val data = (1 to 5).map { i => (i, i) }
+ data.toDF("key", "value").createOrReplaceTempView("test")
+ checkAnswer(
+sql("""FROM
+ |(FROM test SELECT TRANSFORM(key, value) USING 'cat' AS
(`thing1` int, thing2 string))
Review comment:
thanks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616010038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26149/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616010033 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
SparkQA commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616009863 **[Test build #121465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121465/testReport)** for PR 28255 at commit [`da210ea`](https://github.com/apache/spark/commit/da210ea809c3e37e51ef1cc1bafe5e7f9fe2c2ca). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
viirya commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-616008957 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] turboFei commented on a change in pull request #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
turboFei commented on a change in pull request #28239: [SPARK-31467][SQL][TEST]
Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
URL: https://github.com/apache/spark/pull/28239#discussion_r410793370
##
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
##
@@ -898,33 +913,39 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils
with TestHiveSingleton {
}
test("resolve udtf in projection #1") {
-val ds = (1 to 5).map(i => s"""{"a":[$i, ${i + 1}]}""").toDS()
-read.json(ds).createOrReplaceTempView("data")
-val df = sql("SELECT explode(a) AS val FROM data")
-val col = df("val")
+withTempView("data") {
+ val ds = (1 to 5).map(i => s"""{"a":[$i, ${i + 1}]}""").toDS()
+ read.json(ds).createOrReplaceTempView("data")
+ val df = sql("SELECT explode(a) AS val FROM data")
+ val col = df("val")
Review comment:
yes, should we remove this line?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-616003522 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-616003524 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26148/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-616003522 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
AmplabJenkins removed a comment on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-616003524 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26148/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
SparkQA commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-616003384 **[Test build #121464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121464/testReport)** for PR 27690 at commit [`073b2e5`](https://github.com/apache/spark/commit/073b2e5aa299225dae830542247be0f15488eba9). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] kiszk commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage
kiszk commented on issue #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage URL: https://github.com/apache/spark/pull/27690#issuecomment-616003139 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0
AmplabJenkins removed a comment on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0 URL: https://github.com/apache/spark/pull/28235#issuecomment-615999311 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121462/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0
AmplabJenkins commented on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0 URL: https://github.com/apache/spark/pull/28235#issuecomment-615999308 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0
AmplabJenkins removed a comment on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0 URL: https://github.com/apache/spark/pull/28235#issuecomment-615999308 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0
SparkQA commented on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0 URL: https://github.com/apache/spark/pull/28235#issuecomment-615999299 **[Test build #121462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121462/testReport)** for PR 28235 at commit [`7f47570`](https://github.com/apache/spark/commit/7f47570d78df8b5ab2598c9f085e7fc62cb063da). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0
AmplabJenkins commented on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0 URL: https://github.com/apache/spark/pull/28235#issuecomment-615999311 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121462/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0
SparkQA removed a comment on issue #28235: [SPARK-31464][BUILD][SS][test-maven] Upgrade Kafka to 2.5.0 URL: https://github.com/apache/spark/pull/28235#issuecomment-615983603 **[Test build #121462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121462/testReport)** for PR 28235 at commit [`7f47570`](https://github.com/apache/spark/commit/7f47570d78df8b5ab2598c9f085e7fc62cb063da). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-615999011 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-615999012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121460/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-615999012 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121460/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
AmplabJenkins removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-615999011 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
SparkQA removed a comment on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-615966442 **[Test build #121460 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121460/testReport)** for PR 28239 at commit [`cdde32f`](https://github.com/apache/spark/commit/cdde32f9fba62899ade205d8bdf93dc09fd34948). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException
SparkQA commented on issue #28239: [SPARK-31467][SQL][TEST] Refactor the tests in hive/SQLQuerySuite to prevent TableAlreadyExistsException URL: https://github.com/apache/spark/pull/28239#issuecomment-615998786 **[Test build #121460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121460/testReport)** for PR 28239 at commit [`cdde32f`](https://github.com/apache/spark/commit/cdde32f9fba62899ade205d8bdf93dc09fd34948). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-615997693 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121461/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-615997689 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins removed a comment on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-615997689 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception
AmplabJenkins commented on issue #28255: [SPARK-31484][Core] Add stage attempt number to temp checkpoint filename to avoid file already existing exception URL: https://github.com/apache/spark/pull/28255#issuecomment-615997693 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121461/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
