[GitHub] [spark] AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516274823 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516274827 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108362/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25294: [SPARK-28476][SQL][WIP] Support ALTER DATABASE SET LOCATION
SparkQA commented on issue #25294: [SPARK-28476][SQL][WIP] Support ALTER DATABASE SET LOCATION URL: https://github.com/apache/spark/pull/25294#issuecomment-516274682 **[Test build #108360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108360/testReport)** for PR 25294 at commit [`361d70f`](https://github.com/apache/spark/commit/361d70f1af1175a47f699334218a7f505fbe5489). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class AlterDatabaseLocationCommand(databaseName: String, location: String)` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
SparkQA removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516229493 **[Test build #108362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108362/testReport)** for PR 25040 at commit [`9e04c6e`](https://github.com/apache/spark/commit/9e04c6ee93ed77620c7e621c3ff4ef45b3f07e18). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
SparkQA commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516274360 **[Test build #108362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108362/testReport)** for PR 25040 at commit [`9e04c6e`](https://github.com/apache/spark/commit/9e04c6ee93ed77620c7e621c3ff4ef45b3f07e18). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
SparkQA commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516273962 **[Test build #108370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108370/testReport)** for PR 25134 at commit [`4ce0d33`](https://github.com/apache/spark/commit/4ce0d335287249a89457a57f4ff34d7f39690f21). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516273432 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516273439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13465/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516273432 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516273439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13465/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
AmplabJenkins removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516273291 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108358/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
AmplabJenkins removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516273284 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516273291 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108358/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
AmplabJenkins commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516273284 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
SparkQA removed a comment on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516226504 **[Test build #108358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108358/testReport)** for PR 25040 at commit [`2b77c80`](https://github.com/apache/spark/commit/2b77c801b8e5de79b3bacdc1ae4425934c5cb90a). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] felixcheung commented on a change in pull request #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3
felixcheung commented on a change in pull request #25289: [SPARK-27889][INFRA]
Make development scripts under dev/ support Python 3
URL: https://github.com/apache/spark/pull/25289#discussion_r308542017
##
File path: python/run-tests.py
##
@@ -161,8 +161,13 @@ def run_individual_python_test(target_dir, test_name,
pyspark_python):
def get_default_python_executables():
python_execs = [x for x in ["python2.7", "python3.6", "pypy"] if which(x)]
-if "python2.7" not in python_execs:
-LOGGER.warning("Not testing against `python2.7` because it could not
be found; falling"
+if ("python3.6" not in python_execs) and which("python3"):
+LOGGER.warning("Not testing against `python3.6` because it could not
be found; falling"
+ " back to `python3` instead")
+python_execs.insert(0, "python3")
+
+if ("python2.7" not in python_execs) and (not
python_execs[0].startswith("python3")):
Review comment:
should we handle `python2` also?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables.
SparkQA commented on issue #25040: [SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables. URL: https://github.com/apache/spark/pull/25040#issuecomment-516272815 **[Test build #108358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108358/testReport)** for PR 25040 at commit [`2b77c80`](https://github.com/apache/spark/commit/2b77c801b8e5de79b3bacdc1ae4425934c5cb90a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] WeichenXu123 commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
WeichenXu123 commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516272212 Jenkins, retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql
AmplabJenkins removed a comment on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql URL: https://github.com/apache/spark/pull/25148#issuecomment-516271417 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql
AmplabJenkins commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql URL: https://github.com/apache/spark/pull/25148#issuecomment-516271425 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108363/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql
AmplabJenkins removed a comment on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql URL: https://github.com/apache/spark/pull/25148#issuecomment-516271425 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108363/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql
AmplabJenkins commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql URL: https://github.com/apache/spark/pull/25148#issuecomment-516271417 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql
SparkQA removed a comment on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql URL: https://github.com/apache/spark/pull/25148#issuecomment-516232583 **[Test build #108363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108363/testReport)** for PR 25148 at commit [`6261c87`](https://github.com/apache/spark/commit/6261c875fbfa8af50bf7c3982e6f1489b7852745). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql
SparkQA commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql URL: https://github.com/apache/spark/pull/25148#issuecomment-516270965 **[Test build #108363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108363/testReport)** for PR 25148 at commit [`6261c87`](https://github.com/apache/spark/commit/6261c875fbfa8af50bf7c3982e6f1489b7852745). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array
beliefer commented on a change in pull request #25172: [SPARK-28412][SQL] ANSI
SQL: OVERLAY function support byte array
URL: https://github.com/apache/spark/pull/25172#discussion_r308540317
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
##
@@ -458,6 +458,7 @@ case class StringReplace(srcExpr: Expression, searchExpr:
Expression, replaceExp
object Overlay {
def calculate(input: UTF8String, replace: UTF8String, pos: Int, len: Int):
UTF8String = {
+require(pos > 0, "If you specify `pos`, it must be a positive whole
number.")
Review comment:
Yes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #25247: [SPARK-28319][SQL] Implement SHOW TABLES for Data Source V2 Tables
imback82 commented on a change in pull request #25247: [SPARK-28319][SQL]
Implement SHOW TABLES for Data Source V2 Tables
URL: https://github.com/apache/spark/pull/25247#discussion_r308459411
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ShowTablesExec.scala
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.datasources.v2
+
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.spark.rdd.RDD
+import org.apache.spark.sql.catalog.v2.{Identifier, TableCatalog}
+import org.apache.spark.sql.catalyst.InternalRow
+import org.apache.spark.sql.catalyst.encoders.RowEncoder
+import org.apache.spark.sql.catalyst.expressions.{AttributeReference,
GenericRowWithSchema}
+import org.apache.spark.sql.catalyst.plans.ShowTablesSchema
+import org.apache.spark.sql.catalyst.util.StringUtils
+import org.apache.spark.sql.execution.LeafExecNode
+
+/**
+ * Physical plan node for showing tables.
+ */
+case class ShowTablesExec(catalog: TableCatalog, ident: Identifier, pattern:
Option[String])
+ extends LeafExecNode {
+ override def output: Seq[AttributeReference] = ShowTablesSchema.attributes()
+
+ override protected def doExecute(): RDD[InternalRow] = {
+val rows = new ArrayBuffer[InternalRow]()
+val encoder = RowEncoder(ShowTablesSchema.schema).resolveAndBind()
+
+val tables = catalog.listTables(ident.namespace() :+ ident.name())
+tables.map { table =>
+ if (pattern.map(StringUtils.filterPattern(Seq(table.name()),
_).nonEmpty).getOrElse(true)) {
+rows += encoder.toRow(
+ new GenericRowWithSchema(
+// TODO: there is no v2 catalog API to retrieve 'isTemporary',
+// and it is set to false for the time being.
+Array(table.namespace().mkString("."), table.name(), false),
Review comment:
@rdblue does it make sense to use the whole namespace as a database name?
```
CREATE TABLE testcat.n1.n2.db.table_name (id bigint, data string) USING foo
```
Should it display the database as `db` , `n1.n2.db` or `testcat.n1.n2`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #25107: [SPARK-28344][SQL] detect ambiguous self-join and fail the query
gatorsmile commented on a change in pull request #25107: [SPARK-28344][SQL]
detect ambiguous self-join and fail the query
URL: https://github.com/apache/spark/pull/25107#discussion_r308538156
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameSelfJoinSuite.scala
##
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql
+
+import org.apache.spark.sql.functions.{count, sum}
+import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.sql.test.SharedSparkSession
+
+class DataFrameSelfJoinSuite extends QueryTest with SharedSparkSession {
Review comment:
Also need a test case for colRegex
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array
maropu commented on a change in pull request #25172: [SPARK-28412][SQL] ANSI
SQL: OVERLAY function support byte array
URL: https://github.com/apache/spark/pull/25172#discussion_r308537430
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala
##
@@ -458,6 +458,7 @@ case class StringReplace(srcExpr: Expression, searchExpr:
Expression, replaceExp
object Overlay {
def calculate(input: UTF8String, replace: UTF8String, pos: Int, len: Int):
UTF8String = {
+require(pos > 0, "If you specify `pos`, it must be a positive whole
number.")
Review comment:
This change isn't related to this pr, right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array
maropu commented on a change in pull request #25172: [SPARK-28412][SQL] ANSI
SQL: OVERLAY function support byte array
URL: https://github.com/apache/spark/pull/25172#discussion_r308536722
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala
##
@@ -452,6 +452,45 @@ class StringExpressionsSuite extends SparkFunSuite with
ExpressionEvalHelper {
// scalastyle:on
}
+ test("overlay for byte array") {
+val input = Literal(Array[Byte](1, 2, 3, 4, 5, 6, 7, 8, 9))
+checkEvaluation(new Overlay(input, Literal(Array[Byte](-1)),
+ Literal.create(6, IntegerType)), Array[Byte](1, 2, 3, 4, 5, -1, 7, 8, 9))
Review comment:
In this pr, plz follow the current behaivour of overlay(string, string). If
you wanna to change this behaivour, you can make an another pr for that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
dongjoon-hyun commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516266063 Thank you for updating again, @beliefer . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #24923: [SPARK-28071][SQL][TEST] Port strings.sql
wangyum commented on a change in pull request #24923: [SPARK-28071][SQL][TEST]
Port strings.sql
URL: https://github.com/apache/spark/pull/24923#discussion_r308536070
##
File path: sql/core/src/test/resources/sql-tests/results/pgSQL/strings.sql.out
##
@@ -0,0 +1,718 @@
+-- Automatically generated by SQLQueryTestSuite
+-- Number of queries: 88
+
+
+-- !query 0
+SELECT 'first line'
+' - next line'
+ ' - third line'
+ AS `Three lines to one`
+-- !query 0 schema
+struct
+-- !query 0 output
+first line - next line - third line
+
+
+-- !query 1
+SELECT 'first line'
+' - next line' /* this comment is not allowed here */
+' - third line'
+ AS `Illegal comment within continuation`
+-- !query 1 schema
+struct
+-- !query 1 output
+first line - next line - third line
+
+
+-- !query 2
+SELECT binary('\\xDeAdBeEf')
+-- !query 2 schema
+struct
+-- !query 2 output
+\xDeAdBeEf
Review comment:
Maybe the reason is output style:
```sql
postgres=# SET bytea_output TO hex;
SET
postgres=# SELECT E'\\xDeAdBeEf'::bytea;
bytea
\xdeadbeef
(1 row)
postgres=# SET bytea_output TO escape;
SET
postgres=# SELECT E'\\xDeAdBeEf'::bytea;
bytea
--
\336\255\276\357
(1 row)
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25163: [SPARK-28406][SQL][TEST] Port union.sql
maropu commented on a change in pull request #25163: [SPARK-28406][SQL][TEST]
Port union.sql
URL: https://github.com/apache/spark/pull/25163#discussion_r308535745
##
File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/union.sql
##
@@ -0,0 +1,472 @@
+--
+-- Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+--
+--
+-- UNION (also INTERSECT, EXCEPT)
+--
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/union.sql
+--
+
+CREATE OR REPLACE TEMPORARY VIEW INT4_TBL AS SELECT * FROM
+ (VALUES (0), (123456), (-123456), (2147483647), (-2147483647))
+ AS v(f1);
+CREATE OR REPLACE TEMPORARY VIEW INT8_TBL AS SELECT * FROM
+ (VALUES
+(123, 456),
+(123, 4567890123456789),
+(4567890123456789, 123),
+(4567890123456789, 4567890123456789),
+(4567890123456789, -4567890123456789))
+ AS v(q1, q2);
+CREATE OR REPLACE TEMPORARY VIEW FLOAT8_TBL AS SELECT * FROM
+ (VALUES (0.0), (-34.84), (-1004.30),
+(CAST('-1.2345678901234e+200' AS DOUBLE)), (CAST('-1.2345678901234e-200'
AS DOUBLE)))
+ AS v(f1);
+
+-- Simple UNION constructs
+
+SELECT 1 AS two UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS one UNION SELECT 1 ORDER BY 1;
+
+SELECT 1 AS two UNION ALL SELECT 2;
+
+SELECT 1 AS two UNION ALL SELECT 1;
+
+SELECT 1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1;
+
+SELECT 1 AS two UNION SELECT 2 UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT 1.1 AS two UNION SELECT 2.2 ORDER BY 1;
+
+-- Mixed types
+
+SELECT 1.1 AS two UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS two UNION SELECT 2.2 ORDER BY 1;
+
+SELECT 1 AS one UNION SELECT double(1.0) ORDER BY 1;
+
+SELECT 1.1 AS two UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT double(1.0) AS two UNION ALL SELECT 1 ORDER BY 1;
+
+SELECT 1.1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1;
+
+SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1;
+
+SELECT 1.1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT 1.1 AS two UNION (SELECT 2 UNION ALL SELECT 2) ORDER BY 1;
+
+--
+-- Try testing from tables...
+--
+
+SELECT f1 AS five FROM FLOAT8_TBL
+UNION
+SELECT f1 FROM FLOAT8_TBL
+ORDER BY 1;
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL
+SELECT f1 FROM FLOAT8_TBL;
+
+SELECT f1 AS nine FROM FLOAT8_TBL
+UNION
+SELECT f1 FROM INT4_TBL
+ORDER BY 1;
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL
+SELECT f1 FROM INT4_TBL;
+
+SELECT f1 AS five FROM FLOAT8_TBL
+ WHERE f1 BETWEEN -1e6 AND 1e6
+UNION
+SELECT f1 FROM INT4_TBL
+ WHERE f1 BETWEEN 0 AND 100
+ORDER BY 1;
+
+-- [SPARK-28298] Fully support char and varchar types
+-- SELECT CAST(f1 AS char(4)) AS three FROM VARCHAR_TBL
+-- UNION
+-- SELECT f1 FROM CHAR_TBL
+-- ORDER BY 1;
+
+-- SELECT f1 AS three FROM VARCHAR_TBL
+-- UNION
+-- SELECT CAST(f1 AS varchar) FROM CHAR_TBL
+-- ORDER BY 1;
+
+-- SELECT f1 AS eight FROM VARCHAR_TBL
+-- UNION ALL
+-- SELECT f1 FROM CHAR_TBL;
+
+-- SELECT f1 AS five FROM TEXT_TBL
+-- UNION
+-- SELECT f1 FROM VARCHAR_TBL
+-- UNION
+-- SELECT TRIM(TRAILING FROM f1) FROM CHAR_TBL
+-- ORDER BY 1;
+
+--
+-- INTERSECT and EXCEPT
+--
+
+SELECT q2 FROM int8_tbl INTERSECT SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl INTERSECT ALL SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT SELECT q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q2 FROM int8_tbl ORDER BY 1;
+
+-- Spark SQL do not support update
+-- SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl FOR NO KEY
UPDATE;
+
+-- nested cases
+(SELECT 1,2,3 UNION SELECT 4,5,6) INTERSECT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6 ORDER BY 1,2) INTERSECT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6) EXCEPT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6 ORDER BY 1,2) EXCEPT SELECT 4,5,6;
+
+-- exercise both hashed and sorted implementations of INTERSECT/EXCEPT
+
+-- set enable_hashagg to on;
+
+-- explain (costs off)
+-- select count(*) from
+-- ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+select count(*) from
+ ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+
+-- explain (costs off)
+-- select unique1 from tenk1 except select unique2 from tenk1 where unique2 !=
10;
+select unique1 from tenk1 except select unique2 from tenk1 where unique2 != 10;
+
+-- set enable_hashagg to off;
+
+-- explain (costs off)
+-- select count(*) from
+-- ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+select count(*) from
+ ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+
+-- explain (costs off)
+-- select unique1 from
[GitHub] [spark] maropu commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql
maropu commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql URL: https://github.com/apache/spark/pull/24923#issuecomment-516264911 Looks fine except for one comment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #25295: [SPARK-28560][SQL] Optimize shuffle reader to local shuffle reader when smj converted to bhj in adaptive execution
wangyum commented on a change in pull request #25295: [SPARK-28560][SQL]
Optimize shuffle reader to local shuffle reader when smj converted to bhj in
adaptive execution
URL: https://github.com/apache/spark/pull/25295#discussion_r308535322
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizedLocalShuffleReader.scala
##
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.execution.adaptive
+
+import org.apache.spark.sql.catalyst.rules.Rule
+import org.apache.spark.sql.execution.SparkPlan
+import org.apache.spark.sql.execution.adaptive.rule.CoalescedShuffleReaderExec
+import org.apache.spark.sql.execution.joins.{BroadcastHashJoinExec, BuildLeft,
BuildRight}
+import org.apache.spark.sql.internal.SQLConf
+
+case class OptimizedLocalShuffleReader(conf: SQLConf) extends Rule[SparkPlan] {
+ override def apply(plan: SparkPlan): SparkPlan = {
+if (!conf.optimizedLocalShuffleReaderEnabled) {
+ return plan
+}
+
+plan.transformUp {
+ case bhj: BroadcastHashJoinExec =>
+bhj.buildSide match {
+ case BuildLeft if
(bhj.right.isInstanceOf[CoalescedShuffleReaderExec]) =>
+bhj.right.asInstanceOf[CoalescedShuffleReaderExec].isLocal = true
+ case BuildRight if
(bhj.left.isInstanceOf[CoalescedShuffleReaderExec]) =>
+bhj.left.asInstanceOf[CoalescedShuffleReaderExec].isLocal = true
+ case _ => None
+}
+bhj
+}
+ }
+}
Review comment:
Add a new line.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #25295: [SPARK-28560][SQL] Optimize shuffle reader to local shuffle reader when smj converted to bhj in adaptive execution
wangyum commented on a change in pull request #25295: [SPARK-28560][SQL] Optimize shuffle reader to local shuffle reader when smj converted to bhj in adaptive execution URL: https://github.com/apache/spark/pull/25295#discussion_r308535350 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/ReduceNumShufflePartitions.scala ## @@ -19,15 +19,14 @@ package org.apache.spark.sql.execution.adaptive.rule import scala.collection.mutable.ArrayBuffer import scala.concurrent.duration.Duration - Review comment: Do not remove this line. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
SparkQA commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516262945 **[Test build #108369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108369/testReport)** for PR 25265 at commit [`f151fbf`](https://github.com/apache/spark/commit/f151fbf45db24bfc11b7e59e7fec1271877836d4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516262530 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516262534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13464/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516262534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13464/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516262530 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST] Port strings.sql
maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST]
Port strings.sql
URL: https://github.com/apache/spark/pull/24923#discussion_r308533221
##
File path: sql/core/src/test/resources/sql-tests/results/pgSQL/strings.sql.out
##
@@ -0,0 +1,718 @@
+-- Automatically generated by SQLQueryTestSuite
+-- Number of queries: 88
+
+
+-- !query 0
+SELECT 'first line'
+' - next line'
+ ' - third line'
+ AS `Three lines to one`
+-- !query 0 schema
+struct
+-- !query 0 output
+first line - next line - third line
+
+
+-- !query 1
+SELECT 'first line'
+' - next line' /* this comment is not allowed here */
+' - third line'
+ AS `Illegal comment within continuation`
+-- !query 1 schema
+struct
+-- !query 1 output
+first line - next line - third line
+
+
+-- !query 2
+SELECT binary('\\xDeAdBeEf')
+-- !query 2 schema
+struct
+-- !query 2 output
+\xDeAdBeEf
Review comment:
different output?
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/strings.out#L160
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #25163: [SPARK-28406][SQL][TEST] Port union.sql
wangyum commented on a change in pull request #25163: [SPARK-28406][SQL][TEST]
Port union.sql
URL: https://github.com/apache/spark/pull/25163#discussion_r308533121
##
File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/union.sql
##
@@ -0,0 +1,472 @@
+--
+-- Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+--
+--
+-- UNION (also INTERSECT, EXCEPT)
+--
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/union.sql
+--
+
+CREATE OR REPLACE TEMPORARY VIEW INT4_TBL AS SELECT * FROM
+ (VALUES (0), (123456), (-123456), (2147483647), (-2147483647))
+ AS v(f1);
+CREATE OR REPLACE TEMPORARY VIEW INT8_TBL AS SELECT * FROM
+ (VALUES
+(123, 456),
+(123, 4567890123456789),
+(4567890123456789, 123),
+(4567890123456789, 4567890123456789),
+(4567890123456789, -4567890123456789))
+ AS v(q1, q2);
+CREATE OR REPLACE TEMPORARY VIEW FLOAT8_TBL AS SELECT * FROM
+ (VALUES (0.0), (-34.84), (-1004.30),
+(CAST('-1.2345678901234e+200' AS DOUBLE)), (CAST('-1.2345678901234e-200'
AS DOUBLE)))
+ AS v(f1);
+
+-- Simple UNION constructs
+
+SELECT 1 AS two UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS one UNION SELECT 1 ORDER BY 1;
+
+SELECT 1 AS two UNION ALL SELECT 2;
+
+SELECT 1 AS two UNION ALL SELECT 1;
+
+SELECT 1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1;
+
+SELECT 1 AS two UNION SELECT 2 UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT 1.1 AS two UNION SELECT 2.2 ORDER BY 1;
+
+-- Mixed types
+
+SELECT 1.1 AS two UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS two UNION SELECT 2.2 ORDER BY 1;
+
+SELECT 1 AS one UNION SELECT double(1.0) ORDER BY 1;
+
+SELECT 1.1 AS two UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT double(1.0) AS two UNION ALL SELECT 1 ORDER BY 1;
+
+SELECT 1.1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1;
+
+SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1;
+
+SELECT 1.1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT 1.1 AS two UNION (SELECT 2 UNION ALL SELECT 2) ORDER BY 1;
+
+--
+-- Try testing from tables...
+--
+
+SELECT f1 AS five FROM FLOAT8_TBL
+UNION
+SELECT f1 FROM FLOAT8_TBL
+ORDER BY 1;
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL
+SELECT f1 FROM FLOAT8_TBL;
+
+SELECT f1 AS nine FROM FLOAT8_TBL
+UNION
+SELECT f1 FROM INT4_TBL
+ORDER BY 1;
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL
+SELECT f1 FROM INT4_TBL;
+
+SELECT f1 AS five FROM FLOAT8_TBL
+ WHERE f1 BETWEEN -1e6 AND 1e6
+UNION
+SELECT f1 FROM INT4_TBL
+ WHERE f1 BETWEEN 0 AND 100
+ORDER BY 1;
+
+-- [SPARK-28298] Fully support char and varchar types
+-- SELECT CAST(f1 AS char(4)) AS three FROM VARCHAR_TBL
+-- UNION
+-- SELECT f1 FROM CHAR_TBL
+-- ORDER BY 1;
+
+-- SELECT f1 AS three FROM VARCHAR_TBL
+-- UNION
+-- SELECT CAST(f1 AS varchar) FROM CHAR_TBL
+-- ORDER BY 1;
+
+-- SELECT f1 AS eight FROM VARCHAR_TBL
+-- UNION ALL
+-- SELECT f1 FROM CHAR_TBL;
+
+-- SELECT f1 AS five FROM TEXT_TBL
+-- UNION
+-- SELECT f1 FROM VARCHAR_TBL
+-- UNION
+-- SELECT TRIM(TRAILING FROM f1) FROM CHAR_TBL
+-- ORDER BY 1;
+
+--
+-- INTERSECT and EXCEPT
+--
+
+SELECT q2 FROM int8_tbl INTERSECT SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl INTERSECT ALL SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT SELECT q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q2 FROM int8_tbl ORDER BY 1;
+
+-- Spark SQL do not support update
+-- SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl FOR NO KEY
UPDATE;
+
+-- nested cases
+(SELECT 1,2,3 UNION SELECT 4,5,6) INTERSECT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6 ORDER BY 1,2) INTERSECT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6) EXCEPT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6 ORDER BY 1,2) EXCEPT SELECT 4,5,6;
+
+-- exercise both hashed and sorted implementations of INTERSECT/EXCEPT
+
+-- set enable_hashagg to on;
+
+-- explain (costs off)
+-- select count(*) from
+-- ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+select count(*) from
+ ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+
+-- explain (costs off)
+-- select unique1 from tenk1 except select unique2 from tenk1 where unique2 !=
10;
+select unique1 from tenk1 except select unique2 from tenk1 where unique2 != 10;
+
+-- set enable_hashagg to off;
+
+-- explain (costs off)
+-- select count(*) from
+-- ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+select count(*) from
+ ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+
+-- explain (costs off)
+-- select unique1
[GitHub] [spark] sarutak commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
sarutak commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516261636 retest this please. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST] Port strings.sql
maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST]
Port strings.sql
URL: https://github.com/apache/spark/pull/24923#discussion_r308532011
##
File path: sql/core/src/test/resources/sql-tests/results/pgSQL/strings.sql.out
##
@@ -0,0 +1,718 @@
+-- Automatically generated by SQLQueryTestSuite
+-- Number of queries: 88
+
+
+-- !query 0
+SELECT 'first line'
+' - next line'
+ ' - third line'
+ AS `Three lines to one`
+-- !query 0 schema
+struct
+-- !query 0 output
+first line - next line - third line
+
+
+-- !query 1
+SELECT 'first line'
+' - next line' /* this comment is not allowed here */
+' - third line'
+ AS `Illegal comment within continuation`
+-- !query 1 schema
+struct
+-- !query 1 output
+first line - next line - third line
+
+
+-- !query 2
+SELECT binary('\\xDeAdBeEf')
+-- !query 2 schema
+struct
+-- !query 2 output
+\xDeAdBeEf
+
+
+-- !query 3
+SELECT binary('\\x De Ad Be Ef ')
+-- !query 3 schema
+struct
+-- !query 3 output
+\x De Ad Be Ef
Review comment:
different output?
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/strings.out#L111
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on a change in pull request #25148: [SPARK-28326][SQL][TEST] Port join.sql
wangyum commented on a change in pull request #25148: [SPARK-28326][SQL][TEST]
Port join.sql
URL: https://github.com/apache/spark/pull/25148#discussion_r308532447
##
File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/join.sql
##
@@ -0,0 +1,2079 @@
+--
+-- Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+--
+--
+-- JOIN
+-- Test JOIN clauses
+--
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/join.sql
+--
+CREATE OR REPLACE TEMPORARY VIEW INT4_TBL AS SELECT * FROM
+ (VALUES (0), (123456), (-123456), (2147483647), (-2147483647))
+ AS v(f1);
+CREATE OR REPLACE TEMPORARY VIEW INT8_TBL AS SELECT * FROM
+ (VALUES
+(123, 456),
+(123, 4567890123456789),
+(4567890123456789, 123),
+(4567890123456789, 4567890123456789),
+(4567890123456789, -4567890123456789))
+ AS v(q1, q2);
+CREATE OR REPLACE TEMPORARY VIEW FLOAT8_TBL AS SELECT * FROM
+ (VALUES (0.0), (1004.30), (-34.84),
+(cast('1.2345678901234e+200' as double)), (cast('1.2345678901234e-200' as
double)))
+ AS v(f1);
+CREATE OR REPLACE TEMPORARY VIEW TEXT_TBL AS SELECT * FROM
+ (VALUES ('doh!'), ('hi de ho neighbor'))
+ AS v(f1);
+CREATE OR REPLACE TEMPORARY VIEW tenk2 AS SELECT * FROM tenk1;
+
+CREATE TABLE J1_TBL (
+ i integer,
+ j integer,
+ t string
+) USING parquet;
+
+CREATE TABLE J2_TBL (
+ i integer,
+ k integer
+) USING parquet;
+
+
+INSERT INTO J1_TBL VALUES (1, 4, 'one');
+INSERT INTO J1_TBL VALUES (2, 3, 'two');
+INSERT INTO J1_TBL VALUES (3, 2, 'three');
+INSERT INTO J1_TBL VALUES (4, 1, 'four');
+INSERT INTO J1_TBL VALUES (5, 0, 'five');
+INSERT INTO J1_TBL VALUES (6, 6, 'six');
+INSERT INTO J1_TBL VALUES (7, 7, 'seven');
+INSERT INTO J1_TBL VALUES (8, 8, 'eight');
+INSERT INTO J1_TBL VALUES (0, NULL, 'zero');
+INSERT INTO J1_TBL VALUES (NULL, NULL, 'null');
+INSERT INTO J1_TBL VALUES (NULL, 0, 'zero');
+
+INSERT INTO J2_TBL VALUES (1, -1);
+INSERT INTO J2_TBL VALUES (2, 2);
+INSERT INTO J2_TBL VALUES (3, -3);
+INSERT INTO J2_TBL VALUES (2, 4);
+INSERT INTO J2_TBL VALUES (5, -5);
+INSERT INTO J2_TBL VALUES (5, -5);
+INSERT INTO J2_TBL VALUES (0, NULL);
+INSERT INTO J2_TBL VALUES (NULL, NULL);
+INSERT INTO J2_TBL VALUES (NULL, 0);
+
+-- [SPARK-20856] Do not need onerow because it only used for test statement
using nested joins
+-- useful in some tests below
+-- create temp table onerow();
+-- insert into onerow default values;
+-- analyze onerow;
+
+
+--
+-- CORRELATION NAMES
+-- Make sure that table/column aliases are supported
+-- before diving into more complex join syntax.
+--
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL AS tx;
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL tx;
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL AS t1 (a, b, c);
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL t1 (a, b, c);
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL t1 (a, b, c), J2_TBL t2 (d, e);
+
+-- [SPARK-28377] Fully support correlation names in the FROM clause
+-- SELECT '' AS "xxx", t1.a, t2.e
+-- FROM J1_TBL t1 (a, b, c), J2_TBL t2 (d, e)
+-- WHERE t1.a = t2.d;
+
+
+--
+-- CROSS JOIN
+-- Qualifications are not allowed on cross joins,
+-- which degenerate into a standard unqualified inner join.
+--
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL CROSS JOIN J2_TBL;
+
+-- ambiguous column
+SELECT '' AS `xxx`, i, k, t
+ FROM J1_TBL CROSS JOIN J2_TBL;
+
+-- resolve previous ambiguity by specifying the table name
+SELECT '' AS `xxx`, t1.i, k, t
+ FROM J1_TBL t1 CROSS JOIN J2_TBL t2;
+
+SELECT '' AS `xxx`, ii, tt, kk
+ FROM (J1_TBL CROSS JOIN J2_TBL)
+AS tx (ii, jj, tt, ii2, kk);
+
+-- [SPARK-28377] Fully support correlation names in the FROM clause
+-- SELECT '' AS `xxx`, tx.ii, tx.jj, tx.kk
+-- FROM (J1_TBL t1 (a, b, c) CROSS JOIN J2_TBL t2 (d, e))
+-- AS tx (ii, jj, tt, ii2, kk);
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL CROSS JOIN J2_TBL a CROSS JOIN J2_TBL b;
+
+
+--
+--
+-- Inner joins (equi-joins)
+--
+--
+
+--
+-- Inner joins (equi-joins) with USING clause
+-- The USING syntax changes the shape of the resulting table
+-- by including a column in the USING clause only once in the result.
+--
+
+-- Inner equi-join on specified column
+SELECT '' AS `xxx`, *
+ FROM J1_TBL INNER JOIN J2_TBL USING (i);
+
+-- Same as above, slightly different syntax
+SELECT '' AS `xxx`, *
+ FROM J1_TBL JOIN J2_TBL USING (i);
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL t1 (a, b, c) JOIN J2_TBL t2 (a, d) USING (a)
+ ORDER BY a, d;
+
+-- [SPARK-28377] Fully support correlation names in the FROM clause
+-- SELECT '' AS `xxx`, *
+-- FROM J1_TBL t1 (a, b, c) JOIN J2_TBL t2 (a, b) USING (b)
+-- ORDER BY b, t1.a;
+
+
+--
+-- NATURAL JOIN
+-- Inner equi-join on all columns with the same name
+--
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL NATURAL JOIN J2_TBL;
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL t1 (a, b, c) NATURAL JOIN J2_TBL t2 (a, d);
+
+SELECT '' AS `xxx`, *
+ FROM J1_TBL t1 (a, b, c) NATURAL JOIN J2_TBL t2 (d, a);
+
+-- [SPARK-28377] Fully support correlation names in the FROM
[GitHub] [spark] wangyum commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql
wangyum commented on issue #25148: [SPARK-28326][SQL][TEST] Port join.sql URL: https://github.com/apache/spark/pull/25148#issuecomment-516260218 Thank you @maropu This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST] Port strings.sql
maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST]
Port strings.sql
URL: https://github.com/apache/spark/pull/24923#discussion_r308532011
##
File path: sql/core/src/test/resources/sql-tests/results/pgSQL/strings.sql.out
##
@@ -0,0 +1,718 @@
+-- Automatically generated by SQLQueryTestSuite
+-- Number of queries: 88
+
+
+-- !query 0
+SELECT 'first line'
+' - next line'
+ ' - third line'
+ AS `Three lines to one`
+-- !query 0 schema
+struct
+-- !query 0 output
+first line - next line - third line
+
+
+-- !query 1
+SELECT 'first line'
+' - next line' /* this comment is not allowed here */
+' - third line'
+ AS `Illegal comment within continuation`
+-- !query 1 schema
+struct
+-- !query 1 output
+first line - next line - third line
+
+
+-- !query 2
+SELECT binary('\\xDeAdBeEf')
+-- !query 2 schema
+struct
+-- !query 2 output
+\xDeAdBeEf
+
+
+-- !query 3
+SELECT binary('\\x De Ad Be Ef ')
+-- !query 3 schema
+struct
+-- !query 3 output
+\x De Ad Be Ef
Review comment:
different output?
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/expected/strings.out#L111
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST] Port strings.sql
maropu commented on a change in pull request #24923: [SPARK-28071][SQL][TEST]
Port strings.sql
URL: https://github.com/apache/spark/pull/24923#discussion_r308531894
##
File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/strings.sql
##
@@ -0,0 +1,656 @@
+--
+-- Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+--
+-- STRINGS
+-- --
https://github.com/postgres/postgres/blob/REL_12_BETA1/src/test/regress/sql/strings.sql
+-- Test various data entry syntaxes.
+--
+
+-- [SPARK-28073] ANSI SQL: Character literals
+-- SQL string continuation syntax
+-- E021-03 character string literals
+-- SELECT 'first line'
+-- ' - next line'
+-- ' - third line'
+-- AS "Three lines to one";
+
+-- illegal string continuation syntax
+-- SELECT 'first line'
+-- ' - next line' /* this comment is not allowed here */
+-- ' - third line'
+-- AS "Illegal comment within continuation";
+
+-- Unicode escapes
+-- SET standard_conforming_strings TO on;
+
+SELECT U&'d\0061t\+61' AS U&"d\0061t\+61";
+SELECT U&'d!0061t\+61' UESCAPE '!' AS U&"d*0061t\+61" UESCAPE '*';
+
+SELECT U&' \' UESCAPE '!' AS "tricky";
+SELECT 'tricky' AS U&"\" UESCAPE '!';
+
+SELECT U&'wrong: \061';
+SELECT U&'wrong: \+0061';
+SELECT U&'wrong: +0061' UESCAPE '+';
+
+-- SET standard_conforming_strings TO off;
+
+SELECT U&'d\0061t\+61' AS U&"d\0061t\+61";
+SELECT U&'d!0061t\+61' UESCAPE '!' AS U&"d*0061t\+61" UESCAPE '*';
+
+SELECT U&' \' UESCAPE '!' AS "tricky";
+SELECT 'tricky' AS U&"\" UESCAPE '!';
+
+SELECT U&'wrong: \061';
+SELECT U&'wrong: \+0061';
+SELECT U&'wrong: +0061' UESCAPE '+';
+
+-- RESET standard_conforming_strings;
+
+-- Spark SQL only support escape mode
+-- bytea
+-- SET bytea_output TO hex;
+-- SELECT E'\\xDeAdBeEf'::bytea;
+-- SELECT E'\\x De Ad Be Ef '::bytea;
+-- SELECT E'\\xDeAdBeE'::bytea;
+-- SELECT E'\\xDeAdBeEx'::bytea;
+-- SELECT E'\\xDe00BeEf'::bytea;
+-- SELECT E'DeAdBeEf'::bytea;
+-- SELECT E'De\\000dBeEf'::bytea;
+-- SELECT E'De\123dBeEf'::bytea;
+-- SELECT E'De\\123dBeEf'::bytea;
+-- SELECT E'De\\678dBeEf'::bytea;
+
+-- SET bytea_output TO escape;
+SELECT binary('\\xDeAdBeEf');
+SELECT binary('\\x De Ad Be Ef ');
+SELECT binary('\\xDe00BeEf');
+SELECT binary('DeAdBeEf');
+SELECT binary('De\\000dBeEf');
+SELECT binary('De\\123dBeEf');
+
+-- Skip these tests because we do not have char/varchar type
+--
+-- test conversions between various string types
+-- E021-10 implicit casting among the character data types
+--
+
+-- SELECT CAST(f1 AS text) AS "text(char)" FROM CHAR_TBL;
+
+-- SELECT CAST(f1 AS text) AS "text(varchar)" FROM VARCHAR_TBL;
+
+-- SELECT CAST(name 'namefield' AS text) AS "text(name)";
+
+-- since this is an explicit cast, it should truncate w/o error:
+-- SELECT CAST(f1 AS char(10)) AS "char(text)" FROM TEXT_TBL;
+-- note: implicit-cast case is tested in char.sql
+
+-- SELECT CAST(f1 AS char(20)) AS "char(text)" FROM TEXT_TBL;
+
+-- SELECT CAST(f1 AS char(10)) AS "char(varchar)" FROM VARCHAR_TBL;
+
+-- SELECT CAST(name 'namefield' AS char(10)) AS "char(name)";
+
+-- SELECT CAST(f1 AS varchar) AS "varchar(text)" FROM TEXT_TBL;
+
+-- SELECT CAST(f1 AS varchar) AS "varchar(char)" FROM CHAR_TBL;
+
+-- SELECT CAST(name 'namefield' AS varchar) AS "varchar(name)";
+
+--
+-- test SQL string functions
+-- E### and T### are feature reference numbers from SQL99
+--
+
+-- E021-09 trim function
+SELECT TRIM(BOTH FROM ' bunch o blanks ') = 'bunch o blanks' AS `bunch o
blanks`;
+
+SELECT TRIM(LEADING FROM ' bunch o blanks ') = 'bunch o blanks ' AS `bunch
o blanks `;
+
+SELECT TRIM(TRAILING FROM ' bunch o blanks ') = ' bunch o blanks' AS `
bunch o blanks`;
+
+SELECT TRIM(BOTH 'x' FROM 'xsome Xsx') = 'some Xs' AS `some Xs`;
+
+-- E021-06 substring expression
+SELECT SUBSTRING('1234567890' FROM 3) = '34567890' AS `34567890`;
+
+SELECT SUBSTRING('1234567890' FROM 4 FOR 3) = '456' AS `456`;
+
+-- [SPARK-28076] Support regular expression substring
+-- T581 regular expression substring (with SQL's bizarre regexp syntax)
+-- SELECT SUBSTRING('abcdefg' FROM 'a#"(b_d)#"%' FOR '#') AS "bcd";
+
+-- No match should return NULL
+-- SELECT SUBSTRING('abcdefg' FROM '#"(b_d)#"%' FOR '#') IS NULL AS "True";
+
+-- Null inputs should return NULL
+-- SELECT SUBSTRING('abcdefg' FROM '%' FOR NULL) IS NULL AS "True";
+-- SELECT SUBSTRING(NULL FROM '%' FOR '#') IS NULL AS "True";
+-- SELECT SUBSTRING('abcdefg' FROM NULL FOR '#') IS NULL AS "True";
+
+-- The first and last parts should act non-greedy
+-- SELECT SUBSTRING('abcdefg' FROM 'a#"%#"g' FOR '#') AS "bcdef";
+-- SELECT SUBSTRING('abcdefg' FROM 'a*#"%#"g*' FOR '#') AS "abcdefg";
+
+-- Vertical bar in any part affects only that part
+-- SELECT SUBSTRING('abcdefg' FROM 'a|b#"%#"g' FOR '#') AS "bcdef";
+-- SELECT SUBSTRING('abcdefg' FROM 'a#"%#"x|g' FOR '#') AS "bcdef";
+-- SELECT SUBSTRING('abcdefg' FROM 'a#"%|ab#"g' FOR '#') AS "bcdef";
+
+-- Can't have more than two part separators
+--
[GitHub] [spark] AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516259755 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108364/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516259751 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516259751 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
SparkQA removed a comment on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516241582 **[Test build #108364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108364/testReport)** for PR 25265 at commit [`f151fbf`](https://github.com/apache/spark/commit/f151fbf45db24bfc11b7e59e7fec1271877836d4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
AmplabJenkins commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516259755 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108364/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options
SparkQA commented on issue #25265: [SPARK-28525][DEPLOY] Allow Launcher to be applied Java options URL: https://github.com/apache/spark/pull/25265#issuecomment-516259585 **[Test build #108364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108364/testReport)** for PR 25265 at commit [`f151fbf`](https://github.com/apache/spark/commit/f151fbf45db24bfc11b7e59e7fec1271877836d4). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array
beliefer commented on issue #25172: [SPARK-28412][SQL] ANSI SQL: OVERLAY function support byte array URL: https://github.com/apache/spark/pull/25172#issuecomment-516257720 @maropu Could you continue to take a look at this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
beliefer commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516257276 > Please fix [#25074 (comment)](https://github.com/apache/spark/pull/25074#discussion_r308523551) , @beliefer . I have reverted that code. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
SparkQA commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516256940 **[Test build #108368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108368/testReport)** for PR 25074 at commit [`4794b67`](https://github.com/apache/spark/commit/4794b67f7b8721f9fb535f0bcf5c06b676b9cba4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
AmplabJenkins commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516256500 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL]
Support ANSI SQL Boolean-Predicate syntax
URL: https://github.com/apache/spark/pull/25074#discussion_r308529327
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala
##
@@ -110,6 +110,13 @@ package object dsl {
def isNull: Predicate = IsNull(expr)
def isNotNull: Predicate = IsNotNull(expr)
+def isTrue: Predicate = IsTrue(expr)
+def isNotTrue: Predicate = IsNotTrue(expr)
+def isFalse: Predicate = IsFalse(expr)
+def isNotFalse: Predicate = IsNotFalse(expr)
+def isUnknown: Predicate = IsUnknown(expr)
+def isNotUnknown: Predicate = IsNotUnknown(expr)
Review comment:
> No, @beliefer . You are confused with Scala DSL and SQL.
> This is `sql/catalyst/dsl` which providing Scala API. This is never ANSI
SQL.
> SQL Standard doesn't have `def isNotUnknown`. :)
Thanks for your prompt. I am confused indeed and clear now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
AmplabJenkins removed a comment on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516256503 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13463/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
AmplabJenkins commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516256503 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13463/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
AmplabJenkins removed a comment on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516256500 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
beliefer commented on issue #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074#issuecomment-516256143 > Also, I slightly updated the PR description, @beliefer . Thanks very much. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer closed pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
beliefer closed pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer opened a new pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
beliefer opened a new pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax URL: https://github.com/apache/spark/pull/25074 ## What changes were proposed in this pull request? This PR aims to support ANSI SQL `Boolean-Predicate` syntax. ```sql expression IS [NOT] TRUE expression IS [NOT] FALSE expression IS [NOT] UNKNOWN ``` There are some mainstream database support this syntax. - **PostgreSQL:** https://www.postgresql.org/docs/9.1/functions-comparison.html - **Hive:** https://issues.apache.org/jira/browse/HIVE-13583 - **Redshift:** https://docs.aws.amazon.com/redshift/latest/dg/r_Boolean_type.html - **Vertica:** https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/SQLReferenceManual/LanguageElements/Predicates/Boolean-predicate.htm For example: ```sql spark-sql> select null is true; false spark-sql> select null is not true; true spark-sql> select false is true; false spark-sql> select false is not true; true spark-sql> select true is true; true spark-sql> select true is not true; false spark-sql> select null is false; false spark-sql> select null is not false; true spark-sql> select false is false; true spark-sql> select false is not false; false spark-sql> select true is false; false spark-sql> select true is not false; true spark-sql> select null is unknown; true spark-sql> select null is not unknown; false spark-sql> select false is unknown; false spark-sql> select false is not unknown; true spark-sql> select true is unknown; false spark-sql> select true is not unknown; true ``` **Note**: A null input is treated as the logical value "unknown". ## How was this patch tested? Pass the Jenkins with the newly added test cases. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on issue #25163: [SPARK-28406][SQL][TEST] Port union.sql
maropu commented on issue #25163: [SPARK-28406][SQL][TEST] Port union.sql URL: https://github.com/apache/spark/pull/25163#issuecomment-516254879 Looks ok except for one comment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25163: [SPARK-28406][SQL][TEST] Port union.sql
maropu commented on a change in pull request #25163: [SPARK-28406][SQL][TEST]
Port union.sql
URL: https://github.com/apache/spark/pull/25163#discussion_r308527534
##
File path: sql/core/src/test/resources/sql-tests/inputs/pgSQL/union.sql
##
@@ -0,0 +1,472 @@
+--
+-- Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
+--
+--
+-- UNION (also INTERSECT, EXCEPT)
+--
https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/union.sql
+--
+
+CREATE OR REPLACE TEMPORARY VIEW INT4_TBL AS SELECT * FROM
+ (VALUES (0), (123456), (-123456), (2147483647), (-2147483647))
+ AS v(f1);
+CREATE OR REPLACE TEMPORARY VIEW INT8_TBL AS SELECT * FROM
+ (VALUES
+(123, 456),
+(123, 4567890123456789),
+(4567890123456789, 123),
+(4567890123456789, 4567890123456789),
+(4567890123456789, -4567890123456789))
+ AS v(q1, q2);
+CREATE OR REPLACE TEMPORARY VIEW FLOAT8_TBL AS SELECT * FROM
+ (VALUES (0.0), (-34.84), (-1004.30),
+(CAST('-1.2345678901234e+200' AS DOUBLE)), (CAST('-1.2345678901234e-200'
AS DOUBLE)))
+ AS v(f1);
+
+-- Simple UNION constructs
+
+SELECT 1 AS two UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS one UNION SELECT 1 ORDER BY 1;
+
+SELECT 1 AS two UNION ALL SELECT 2;
+
+SELECT 1 AS two UNION ALL SELECT 1;
+
+SELECT 1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1;
+
+SELECT 1 AS two UNION SELECT 2 UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT 1.1 AS two UNION SELECT 2.2 ORDER BY 1;
+
+-- Mixed types
+
+SELECT 1.1 AS two UNION SELECT 2 ORDER BY 1;
+
+SELECT 1 AS two UNION SELECT 2.2 ORDER BY 1;
+
+SELECT 1 AS one UNION SELECT double(1.0) ORDER BY 1;
+
+SELECT 1.1 AS two UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT double(1.0) AS two UNION ALL SELECT 1 ORDER BY 1;
+
+SELECT 1.1 AS three UNION SELECT 2 UNION SELECT 3 ORDER BY 1;
+
+SELECT double(1.1) AS two UNION SELECT 2 UNION SELECT double(2.0) ORDER BY 1;
+
+SELECT 1.1 AS three UNION SELECT 2 UNION ALL SELECT 2 ORDER BY 1;
+
+SELECT 1.1 AS two UNION (SELECT 2 UNION ALL SELECT 2) ORDER BY 1;
+
+--
+-- Try testing from tables...
+--
+
+SELECT f1 AS five FROM FLOAT8_TBL
+UNION
+SELECT f1 FROM FLOAT8_TBL
+ORDER BY 1;
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL
+SELECT f1 FROM FLOAT8_TBL;
+
+SELECT f1 AS nine FROM FLOAT8_TBL
+UNION
+SELECT f1 FROM INT4_TBL
+ORDER BY 1;
+
+SELECT f1 AS ten FROM FLOAT8_TBL
+UNION ALL
+SELECT f1 FROM INT4_TBL;
+
+SELECT f1 AS five FROM FLOAT8_TBL
+ WHERE f1 BETWEEN -1e6 AND 1e6
+UNION
+SELECT f1 FROM INT4_TBL
+ WHERE f1 BETWEEN 0 AND 100
+ORDER BY 1;
+
+-- [SPARK-28298] Fully support char and varchar types
+-- SELECT CAST(f1 AS char(4)) AS three FROM VARCHAR_TBL
+-- UNION
+-- SELECT f1 FROM CHAR_TBL
+-- ORDER BY 1;
+
+-- SELECT f1 AS three FROM VARCHAR_TBL
+-- UNION
+-- SELECT CAST(f1 AS varchar) FROM CHAR_TBL
+-- ORDER BY 1;
+
+-- SELECT f1 AS eight FROM VARCHAR_TBL
+-- UNION ALL
+-- SELECT f1 FROM CHAR_TBL;
+
+-- SELECT f1 AS five FROM TEXT_TBL
+-- UNION
+-- SELECT f1 FROM VARCHAR_TBL
+-- UNION
+-- SELECT TRIM(TRAILING FROM f1) FROM CHAR_TBL
+-- ORDER BY 1;
+
+--
+-- INTERSECT and EXCEPT
+--
+
+SELECT q2 FROM int8_tbl INTERSECT SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl INTERSECT ALL SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q2 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q1 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT SELECT q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q2 FROM int8_tbl ORDER BY 1;
+
+SELECT q1 FROM int8_tbl EXCEPT ALL SELECT DISTINCT q2 FROM int8_tbl ORDER BY 1;
+
+-- Spark SQL do not support update
+-- SELECT q1 FROM int8_tbl EXCEPT ALL SELECT q1 FROM int8_tbl FOR NO KEY
UPDATE;
+
+-- nested cases
+(SELECT 1,2,3 UNION SELECT 4,5,6) INTERSECT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6 ORDER BY 1,2) INTERSECT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6) EXCEPT SELECT 4,5,6;
+(SELECT 1,2,3 UNION SELECT 4,5,6 ORDER BY 1,2) EXCEPT SELECT 4,5,6;
+
+-- exercise both hashed and sorted implementations of INTERSECT/EXCEPT
+
+-- set enable_hashagg to on;
+
+-- explain (costs off)
+-- select count(*) from
+-- ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+select count(*) from
+ ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+
+-- explain (costs off)
+-- select unique1 from tenk1 except select unique2 from tenk1 where unique2 !=
10;
+select unique1 from tenk1 except select unique2 from tenk1 where unique2 != 10;
+
+-- set enable_hashagg to off;
+
+-- explain (costs off)
+-- select count(*) from
+-- ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+select count(*) from
+ ( select unique1 from tenk1 intersect select fivethous from tenk1 ) ss;
+
+-- explain (costs off)
+-- select unique1 from
[GitHub] [spark] beliefer commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
beliefer commented on a change in pull request #25074: [SPARK-27924][SQL]
Support ANSI SQL Boolean-Predicate syntax
URL: https://github.com/apache/spark/pull/25074#discussion_r308527268
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
##
@@ -840,3 +840,81 @@ case class GreaterThanOrEqual(left: Expression, right:
Expression)
protected override def nullSafeEval(input1: Any, input2: Any): Any =
ordering.gteq(input1, input2)
}
+
+trait BooleanTest extends UnaryExpression with Predicate with
ExpectsInputTypes {
+
+ def boolValue: Boolean
+ def whenNull: Boolean
+
+ override def nullable: Boolean = false
+ override def inputTypes: Seq[DataType] = Seq(BooleanType)
+
+ override def eval(input: InternalRow): Any = {
+val value = child.eval(input)
+Option(value) match {
+ case None => whenNull
+ case other => if (whenNull) {
+value == !boolValue
+ } else {
+value == boolValue
+ }
+}
+ }
+
+ override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
+val eval = child.genCode(ctx)
+ev.copy(code = code"""
+ ${eval.code}
+ ${CodeGenerator.javaType(dataType)} ${ev.value} =
${CodeGenerator.defaultValue(dataType)};
+ if (${eval.isNull}) {
+${ev.value} = $whenNull;
+ } else if ($whenNull) {
+${ev.value} = ${eval.value} == !$boolValue;
+ } else {
+${ev.value} = ${eval.value} == $boolValue;
+ }
+ """, isNull = FalseLiteral)
+ }
+}
+
+case class IsTrue(child: Expression) extends BooleanTest {
+ override def boolValue: Boolean = true
+ override def whenNull: Boolean = false
+ override def sql: String = s"(${child.sql} IS TRUE)"
+}
+
+case class IsNotTrue(child: Expression) extends BooleanTest {
+ override def boolValue: Boolean = true
+ override def whenNull: Boolean = true
+ override def sql: String = s"(${child.sql} IS NOT TRUE)"
+}
+
+case class IsFalse(child: Expression) extends BooleanTest {
+ override def boolValue: Boolean = false
+ override def whenNull: Boolean = false
+ override def sql: String = s"(${child.sql} IS FALSE)"
+}
+
+case class IsNotFalse(child: Expression) extends BooleanTest {
+ override def boolValue: Boolean = false
+ override def whenNull: Boolean = true
+ override def sql: String = s"(${child.sql} IS NOT FALSE)"
+}
+
+object IsUnknown {
Review comment:
OK. I will supplement it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql
AmplabJenkins removed a comment on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql URL: https://github.com/apache/spark/pull/24923#issuecomment-516253238 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql
AmplabJenkins removed a comment on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql URL: https://github.com/apache/spark/pull/24923#issuecomment-516253240 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108354/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql
AmplabJenkins commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql URL: https://github.com/apache/spark/pull/24923#issuecomment-516253240 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108354/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql
AmplabJenkins commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql URL: https://github.com/apache/spark/pull/24923#issuecomment-516253238 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#discussion_r308523594 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ## @@ -83,33 +83,39 @@ abstract class StringRegexExpression extends BinaryExpression % matches zero or more characters in the input (similar to .* in posix regular expressions) - The escape character is '\'. If an escape character precedes a special symbol or another - escape character, the following character is matched literally. It is invalid to escape - any other character. - Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order to match "\abc", the pattern should be "\\abc". When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks to Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the pattern to match "\abc" should be "\abc". + * escape - a optional string. The default escape character is the '\' . If an escape character + precedes a special symbol or another escape character, the following character is matched + literally. It is invalid to escape any other character. You can specify '' as escape + character so that disables the escape mechanism, which makes it impossible to turn off + the special meaning of underscore and percent signs in the pattern. Review comment: > If we follow `1)`, we should throw an exception for the empty case? Yes. Should we follow which behavior? Oracle is follow the behavior of ANSI, PostGresql will disables the escape with '', MySQL will ignore the '' and use default escape char. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql
SparkQA removed a comment on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql URL: https://github.com/apache/spark/pull/24923#issuecomment-516213608 **[Test build #108354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108354/testReport)** for PR 24923 at commit [`c2b06e0`](https://github.com/apache/spark/commit/c2b06e0b8504353952077b07bf00322e35975227). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql
SparkQA commented on issue #24923: [SPARK-28071][SQL][TEST] Port strings.sql URL: https://github.com/apache/spark/pull/24923#issuecomment-516252826 **[Test build #108354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108354/testReport)** for PR 24923 at commit [`c2b06e0`](https://github.com/apache/spark/commit/c2b06e0b8504353952077b07bf00322e35975227). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#discussion_r308084147 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ## @@ -83,33 +83,39 @@ abstract class StringRegexExpression extends BinaryExpression % matches zero or more characters in the input (similar to .* in posix regular expressions) - The escape character is '\'. If an escape character precedes a special symbol or another - escape character, the following character is matched literally. It is invalid to escape - any other character. - Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order to match "\abc", the pattern should be "\\abc". When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks to Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the pattern to match "\abc" should be "\abc". + * escape - a optional string. The default escape character is the '\' . If an escape character + precedes a special symbol or another escape character, the following character is matched + literally. It is invalid to escape any other character. You can specify '' as escape + character so that disables the escape mechanism, which makes it impossible to turn off + the special meaning of underscore and percent signs in the pattern. Review comment: ``` SELECT 'a_%b' LIKE 'a\__b', 'a_%b' LIKE 'a\__b' ESCAPE '', 'addb' LIKE 'a_%b', 'addb' LIKE 'a_%b' ESCAPE ''; //PostGresql result is: tf t t // MySQL result is: 1 1 1 1 ``` I execute SQL in Oracle: ``` SELECT * FROM DUAL WHERE DUMMY LIKE 'a\__b' ESCAPE '' 报错：> ORA-01425: escape character must be character string of length 1 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516252573 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108361/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516252566 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516252573 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108361/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
AmplabJenkins commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516252566 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql
AmplabJenkins removed a comment on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql URL: https://github.com/apache/spark/pull/24862#issuecomment-516252132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108355/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql
AmplabJenkins removed a comment on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql URL: https://github.com/apache/spark/pull/24862#issuecomment-516252130 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
SparkQA removed a comment on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516229482 **[Test build #108361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108361/testReport)** for PR 25134 at commit [`4ce0d33`](https://github.com/apache/spark/commit/4ce0d335287249a89457a57f4ff34d7f39690f21). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file
SparkQA commented on issue #25134: [SPARK-28366][CORE] Logging in driver when loading single large unsplittable file URL: https://github.com/apache/spark/pull/25134#issuecomment-516252428 **[Test build #108361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108361/testReport)** for PR 25134 at commit [`4ce0d33`](https://github.com/apache/spark/commit/4ce0d335287249a89457a57f4ff34d7f39690f21). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql
AmplabJenkins commented on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql URL: https://github.com/apache/spark/pull/24862#issuecomment-516252132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108355/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql
AmplabJenkins commented on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql URL: https://github.com/apache/spark/pull/24862#issuecomment-516252130 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #25245: [SPARK-25382][SQL][PYSPARK] Remove ImageSchema.readImages in 3.0
HyukjinKwon commented on issue #25245: [SPARK-25382][SQL][PYSPARK] Remove ImageSchema.readImages in 3.0 URL: https://github.com/apache/spark/pull/25245#issuecomment-516251965 Thanks @srowen and @WeichenXu123 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql
SparkQA removed a comment on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql URL: https://github.com/apache/spark/pull/24862#issuecomment-516214987 **[Test build #108355 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108355/testReport)** for PR 24862 at commit [`c9d3d16`](https://github.com/apache/spark/commit/c9d3d16a19e5c8991f0e458c99b40973e1e553ae). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql
SparkQA commented on issue #24862: [SPARK-28038][SQL][TEST] Port text.sql URL: https://github.com/apache/spark/pull/24862#issuecomment-516251831 **[Test build #108355 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108355/testReport)** for PR 24862 at commit [`c9d3d16`](https://github.com/apache/spark/commit/c9d3d16a19e5c8991f0e458c99b40973e1e553ae). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3
AmplabJenkins commented on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-516251281 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108356/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3
AmplabJenkins removed a comment on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-516251281 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108356/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3
AmplabJenkins commented on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-516251277 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3
AmplabJenkins removed a comment on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-516251277 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3
SparkQA removed a comment on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-516222020 **[Test build #108356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108356/testReport)** for PR 25289 at commit [`b2aa278`](https://github.com/apache/spark/commit/b2aa278da6572a351f0d43ac89d752bedb18a045). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3
SparkQA commented on issue #25289: [SPARK-27889][INFRA] Make development scripts under dev/ support Python 3 URL: https://github.com/apache/spark/pull/25289#issuecomment-516250952 **[Test build #108356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108356/testReport)** for PR 25289 at commit [`b2aa278`](https://github.com/apache/spark/commit/b2aa278da6572a351f0d43ac89d752bedb18a045). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
maropu commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#discussion_r308523852 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ## @@ -83,33 +83,39 @@ abstract class StringRegexExpression extends BinaryExpression % matches zero or more characters in the input (similar to .* in posix regular expressions) - The escape character is '\'. If an escape character precedes a special symbol or another - escape character, the following character is matched literally. It is invalid to escape - any other character. - Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order to match "\abc", the pattern should be "\\abc". When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks to Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the pattern to match "\abc" should be "\abc". + * escape - a optional string. The default escape character is the '\' . If an escape character + precedes a special symbol or another escape character, the following character is matched + literally. It is invalid to escape any other character. You can specify '' as escape + character so that disables the escape mechanism, which makes it impossible to turn off + the special meaning of underscore and percent signs in the pattern. Review comment: I like the standard behivour, but we should wait for the other review's comment. cc: @dongjoon-hyun @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
dongjoon-hyun commented on a change in pull request #25074: [SPARK-27924][SQL]
Support ANSI SQL Boolean-Predicate syntax
URL: https://github.com/apache/spark/pull/25074#discussion_r308523551
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala
##
@@ -110,6 +110,13 @@ package object dsl {
def isNull: Predicate = IsNull(expr)
def isNotNull: Predicate = IsNotNull(expr)
+def isTrue: Predicate = IsTrue(expr)
+def isNotTrue: Predicate = IsNotTrue(expr)
+def isFalse: Predicate = IsFalse(expr)
+def isNotFalse: Predicate = IsNotFalse(expr)
+def isUnknown: Predicate = IsUnknown(expr)
+def isNotUnknown: Predicate = IsNotUnknown(expr)
Review comment:
No, @beliefer . You are confused with Scala DSL and SQL.
This is `sql/catalyst/dsl` which providing Scala API. This is never ANSI SQL.
SQL Standard doesn't have `def isNotUnknown`. :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
maropu commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#discussion_r308523852 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ## @@ -83,33 +83,39 @@ abstract class StringRegexExpression extends BinaryExpression % matches zero or more characters in the input (similar to .* in posix regular expressions) - The escape character is '\'. If an escape character precedes a special symbol or another - escape character, the following character is matched literally. It is invalid to escape - any other character. - Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order to match "\abc", the pattern should be "\\abc". When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks to Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the pattern to match "\abc" should be "\abc". + * escape - a optional string. The default escape character is the '\' . If an escape character + precedes a special symbol or another escape character, the following character is matched + literally. It is invalid to escape any other character. You can specify '' as escape + character so that disables the escape mechanism, which makes it impossible to turn off + the special meaning of underscore and percent signs in the pattern. Review comment: I like the behivour, but we should wait for the other review's comment. cc: @dongjoon-hyun @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25074: [SPARK-27924][SQL] Support ANSI SQL Boolean-Predicate syntax
dongjoon-hyun commented on a change in pull request #25074: [SPARK-27924][SQL]
Support ANSI SQL Boolean-Predicate syntax
URL: https://github.com/apache/spark/pull/25074#discussion_r308523551
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/dsl/package.scala
##
@@ -110,6 +110,13 @@ package object dsl {
def isNull: Predicate = IsNull(expr)
def isNotNull: Predicate = IsNotNull(expr)
+def isTrue: Predicate = IsTrue(expr)
+def isNotTrue: Predicate = IsNotTrue(expr)
+def isFalse: Predicate = IsFalse(expr)
+def isNotFalse: Predicate = IsNotFalse(expr)
+def isUnknown: Predicate = IsUnknown(expr)
+def isNotUnknown: Predicate = IsNotUnknown(expr)
Review comment:
No, @beliefer . You are confused with Scala DSL and SQL.
This is `sql/catalyst/dsl` which providing Scala API. This is never ANSI SQL.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax
beliefer commented on a change in pull request #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#discussion_r308523594 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala ## @@ -83,33 +83,39 @@ abstract class StringRegexExpression extends BinaryExpression % matches zero or more characters in the input (similar to .* in posix regular expressions) - The escape character is '\'. If an escape character precedes a special symbol or another - escape character, the following character is matched literally. It is invalid to escape - any other character. - Since Spark 2.0, string literals are unescaped in our SQL parser. For example, in order to match "\abc", the pattern should be "\\abc". When SQL config 'spark.sql.parser.escapedStringLiterals' is enabled, it fallbacks to Spark 1.6 behavior regarding string literal parsing. For example, if the config is enabled, the pattern to match "\abc" should be "\abc". + * escape - a optional string. The default escape character is the '\' . If an escape character + precedes a special symbol or another escape character, the following character is matched + literally. It is invalid to escape any other character. You can specify '' as escape + character so that disables the escape mechanism, which makes it impossible to turn off + the special meaning of underscore and percent signs in the pattern. Review comment: > If we follow `1)`, we should throw an exception for the empty case? Yes. Should we follow which behavior? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
