dongjoon-hyun commented on a change in pull request #25357:
[SPARK-28617][SQL][TEST] Fix misplacement when comment is at the end of the
query
URL: https://github.com/apache/spark/pull/25357#discussion_r311394772
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala
##########
@@ -229,6 +229,7 @@ class SQLQueryTestSuite extends QueryTest with
SharedSQLContext {
// List of SQL queries to run
// note: this is not a robust way to split queries using semicolon, but
works for now.
val queries =
code.mkString("\n").split("(?<=[^\\\\]);").map(_.trim).filter(_ != "").toSeq
+
.map(_.split("\n").filterNot(_.startsWith("--")).mkString("\n")).map(_.trim).filter(_
!= "")
Review comment:
While testing my idea, I found that we (this PR and my idea) still miss the
following cases.
-
https://github.com/apache/spark/blob/master/sql/core/src/test/resources/sql-tests/inputs/pgSQL/with.sql#L137-L141
```
CREATE TABLE department (
id INTEGER, -- department ID
parent_department INTEGER, -- upper department ID
name string -- department name
) USING parquet;
```
Technically, this PR is able to remove a single comment or a comment after
`;`.
Also, the comments in the middle seems to be harmless.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
