maropu commented on a change in pull request #29982:
URL: https://github.com/apache/spark/pull/29982#discussion_r503624175
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##########
@@ -519,15 +519,29 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
// Note: [SPARK-31595] if there is a `'` in a double quoted string, or a `"`
in a single quoted
// string, the origin implementation from Hive will not drop the trailing
semicolon as expected,
// hence we refined this function a little bit.
+ // Note: [SPARK-33100] Ignore the content inside bracketed comment and
ignore the comment without
+ // content.
private def splitSemiColon(line: String): JList[String] = {
var insideSingleQuote = false
var insideDoubleQuote = false
- var insideComment = false
+ var insideDashComment = false
Review comment:
`Dash` -> `Simple`
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4#L1816
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##########
@@ -519,15 +519,29 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
// Note: [SPARK-31595] if there is a `'` in a double quoted string, or a `"`
in a single quoted
// string, the origin implementation from Hive will not drop the trailing
semicolon as expected,
// hence we refined this function a little bit.
+ // Note: [SPARK-33100] Ignore the content inside bracketed comment and
ignore the comment without
Review comment:
This statement looks ambiguous, so how about saying `Ignore a semicolon
inside a bracketed comment in spark-sql`?
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##########
@@ -519,15 +519,29 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
// Note: [SPARK-31595] if there is a `'` in a double quoted string, or a `"`
in a single quoted
// string, the origin implementation from Hive will not drop the trailing
semicolon as expected,
// hence we refined this function a little bit.
+ // Note: [SPARK-33100] Ignore the content inside bracketed comment and
ignore the comment without
+ // content.
private def splitSemiColon(line: String): JList[String] = {
var insideSingleQuote = false
var insideDoubleQuote = false
- var insideComment = false
+ var insideDashComment = false
+ var insideBracketedComment = false
+ var bracketedCommentRightBound = -1
var escape = false
var beginIndex = 0
+ var contentBegin = false
Review comment:
Why do we need to track this state? `!insideComment` does not mean that
a cursor exists in contents?
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##########
@@ -519,15 +519,29 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
// Note: [SPARK-31595] if there is a `'` in a double quoted string, or a `"`
in a single quoted
// string, the origin implementation from Hive will not drop the trailing
semicolon as expected,
// hence we refined this function a little bit.
+ // Note: [SPARK-33100] Ignore the content inside bracketed comment and
ignore the comment without
+ // content.
private def splitSemiColon(line: String): JList[String] = {
var insideSingleQuote = false
var insideDoubleQuote = false
- var insideComment = false
+ var insideDashComment = false
+ var insideBracketedComment = false
+ var bracketedCommentRightBound = -1
var escape = false
var beginIndex = 0
+ var contentBegin = false
Review comment:
nit: btw, `content` -> `statement`?
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##########
@@ -550,21 +564,37 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
// Sample query: select "quoted value --"
// ^^ avoids starting a comment
if it's inside quotes.
} else if (hasNext && line.charAt(index + 1) == '-') {
- // ignore quotes and ;
- insideComment = true
+ // ignore quotes and ; in dash-comment
+ insideDashComment = true
}
} else if (line.charAt(index) == ';') {
if (insideSingleQuote || insideDoubleQuote || insideComment) {
// do not split
} else {
- // split, do not include ; itself
- ret.add(line.substring(beginIndex, index))
+ if (contentBegin) {
+ // split, do not include ; itself
+ ret.add(line.substring(beginIndex, index))
+ }
beginIndex = index + 1
+ contentBegin = false
}
} else if (line.charAt(index) == '\n') {
- // with a new line the inline comment should end.
+ // with a new line the inline dash-comment should end.
if (!escape) {
- insideComment = false
+ insideDashComment = false
+ }
+ } else if (line.charAt(index) == '/' && !insideComment) {
+ val hasNext = index + 1 < line.length
+ if (insideSingleQuote || insideDoubleQuote) {
+ // Ignores '/' in any case of quotes
+ } else if (hasNext && line.charAt(index + 1) == '*') {
+ // ignore quotes and ; in bracketed comment
+ insideBracketedComment = true
+ }
+ } else if (line.charAt(index) == '/' && insideBracketedComment) {
+ if (line.charAt(index - 1) == '*') {
+ // record the right bound of bracketed comment
+ bracketedCommentRightBound = index
Review comment:
We need a variable `bracketedCommentRightBound` for tracking a bracketed
comment? I think we can do it without it.
##########
File path:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
##########
@@ -550,21 +564,37 @@ private[hive] class SparkSQLCLIDriver extends CliDriver
with Logging {
// Sample query: select "quoted value --"
// ^^ avoids starting a comment
if it's inside quotes.
} else if (hasNext && line.charAt(index + 1) == '-') {
- // ignore quotes and ;
- insideComment = true
+ // ignore quotes and ; in dash-comment
Review comment:
nit: `// ignore quotes and semicolons in a simple comment`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
