javierivanov commented on issue #27321: [SPARK-30049][SQL] SQL fails to parse when comment contains an unmatched quote character. URL: https://github.com/apache/spark/pull/27321#issuecomment-581578721 Sorry for my delay here... spark.sql will parse only 1 SQL statement, hence the `;` is not accepted by SQL function. ``` scala> sql("""SELECT 1;""") org.apache.spark.sql.catalyst.parser.ParseException: extraneous input ';' expecting <EOF>(line 1, pos 8) == SQL == SELECT 1; --------^^^ ``` In this case the `;` is being catch in the comment. ``` scala> sql("""SELECT 1 -- someone's comment here;""") res1: org.apache.spark.sql.DataFrame = [1: int] ``` The SQLDriver is parsing SQLStatements into a list, and foreach calls the sql context to interpret each. This was failing because the parser was still detecting quotes inside of comments and passed the sql statement like this: ``` scala> sql("""SELECT 1 -- someone's comment here | ;""") org.apache.spark.sql.catalyst.parser.ParseException: extraneous input ';' expecting <EOF>(line 2, pos 0) == SQL == SELECT 1 -- someone's comment here ; ^^^ ``` Let me know for any comments 👍
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
