samsetegne opened a new pull request #27567: [SPARK-30822] Remove semicolon at the end of a sql query URL: https://github.com/apache/spark/pull/27567 # What changes were proposed in this pull request? This change proposes removing a terminating semicolon from queries submitted by the user (if included) instead of raising a parse exception. # Why are the changes needed? When a pyspark user submits a directly executable SQL statement terminated with a semicolon, they receive an `org.apache.spark.sql.catalyst.parser.ParseException` of `mismatched input ";"`. SQL-92 describes a direct SQL statement as having the format of `<directly executable statement> <semicolon>` and the majority of SQL implementations either require the semicolon as a statement terminator, or make it optional (meaning not raising an exception when it's included, seemingly in recognition that it's a common behavior). # Does this PR introduce any user-facing change? No # How was this patch tested? Manual test on a local dev Spark cluster. ### Current behavior: ```python df = spark.sql('select 1') df.show() #+---+ #| 1| #+---+ #| 1| #+---+ df = spark.sql('select 1;') #Traceback (most recent call last): # File "<stdin>", line 1, in <module> # File "/Users/ssetegne/spark/python/pyspark/sql/session.py", line 646, in sql # return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped) # File "/Users/ssetegne/spark/python/lib/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1286, in #__call__ # File "/Users/ssetegne/spark/python/pyspark/sql/utils.py", line 102, in deco # raise converted #pyspark.sql.utils.ParseException: #extraneous input ';' expecting <EOF>(line 1, pos 8) #== SQL == #select 1; #--------^^^ ``` ### Behavior after proposed fix: ```python df = spark.sql('select 1') df.show() #+---+ #| 1 | #+---+ #| 1 | #+---+ df = spark.sql('select 1;') df.show() #+---+ #| 1 | #+---+ #| 1 | #+---+ ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
