Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/17736
For some reference. In 1.6 we used the Catalyst SqlParser to parse the
expression in `Dataframe.filter()`, and we used the Hive (ANTLR based) parser
for parsing for SQL commands. In Spark 2.0 we moved all of this to a single
parser. When porting the parser, I followed the rules in the Hive parser (incl.
the unescaping logic), and this fell through the cracks.
Java/scala normal strings make things mind meltingly confusing. I think it
is fair that we provide an option to disable the parser's unescaping as a way
to get out of this. This might not be the best solution if you use regexes in
both pure SQL and in scala at the same time, but it at least is an improvement.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
