[
https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264452#comment-17264452
]
Noah Kawasaki commented on SPARK-17647:
---------------------------------------
I can also confirm that this issue is not fully resolved. Like what [~swiegleb]
has shown, escape characters are not fully supported.
I have tested Spark versions 2.1, 2.2, 2.3, 2.4, and 3.0 and they all
experience the issue:
{code:java}
# These do not return the expected backslash
SET spark.sql.parser.escapedStringLiterals=false;
SELECT '\\';
> \
(should return \\)
SELECT 'hi\hi';
> hihi
(should return hi\hi)
# These are correctly escaped
SELECT '\"';
> "
SELECT '\'';
> '{code}
If I switch this:
{code:java}
# These now work
SET spark.sql.parser.escapedStringLiterals=true;
SELECT '\\';
> \\
SELECT 'hi\hi';
> hi\hi
# These are now not correctly escaped
SELECT '\"';
> \"
(should return ")
SELECT '\'';
> \'
(should return ' ){code}
So basically we have to choose:
SET spark.sql.parser.escapedStringLiterals=false; if we want backslashes
correctly escaped but not other special characters
SET spark.sql.parser.escapedStringLiterals=true; if we want other special
characters correctly escaped but not backslashes
> SQL LIKE does not handle backslashes correctly
> ----------------------------------------------
>
> Key: SPARK-17647
> URL: https://issues.apache.org/jira/browse/SPARK-17647
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: Xiangrui Meng
> Assignee: Xiangrui Meng
> Priority: Major
> Labels: correctness
> Fix For: 2.1.1, 2.2.0
>
>
> Try the following in SQL shell:
> {code}
> select '\\\\' like '%\\%';
> {code}
> It returned false, which is wrong.
> cc: [~yhuai] [~joshrosen]
> A false-negative considered previously:
> {code}
> select '\\\\' rlike '.*\\\\\\\\.*';
> {code}
> It returned true, which is correct if we assume that the pattern is treated
> as a Java string but not raw string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
