[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264452#comment-17264452 ]
Noah Kawasaki commented on SPARK-17647: --------------------------------------- I can also confirm that this issue is not fully resolved. Like what [~swiegleb] has shown, escape characters are not fully supported. I have tested Spark versions 2.1, 2.2, 2.3, 2.4, and 3.0 and they all experience the issue: {code:java} # These do not return the expected backslash SET spark.sql.parser.escapedStringLiterals=false; SELECT '\\'; > \ (should return \\) SELECT 'hi\hi'; > hihi (should return hi\hi) # These are correctly escaped SELECT '\"'; > " SELECT '\''; > '{code} If I switch this: {code:java} # These now work SET spark.sql.parser.escapedStringLiterals=true; SELECT '\\'; > \\ SELECT 'hi\hi'; > hi\hi # These are now not correctly escaped SELECT '\"'; > \" (should return ") SELECT '\''; > \' (should return ' ){code} So basically we have to choose: SET spark.sql.parser.escapedStringLiterals=false; if we want backslashes correctly escaped but not other special characters SET spark.sql.parser.escapedStringLiterals=true; if we want other special characters correctly escaped but not backslashes > SQL LIKE does not handle backslashes correctly > ---------------------------------------------- > > Key: SPARK-17647 > URL: https://issues.apache.org/jira/browse/SPARK-17647 > Project: Spark > Issue Type: Bug > Components: SQL > Reporter: Xiangrui Meng > Assignee: Xiangrui Meng > Priority: Major > Labels: correctness > Fix For: 2.1.1, 2.2.0 > > > Try the following in SQL shell: > {code} > select '\\\\' like '%\\%'; > {code} > It returned false, which is wrong. > cc: [~yhuai] [~joshrosen] > A false-negative considered previously: > {code} > select '\\\\' rlike '.*\\\\\\\\.*'; > {code} > It returned true, which is correct if we assume that the pattern is treated > as a Java string but not raw string. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org