[ https://issues.apache.org/jira/browse/SPARK-30795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon reassigned SPARK-30795: ------------------------------------ Assignee: Kris Mok > Spark SQL codegen's code() interpolator should treat escapes like Scala's > StringContext.s() > ------------------------------------------------------------------------------------------- > > Key: SPARK-30795 > URL: https://issues.apache.org/jira/browse/SPARK-30795 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.4.5, 3.0.0 > Reporter: Kris Mok > Assignee: Kris Mok > Priority: Major > Fix For: 3.1.0 > > > The {{code()}} string interpolator in Spark SQL's code generator should treat > escapes like Scala's builtin {{StringContext.s()}} interpolator, i.e. it > should treat escapes in the code parts, and should not treat escapes in the > input arguments. > For example, > {code} > val arg = "This is an argument." > val str = s"This is string part 1. $arg This is string part 2." > val code = code"This is string part 1. $arg This is string part 2." > assert(code.toString == str) > {code} > We should expect the {{code()}} interpolator produce the same thing as the > {{StringContext.s()}} interpolator, where only escapes in the string parts > should be treated, while the args should be kept verbatim. > But in the current implementation, due to the eager folding of code parts and > literal input args, the escape treatment is incorrectly done on both code > parts and literal args. > That causes a problem when an arg contains escape sequences and wants to > preserve that in the final produced code string. For example, in {{Like}} > expression's codegen, there's an ugly workaround for this bug: > {code} > // We need double escape to avoid > org.codehaus.commons.compiler.CompileException. > // '\\' will cause exception 'Single quote must be backslash-escaped in > character literal'. > // '\"' will cause exception 'Line break in literal not allowed'. > val newEscapeChar = if (escapeChar == '\"' || escapeChar == '\\') { > s"""\\\\\\$escapeChar""" > } else { > escapeChar > } > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org