Eric Yang created SPARK-61408:
---------------------------------
Summary: MySQL JDBC filter pushdown returns wrong results for
string values containing the yen sign (¥) with Shift-JIS connection encodings
Key: SPARK-61408
URL: https://issues.apache.org/jira/browse/SPARK-61408
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Eric Yang
When a MySQL JDBC connection uses a Shift-JIS-family encoding
(characterEncoding=SJIS, MS932 or EUC_JP), a pushed-down filter on a string
value containing ¥ (U+00A5) can silently return wrong rows, fail with a MySQL
syntax error, or allow SQL injection. The same filters return correct results
with pushDownPredicate=false or characterEncoding=UTF-8.
*Repro* (MySQL 9.6, yen(s VARCHAR(20)) CHARACTER SET utf8mb4 with rows '100',
'¥100', 'a', 'b'):
{code:scala}
val df = spark.read.jdbc(url + "&characterEncoding=SJIS", "yen", new Properties)
df.filter(col("s") === "¥100").collect() // [100]; expected [¥100]
df.filter(col("s") === "a¥").collect() // fails: You have an error
in your SQL syntax
df.filter(col("s") === "x¥' OR 1=1) #").collect() // all 4 rows; expected none
{code}
Follow-up to SPARK-57500, which fixes the same symptoms for
backslash-containing values on encodings where byte 0x5C is a backslash.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]