Jia Li created SPARK-22548: ------------------------------ Summary: Incorrect nested AND expression pushed down to JDBC data source Key: SPARK-22548 URL: https://issues.apache.org/jira/browse/SPARK-22548 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 2.2.0 Reporter: Jia Li
Let’s say I have a JDBC data source table ‘foobar’ with 3 rows: NAME THEID ================== fred 1 mary 2 joe 'foo' "bar" 3 This query returns incorrect result. SELECT * FROM foobar WHERE (THEID > 0 AND TRIM(NAME) = 'mary') OR (NAME = 'fred') It’s supposed to return: fred 1 mary 2 But it returns fred 1 mary 2 joe 'foo' "bar" 3 This is because one leg of the nested AND predicate, TRIM(NAME) = 'mary’, can not be pushed down but is lost during JDBC push down filter translation. The same translation method is also called by Data Source V2. I have a fix for this issue and will open a PR. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org