Cheng Lian created SPARK-5777:
---------------------------------
Summary: Completes data source filter types and remove CatalystScan
Key: SPARK-5777
URL: https://issues.apache.org/jira/browse/SPARK-5777
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.2.1, 1.2.0, 1.3.0
Reporter: Cheng Lian
Assignee: Cheng Lian
Currently the data source API only supports a fraction of common filters, esp.
{{And}} is not supported yet. To workaround this issue and enable full filter
push-down optimization in the Parquet data source, {{CatalystScan}} was
introduced to receive full Catalyst filter expressions. This class should be
removed, since in principle, data source implementations shouldn't touch
Catalyst expressions (which are not part of the public developer API).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
