[
https://issues.apache.org/jira/browse/SPARK-12126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-12126:
---------------------------------
Description:
As suggested
[here|https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=14955646&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14955646],
Currently JDBC datasource only processes the filters pushed down from
{{DataSourceStrategy}}.
Unlike ORC or Parquet, this can process pretty a lot of filters (for example, a
+ b > 3) since it is just about string parsing.
As
[here|https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=15031526&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15031526],
using {{CatalystScan}} trait might be one of solutions.
was:
As suggested
[here](https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=14955646&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14955646),
Currently JDBC datasource only processes the filters pushed down from
{{DataSourceStrategy}}.
Unlike ORC or Parquet, this can process pretty a lot of filters (for example, a
+ b > 3) since it is just about string parsing.
As
[here](https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=15031526&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15031526),
using {{CatalystScan} trait might be one of solutions.
> JDBC datasource processes filters only commonly pushed down.
> ------------------------------------------------------------
>
> Key: SPARK-12126
> URL: https://issues.apache.org/jira/browse/SPARK-12126
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Hyukjin Kwon
>
> As suggested
> [here|https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=14955646&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14955646],
> Currently JDBC datasource only processes the filters pushed down from
> {{DataSourceStrategy}}.
> Unlike ORC or Parquet, this can process pretty a lot of filters (for example,
> a + b > 3) since it is just about string parsing.
> As
> [here|https://issues.apache.org/jira/browse/SPARK-9182?focusedCommentId=15031526&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15031526],
> using {{CatalystScan}} trait might be one of solutions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
