[
https://issues.apache.org/jira/browse/SPARK-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Armbrust resolved SPARK-3109.
-------------------------------------
Resolution: Won't Fix
Hey Alex, I'm going to close this as I'm not sure its actually possible to do
this optimization in the general case. If I'm wrong or if you have a more
specific instance where you think this optimization could work please feel free
to reopen.
> Sql query with OR condition should be handled above PhysicalOperation layer
> ---------------------------------------------------------------------------
>
> Key: SPARK-3109
> URL: https://issues.apache.org/jira/browse/SPARK-3109
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.0.2
> Reporter: Alex Liu
>
> For query like
> {code}
> select d, e from test where a = 1 and b = 1 and c = 1 and d > 20 or d < 0
> {code}
> Spark SQL pushes the whole query to PhysicalOperation. I haven't check how
> Spark SQL internal query plan works, but I think "OR" condition in the above
> query should be handled above physical operation. Physical operation should
> have the following query
> {code} select d, e from test where a = 1 and b = 1 and c = 1 and d > 20
> {code}
> OR
> {code}select d, e from test where d < 0 {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
