[jira] [Assigned] (SPARK-32721) Simplify if clauses with null and boolean
[ https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-32721: --- Assignee: Chao Sun > Simplify if clauses with null and boolean > - > > Key: SPARK-32721 > URL: https://issues.apache.org/jira/browse/SPARK-32721 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.0 >Reporter: Chao Sun >Assignee: Chao Sun >Priority: Major > > The following if clause: > {code:sql} > if(p, null, false) > {code} > can be simplified to: > {code:sql} > and(p, null) > {code} > And similarly, the following clause: > {code:sql} > if(p, null, true) > {code} > can be simplified to: > {code:sql} > or(not(p), null) > {code} > iff predicate {{p}} is deterministic, i.e., can be evaluated to either true > or false, but not null. > {{and}} and {{or}} clauses are more optimization friendly. For instance, by > converting {{if(col > 42, null, false)}} to {{and(col > 42, null)}}, we can > potentially push the filter {{col > 42}} down to data sources to avoid > unnecessary IO. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-32721) Simplify if clauses with null and boolean
[ https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32721: Assignee: Apache Spark > Simplify if clauses with null and boolean > - > > Key: SPARK-32721 > URL: https://issues.apache.org/jira/browse/SPARK-32721 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.0 >Reporter: Chao Sun >Assignee: Apache Spark >Priority: Major > > The following if clause: > {code:sql} > if(p, null, false) > {code} > can be simplified to: > {code:sql} > and(p, null) > {code} > And similarly, the following clause: > {code:sql} > if(p, null, true) > {code} > can be simplified to: > {code:sql} > or(p, null) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-32721) Simplify if clauses with null and boolean
[ https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32721: Assignee: (was: Apache Spark) > Simplify if clauses with null and boolean > - > > Key: SPARK-32721 > URL: https://issues.apache.org/jira/browse/SPARK-32721 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.0 >Reporter: Chao Sun >Priority: Major > > The following if clause: > {code:sql} > if(p, null, false) > {code} > can be simplified to: > {code:sql} > and(p, null) > {code} > And similarly, the following clause: > {code:sql} > if(p, null, true) > {code} > can be simplified to: > {code:sql} > or(p, null) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org