Sylvain Zimmer created SPARK-16807:
--------------------------------------
Summary: Optimize some ABS() statements
Key: SPARK-16807
URL: https://issues.apache.org/jira/browse/SPARK-16807
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Sylvain Zimmer
I'm not a Catalyst expert, but I think some use cases for the ABS() function
could generate simpler code.
This is the code generated when doing something like {{ABS(x - y) > 0}} or
{{ABS(x - y) = 0}} in Spark SQL:
{code}
/* 267 */ float filter_value6 = -1.0f;
/* 268 */ filter_value6 = agg_value27 - agg_value32;
/* 269 */ float filter_value5 = -1.0f;
/* 270 */ filter_value5 = (float)(java.lang.Math.abs(filter_value6));
/* 271 */
/* 272 */ boolean filter_value4 = false;
/* 273 */ filter_value4 =
org.apache.spark.util.Utils.nanSafeCompareFloats(filter_value5, 0.0f) > 0;
/* 274 */ if (!filter_value4) continue;
{code}
Maybe it could all be simplified to something like this?
{code}
filter_value4 = (agg_value27 != agg_value32)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
