Hiroshi Inoue created SPARK-15795:
-------------------------------------
Summary: Enable more optimizations in whole stage codegen when
isNull is a compile-time constant
Key: SPARK-15795
URL: https://issues.apache.org/jira/browse/SPARK-15795
Project: Spark
Issue Type: Improvement
Components: SQL
Reporter: Hiroshi Inoue
Whole stage codegen often creates {{isNull}} variable initialized with constant
_false_, like
{{boolean mapelements_isNull = false || false;}}
If there is no further assignment for this {{isNull}} variable, whole stage
codegen can do more optimizations by assuming {{isNull}} as a compile-time
constant.
In the example below, which is generated for a dataset map operation,
{{mapelements_isNull}} defined at line 115 can be assumed by a compile-time
constant (false).
By assuming this as a constant, the whole stage codegen eliminates
{{zeroOutNullBytes}} at line 119 and an if-statement at line 121.
In addition to the benefits of improved readability of generated code,
eliminating {{zeroOutNullBytes}} will give performance advantage since it is
difficult to remove for Java JIT compiler.
{code}
/* 107 */ // CONSUME: Project [id#0L AS l#3L]
/* 108 */ // CONSUME: DeserializeToObject l#3: bigint, obj#16: bigint
/* 109 */ // CONSUME: MapElements <function1>, obj#17: bigint
/* 110 */ // CONSUME: SerializeFromObject [input[0, bigint, true] AS
value#18L]
/* 111 */ // <function1>.apply
/* 112 */ Object mapelements_obj = ((Expression)
references[1]).eval(null);
/* 113 */ scala.Function1 mapelements_value1 = (scala.Function1)
mapelements_obj;
/* 114 */
/* 115 */ boolean mapelements_isNull = false || false;
/* 116 */ final long mapelements_value = mapelements_isNull ? -1L :
(Long) mapelements_value1.apply(range_value);
/* 117 */
/* 118 */ // CONSUME: WholeStageCodegen
/* 119 */ serializefromobject_rowWriter.zeroOutNullBytes();
/* 120 */
/* 121 */ if (mapelements_isNull) {
/* 122 */ serializefromobject_rowWriter.setNullAt(0);
/* 123 */ } else {
/* 124 */ serializefromobject_rowWriter.write(0, mapelements_value);
/* 125 */ }
/* 126 */ append(serializefromobject_result);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
