[
https://issues.apache.org/jira/browse/SPARK-15795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317707#comment-15317707
]
Apache Spark commented on SPARK-15795:
--------------------------------------
User 'inouehrs' has created a pull request for this issue:
https://github.com/apache/spark/pull/13539
> Enable more optimizations in whole stage codegen when isNull is a
> compile-time constant
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-15795
> URL: https://issues.apache.org/jira/browse/SPARK-15795
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Hiroshi Inoue
>
> Whole stage codegen often creates {{isNull}} variable initialized with
> constant _false_, like
> {{boolean mapelements_isNull = false || false;}}
> If there is no further assignment for this {{isNull}} variable, whole stage
> codegen can do more optimizations by assuming {{isNull}} as a compile-time
> constant.
> In the example below, which is generated for a dataset map operation,
> {{mapelements_isNull}} defined at line 115 can be assumed by a compile-time
> constant (false).
> By assuming this as a constant, the whole stage codegen eliminates
> {{zeroOutNullBytes}} at line 119 and an if-statement at line 121.
> In addition to the benefits of improved readability of generated code,
> eliminating {{zeroOutNullBytes}} will give performance advantage since it is
> difficult to remove for Java JIT compiler.
> {code}
> /* 107 */ // CONSUME: Project [id#0L AS l#3L]
> /* 108 */ // CONSUME: DeserializeToObject l#3: bigint, obj#16: bigint
> /* 109 */ // CONSUME: MapElements <function1>, obj#17: bigint
> /* 110 */ // CONSUME: SerializeFromObject [input[0, bigint, true] AS
> value#18L]
> /* 111 */ // <function1>.apply
> /* 112 */ Object mapelements_obj = ((Expression)
> references[1]).eval(null);
> /* 113 */ scala.Function1 mapelements_value1 = (scala.Function1)
> mapelements_obj;
> /* 114 */
> /* 115 */ boolean mapelements_isNull = false || false;
> /* 116 */ final long mapelements_value = mapelements_isNull ? -1L :
> (Long) mapelements_value1.apply(range_value);
> /* 117 */
> /* 118 */ // CONSUME: WholeStageCodegen
> /* 119 */ serializefromobject_rowWriter.zeroOutNullBytes();
> /* 120 */
> /* 121 */ if (mapelements_isNull) {
> /* 122 */ serializefromobject_rowWriter.setNullAt(0);
> /* 123 */ } else {
> /* 124 */ serializefromobject_rowWriter.write(0, mapelements_value);
> /* 125 */ }
> /* 126 */ append(serializefromobject_result);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
