Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
Regarding the plan tree printout (I removed my debug information), the
```Cast``` performs conversion from ```UnsafeArrayData``` to
```GenericArrayData``` since the target type of ```Cast``` is
```ArrayType<DoubleType>```. Since ```Cast``` shows ```dataType. simpleType```
of the target type, it is shown as ```array<double>```.
Regarding the generated code, we seems to be on the same page. What you
said is not done in ```Cast``` now, and what I did. IIUC, current ```Cast```
[code generation for
array](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L828)
seems to be conservative. Current goal of cast code generation is to create
```GenericArrayData``` object. This code generation always creates
```Object[]``` and assign values into each ```Object[]``` element. Then, the
generated code passes ```Object[]``` to the constructor of
```GenericArrayData```. As you pointed out, if code generation takes care of
nullability, it can avoid to create ```Object[]```. Unfortunately, it is not
done in [the current code]([code generation for
array](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L828)
(Lines 828-862).
The code generated by this PR can also use specialized
```GenericArrayData``` implemented by
https://github.com/apache/spark/pull/13758.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
