Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13758
@cloud-fan , thank you for your comments. If we could do, it would be best.
We could easily do it for DataFrame program that accesses an array with
```Expression``` by keeping ```UnsafeArrayData```.
I think that it is not easy for DataSet program (e.g.
https://github.com/apache/spark/pull/13704) that accesses an array with Java
bytecode. Java bytecode requires Java array object instead of
```UnsafeArrayData```. For now, at least one conversion is required.
I know there is [an
activity](https://issues.apache.org/jira/browse/SPARK-14083) to convert Java
bytecode in Dataset program to ```Expression```. If we could do translate all
of the Java bytecode to `Expression```, it would be best. IIUC, current
```Expression``` has some limitations such as no representation for loop and so
one.
What do you think about each of Dataframe cases?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
