Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13758
@cloud-fan , thank you for your comment. About conversion between primitive
array and unsafe array, I think we are on the same page. The motivation of my
recent PRs is to reduce overhead to change an array format in a generated code
from Dataset program using an primitive array.
Based on this PR and [another
PR](https://github.com/apache/spark/pull/13680), I am preparing another PR to
directly transfer primitive array to unsafe (dense) array by
'''Platform.copy()'''. Currently, while an array for any type is abstracted by
```GenericArrayData```, this PR just wraps an primitive array to alleviate
overhead of data format conversion (e.g. among Java primitive and unsafe dense).
Of course, it would be good to prepare APIs in ```GenericArrayData``` to
convert array format between primitive array and unsafe array.
Thank you for pointing a proposal for a new unsafe format. Let us discuss
it in [another PR](https://github.com/apache/spark/pull/13680).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
