Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/8484#issuecomment-135655783
Hi @viirya , thanks for looking into this!
The reason for this extra copy is that we iterate the array twice:
calculate elements data size and copy elements to array buffer.
A simple solution is to follow `createCodeForStruct`, we can dynamicly grow
the buffer when needed and thus don't need to know the data size ahead.
I have opened https://github.com/apache/spark/pull/8496 to fix it, do you
mind take a look?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
