Github user nburoojy commented on the pull request:
https://github.com/apache/spark/pull/8592#issuecomment-140238919
@rxin I bet you're right about the performance and gc issue. Is there a
performance test suite to quantify this issue? I always hesitate to optimize
prematurely :-)
> rewrite the query under the hood to implement it using an operator that
gives a list of items
Are you suggesting to create a `List` equivalent of `OpenHashSetUDT`?
Mutating a single buffer on `updateExpressions` and `mergeExpressions` would
certainly cut down on the number of Object instantiations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
