Github user mpjlu commented on the issue:
https://github.com/apache/spark/pull/18620
Hi @MLnick , @srowen .
My test showing: pq.poll is not significantly faster than
pq.toArray.sortBy, but significantly faster than pq.toArray.sorted. Seems not
each pq.toArray.sorted (such as used in topByKey) can be replaced by
pq.toArray.sortBy, so use pq.poll to replace pq.toArray.sorted will benefit.
You can compare the performance of pq.sorted, pq.sortBy, and pq.poll using:
https://github.com/apache/spark/pull/18624
The performance of pq.toArray.sortBy is about the same as pq.poll, and
about 20% improvement comparing pq.toArray.sorted.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
