[GitHub] [arrow-datafusion] yjshen edited a comment on pull request #1596: Consolidate sort and external_sort, consolidate N-way merge sort

Tue, 18 Jan 2022 02:18:20 -0800 


yjshen edited a comment on pull request #1596:
URL: 
https://github.com/apache/arrow-datafusion/pull/1596#issuecomment-1015265203


   After a quick investigation with flamegraph:
   
   Flamegraph with this PR:
   
![flamegraph_svg](https://user-images.githubusercontent.com/1387718/149917828-b4a6618b-bc57-4ecc-bd9a-e24422e4e1fd.png)
   
   more than 70% of the time is spent on `SortKeyCursor::compare`, which 
deteriorates to item-wise comparison, compared to Arrow sort kernel which is 
compared on Array.
   
   only nearly 40% of the time is sort related:
   
![flamegraph_svg](https://user-images.githubusercontent.com/1387718/149916388-f8a36dbf-13df-4e92-9c24-2dffce2c0c1e.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to