[GitHub] [arrow-datafusion] yjshen commented on pull request #1596: Consolidate sort and external_sort, consolidate N-way merge sort

Tue, 18 Jan 2022 02:17:26 -0800 


yjshen commented on pull request #1596:
URL: 
https://github.com/apache/arrow-datafusion/pull/1596#issuecomment-1015265203


   Flamegraph with this PR:
   
![flamegraph_svg](https://user-images.githubusercontent.com/1387718/149917828-b4a6618b-bc57-4ecc-bd9a-e24422e4e1fd.png)
   
   
   After a quick investigation with flamegraph, nearly 70% of the time is spent 
on `SortKeyCursor::compare`, which deteriorate to item wise comparison, 
compared to arrow sort kernel which is compared on Array.
   
   only nearly 40% of the time is sort related:
   
![flamegraph_svg](https://user-images.githubusercontent.com/1387718/149916388-f8a36dbf-13df-4e92-9c24-2dffce2c0c1e.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to