[
https://issues.apache.org/jira/browse/MAPREDUCE-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
WuZeyi updated MAPREDUCE-7223:
------------------------------
Docs Text:
was:
Quick-sort performance: The time of quick-sort using unsafe is 16515s,and using
byteBuffer is 21643s.The performance can improve by 31%
unsafe(s) byteBuffer(s) byteBuffer/unsafe
AVG 16515 21643 1.310481735
> Quicksort GetInt performance Issue in Terasort
> ----------------------------------------------
>
> Key: MAPREDUCE-7223
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7223
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 2.7.6, 3.1.2
> Reporter: WuZeyi
> Priority: Major
> Labels: patch, performance
> Fix For: 2.7.6
>
>
> I find a hot spot of 'java.nio.Bits.getIntL' in the Terasort case of Hadoop.
> It is done by shifting four bytes in the byte array each time to get an int.
> This 'getIntL' operation is repeatedly called in the quick-sort of KVbuffer
> which has a complexity of NlogN, and it causes the hot spot.
> The element that is gotten in the quick-sort may be gotten frequently, which
> means it has to be shifted again and again.
> After replacing 'java.nio.Bits.getIntL' with 'unsafe.getInt', the performance
> of quick-sort can be improved by 30%。Terasort can be improved by 10%
> !terasort.png!!makeint.png!
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
