[
https://issues.apache.org/jira/browse/MAPREDUCE-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133214#comment-13133214
]
Chris Douglas commented on MAPREDUCE-3235:
------------------------------------------
bq. One optimization there would be to pack the PARTITION field more tightly -
in most MR jobs, we have <256 reducers, so partition could be a single byte.
Since we know the number of reducers up front, we could easily trade-off space
between the partition ID and the comparison proxy.
Right now, the overlay is all ints. The JVM could/should be smart enough to
handle that. If we start mixing types in metadata records, we'll end up writing
placement new and possibly preventing some optimizations. If (instead) you're
talking about shifting the partition and writing the prefix into the remaining
bytes, or even more generally, defining (partition,key) as the input to an
order-preserving hash function... yikes.
Though viewed askance, that's a decent API, since the default just returns the
partition and follows the existing path.
bq. Currently I added a marker interface with a single method: getPrefix(byte[]
dst, int off, int length). If the key type implements this interface,
getPrefix() is called by collect() to copy the comparison proxy into the kvmeta
buffer. I was thinking last night that it would be better to delegate to the
Serializer implementation there, though.
Agreed; attaching it to the keytype prevents a user from doing obvious things
like reversing the order without subclassing the type. An optional component (a
role comparable to the combiner, I guess) that *may* be used by the framework
to generate a hash that effects the same ordering could be useful. Would it
make sense for this to come from the comparator, rather than the keytype?
Mapping the keytype to a fixed-length record, even forcing an integer, will
probably cover most cases.
bq. I imagine most of the "stock writables" we have in Hadoop could easily
implement this
If this came from the RawComparator attached to most Writables, one would have
to work out how to manage subclasses, or incompatibly make all the Comparators
for those types final (which is probably correct, anyway).
> Improve CPU cache behavior in map side sort
> -------------------------------------------
>
> Key: MAPREDUCE-3235
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3235
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: performance, task
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: mr-3235-poc.txt
>
>
> When running oprofile on a terasort workload, I noticed that a large amount
> of CPU usage was going to MapTask$MapOutputBuffer.compare. Upon disassembling
> this and looking at cycle counters, most of the cycles were going to memory
> loads dereferencing into the array of key-value data -- implying expensive
> cache misses. This can be avoided as follows:
> - rather than simply swapping indexes into the kv array, swap the entire meta
> entries in the meta array. Swapping 16 bytes is only negligibly slower than
> swapping 4 bytes. This requires adding the value-length into the meta array,
> since we used to rely on the previous-in-the-array meta entry to determine
> this. So we replace INDEX with VALUELEN and avoid one layer of indirection.
> - introduce an interface which allows key types to provide a 4-byte
> comparison proxy. For string keys, this can simply be the first 4 bytes of
> the string. The idea is that, if stringCompare(key1.proxy(), key2.proxy()) !=
> 0, then compare(key1, key2) should have the same result. If the proxies are
> equal, the normal comparison method is used. We then include the 4-byte proxy
> as part of the metadata entry, so that for many cases the indirection into
> the data buffer can be avoided.
> On a terasort benchmark, these optimizations plus an optimization to
> WritableComparator.compareBytes dropped the aggregate mapside CPU millis by
> 40%, and the compare() routine mostly dropped off the oprofile results.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
