[
https://issues.apache.org/jira/browse/TEZ-1593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated TEZ-1593:
-------------------------
Description:
The current implementation of PipelinedSorter has a slow section which revolves
around key comparisons - this was relevant when the implementation used direct
byte buffers to back the kvbuffer.
{code}
kvbuffer.position(istart);
kvbuffer.get(ki, 0, ilen);
kvbuffer.position(jstart);
kvbuffer.get(kj, 0, jlen);
// sort by key
final int cmp = comparator.compare(ki, 0, ilen, kj, 0, jlen);
{code}
The kvbuffer.get into the arrays ki and kj are the slowest part of the
comparator operation.
was:
The current implementation of PipelinedSorter has a slow section which revolves
around key comparisons.
{code}
kvbuffer.position(istart);
kvbuffer.get(ki, 0, ilen);
kvbuffer.position(jstart);
kvbuffer.get(kj, 0, jlen);
// sort by key
final int cmp = comparator.compare(ki, 0, ilen, kj, 0, jlen);
{code}
The kvbuffer.get into the arrays ki and kj are the slowest part of the
comparator operation.
> Refactor PipelinedSorter to remove all MMAP based ByteBuffer references
> -----------------------------------------------------------------------
>
> Key: TEZ-1593
> URL: https://issues.apache.org/jira/browse/TEZ-1593
> Project: Apache Tez
> Issue Type: Bug
> Affects Versions: 0.6.0
> Reporter: Gopal V
> Assignee: Gopal V
> Labels: Performance
>
> The current implementation of PipelinedSorter has a slow section which
> revolves around key comparisons - this was relevant when the implementation
> used direct byte buffers to back the kvbuffer.
> {code}
> kvbuffer.position(istart);
> kvbuffer.get(ki, 0, ilen);
> kvbuffer.position(jstart);
> kvbuffer.get(kj, 0, jlen);
> // sort by key
> final int cmp = comparator.compare(ki, 0, ilen, kj, 0, jlen);
> {code}
> The kvbuffer.get into the arrays ki and kj are the slowest part of the
> comparator operation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
