[
https://issues.apache.org/jira/browse/HADOOP-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12741993#action_12741993
]
Scott Carey commented on HADOOP-6166:
-------------------------------------
We probably want to have Todd's concurrency test from HADOOP-5318 run to make
sure the larger lookup table doesn't slow things down under concurrency.
We might also want to try the old four at a time code for 4 <= len < 8.
We should also confirm the results on one of the other systems we tested in the
past. I won't be able to do that for a couple days, but it should be easy then.
How many other variants did you try? Intel's C code does some strange things to
group CRC's by 3. Maybe something like the below is worth testing if it hasn't
already been done haven't already:
{code}+public class Crc32_8_8e extends Crc32Base {
public void update(byte[] b, int off, int len) {
while(len > 7) {
int c0 = b[off++] ^ crc;
int c1 = b[off++] ^ (crc >>>= 8);
int c2 = b[off++] ^ (crc >>>= 8);
int c3 = b[off++] ^ (crc >>>= 8);
crc = T8_7[c0 & 0xff] ^ T8_6[c1 & 0xff] ^ T8_5[c2 & 0xff] ^ T8_4[c3 &
0xff]; // three xors
crc ^= T8_3[b[off++] & 0xff] ^ T8_2[b[off++] & 0xff]; // two xors
crc ^= T8_1[b[off++] & 0xff] ^ T8_0[b[off++] & 0xff]; // two xors
len -= 8;
}
while(len > 0) {
crc = (crc >>> 8) ^ T8_0[(crc ^ b[off++]) & 0xff];
len--;
}
}
}{code}
> Improve PureJavaCrc32
> ---------------------
>
> Key: HADOOP-6166
> URL: https://issues.apache.org/jira/browse/HADOOP-6166
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: c6166_20090722.patch, c6166_20090722_benchmark_32VM.txt,
> c6166_20090722_benchmark_64VM.txt, c6166_20090727.patch,
> c6166_20090728.patch, c6166_20090810.patch
>
>
> Got some ideas to improve CRC32 calculation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
