[
https://issues.apache.org/jira/browse/MAPREDUCE-6643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326554#comment-15326554
]
Akira AJISAKA edited comment on MAPREDUCE-6643 at 6/12/16 5:38 PM:
-------------------------------------------------------------------
Thanks [~lewuathe] for updating the patch! Mostly looks good to me. Some
comments:
{code}
if (start == 0) {
adjStart = Math.max(0L, start - FIRST_BZIP2_BLOCK_MARKER_POSITION);
{code}
1. It seems that adjStart is always zero. If that is true, we should set zero
instead of using max function.
{code}
// Other than the fist of file, the marker size is 6 bytes.
{code}
2. fist -> first
3. Would you fix the checkstyle warning?
{code}
// corner case when we have byte alignment and position of stream are same
int[] lengths = {471507, 473608};
int[] numSplits = {218, 110};
for (int i = 0; i < 2; i++) {
verifyPartitions(lengths[i], numSplits[i], file, codec, conf);
}
{code}
4. (nit) The following code is simpler for me:
{code}
// corner case when we have byte alignment and position of stream are same
verifyPartitions(471507, 218, file, codec, conf);
verifyPartitions(473608, 110, file, codec, conf);
{code}
was (Author: ajisakaa):
Thanks [~lewuathe] for updating the patch! Mostly looks good to me. Some
comments:
{code}
if (start == 0) {
adjStart = Math.max(0L, start - FIRST_BZIP2_BLOCK_MARKER_POSITION);
{code}
1. It seems that adjStart is always zero. If that is true, we should set zero
instead of using max function.
{code}
// Other than the fist of file, the marker size is 6 bytes.
{code}
2. fist -> first
3. Would you fix the checkstyle warning?
{code}
// corner case when we have byte alignment and position of stream are same
int[] lengths = {471507, 473608};
int[] numSplits = {218, 110};
for (int i = 0; i < 2; i++) {
verifyPartitions(lengths[i], numSplits[i], file, codec, conf);
}
{code}
4. (nit) The following code is simpler for me:
{code}
// corner case when we have byte alignment and position of stream are same
verifyPartition(471507, 218, file, codec, conf);
verifyPartition(473608, 110, file, codec, conf);
{code}
> org.apache.hadoop.mapred.TestTextInputFormat.testSplitableCodecs failed
> -----------------------------------------------------------------------
>
> Key: MAPREDUCE-6643
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6643
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Haibo Chen
> Assignee: Kai Sasaki
> Attachments: MAPREDUCE-6643.01.patch, MAPREDUCE-6643.02.patch,
> MAPREDUCE-6643.03.patch
>
>
> Unit test TestTextInputFormat.testSplitableCodecs() failed when the seed is
> 1313094493.
> Stacktrace
> java.lang.AssertionError: Key in multiple partitions.
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.hadoop.mapred.TestTextInputFormat.testSplitableCodecs(TestTextInputFormat.java:223)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
