[
https://issues.apache.org/jira/browse/MAPREDUCE-6451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940334#comment-14940334
]
Eric Payne commented on MAPREDUCE-6451:
---------------------------------------
[~kshukla], Thanks for providing this fix! It looks good in general, but I have
a few suggestions.
For the checkstyle warnings, please document the ones you will not be fixing
and the reason why. None of them are much of a problem, but I personally would
like to see the following fixes (both are in {{DynamicInputFormat.java}}):
- Please put the left parenthesis on the previous line
{code}
+ public DynamicInputChunkContext<K, V> getChunkContext
+ (Configuration configuration) throws IOException{
{code}
- I know it's just whitespace, but it does look a little awkward, so if you
could, please change the indentation:
{code}
+ DistCpUtils.getFileSize(chunkFilePath,
+ chunkContext.getConfiguration()), null), taskAttemptContext);
}
{code}
In {{TestDynamicInputFormat.java}}:
- I like the assertions to include a string with an error message that is
provided when the assertion fails. I recognize that the other assertions in
this file don't use that format, but I think it helps when running the tests.
So, for example, something like this:
{code}
+ Assert.assertTrue("Contexts from different DynamicInputChunkContext
objects should be different.", !firstContext.equals(thirdContext));
{code}
- I didn't find any unit tests for the original functionality that got moved
from {{DynamicInputChunk}} to {{DynamicInputChunkContext}}. If they don't
exist, can you please open a separate JIRA to cover that?
> DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-6451
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6451
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 2.6.0
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: MAPREDUCE-6451-v1.patch, MAPREDUCE-6451-v2.patch,
> MAPREDUCE-6451-v3.patch
>
>
> DistCp when used with dynamic strategy does not update the chunkFilePath and
> other static variables any time other than for the first job. This is seen
> when DistCp::run() is used.
> A single copy succeeds but multiple jobs finish successfully without any real
> copying.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
