[
https://issues.apache.org/jira/browse/MAPREDUCE-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907497#action_12907497
]
Greg Roelofs commented on MAPREDUCE-2020:
-----------------------------------------
Substantive:
* {{"fs.AbstractFileSystem.file.impl"}} probably should be new
{{JobContext.FOO}} style
** 3 instances (at least)
* Merger.java: {{merge()}} method madness: how many do we need? already had
7; now have 14... where does it end??
* MultiFileInputFormat.java: lose {{import org.apache.hadoop.fs.FileSystem}}:
not used; slows build, adds confusion
** probably ditto MultiFileSplit.java and TestMRAsyncDiskService.java
* MergeManager.java: _massive_ pile of duplicated constructor and
finalMerge() code: share! (may have previously allowed in case of "short-term
transition," but Hadoop API transitions are _not_ short-term => high risk of
mismatch-errors in future changes; should share code where possible even for
"temporary" cases, e.g., by calling private helper function from both copies)
Cosmetic:
* still adding trailing whitespace (IFile.java, Merger.java)
** if necessary, fire up vim on diff and search: /^+.* $
* still bad wraps, e.g.:
{noformat}
+ Deserializer<T> deserializer = (Deserializer<T>) factory
+ .getDeserializer(cls);
{noformat}
or lack of wrap:
{noformat}
+ job.set("fs.AbstractFileSystem.file.impl",
"org.apache.hadoop.fs.local.RawLocalFs");
{noformat}
** if line starts with a period, almost guaranteed to be wrong
* avoid superfluous "this." decorations (Merger.java, MergeManager.java):
{noformat}
+ this.mapOutputsCounter = mergedMapOutputsCounter;
+ this.localFC = null;
+ this.rfc = null;
{noformat}
> Use new FileContext APIs for all mapreduce components
> ------------------------------------------------------
>
> Key: MAPREDUCE-2020
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2020
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.22.0
> Reporter: Krishna Ramachandran
> Assignee: Krishna Ramachandran
> Attachments: mapred-2020-1.patch, mapred-2020-4.patch,
> mapred-2020-5.patch, mapred-2020.patch
>
>
> Migrate mapreduce components to using improved FileContext APIs implemented in
> HADOOP-4952 and
> HADOOP-6223
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
