[
https://issues.apache.org/jira/browse/MAPREDUCE-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Ramachandran updated MAPREDUCE-2020:
--------------------------------------------
Attachment: mapred-2020-7.patch
Updated patch with more components using FileContext APIs
Changes since last rev (so only need to focus on these)
Updated src/java/org/apache/hadoop/mapred/JobTracker.java (due to trunk
changes)
filecache
---------
src/java/org/apache/hadoop/mapreduce/filecache/TaskDistributedCache.java
src/java/org/apache/hadoop/mapreduce/filecache/DistributedCache.java
src/java/org/apache/hadoop/mapreduce/filecache/TrackerDistributedCacheManager.java
Job Client components
--------------------------
src/java/org/apache/hadoop/mapreduce/Cluster.java
src/java/org/apache/hadoop/mapreduce/Job.java
src/java/org/apache/hadoop/mapreduce/JobSubmitter.java
Others
----------
src/java/org/apache/hadoop/mapred/JobInProgress.java
src/java/org/apache/hadoop/mapred/LocalJobRunner.java
src/java/org/apache/hadoop/mapred/lib/FilterOutputFormat.java
src/java/org/apache/hadoop/mapred/lib/NullOutputFormat.java
src/java/org/apache/hadoop/mapred/lib/db/DBOutputFormat.java
src/java/org/apache/hadoop/mapred/FileOutputFormat.java
All public APIs with dependence on FileSystem are left alone for compatibility
(tools like oozie or pig may use these)
Most private references/APIs are migrated
Though the migration was straight forward some inconsistent usage caused
problems
Example,
path.getFileSystem(conf) (no equivalent API and DFS team's recommendation is
not to use this)
getting RawFileSystem FileSystem.getRaw() - workaround
copyTo/FromLocal - use a generalize copy() from FileContext.Util class which
requires fully qualified path
fs.getUri() - not available via context (workaround)
FileSystem.closeAllforUGI() - not available yet
getFileSystemName() - is a public API in JT why this is needed
> Use new FileContext APIs for all mapreduce components
> ------------------------------------------------------
>
> Key: MAPREDUCE-2020
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2020
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 0.22.0
> Reporter: Krishna Ramachandran
> Assignee: Krishna Ramachandran
> Attachments: mapred-2020-1.patch, mapred-2020-4.patch,
> mapred-2020-5.patch, mapred-2020-6.patch, mapred-2020-7.patch,
> mapred-2020.patch
>
>
> Migrate mapreduce components to using improved FileContext APIs implemented in
> HADOOP-4952 and
> HADOOP-6223
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
