[
https://issues.apache.org/jira/browse/HADOOP-1296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492086
]
Konstantin Shvachko commented on HADOOP-1296:
---------------------------------------------
There is a lot of similarities with HADOOP-894. I think these two should be
treated jointly.
getFileCacheHints() performs any meaningful actions only in case of HDFS. In
the case it can
just call getBlockLocations() for the file and convert the results in a file
system independent form.
BlockInfo is not a good name for that form since FSNamesystem already has a
private
class called BlockInfo. Besides, LocalFileSystem or Memory or S3 don't have
blocks.
Asking for all file blocks can be done by setting start=0 and len=Long.MAX_VALUE
> Improve interface to FileSystem.getFileCacheHints
> -------------------------------------------------
>
> Key: HADOOP-1296
> URL: https://issues.apache.org/jira/browse/HADOOP-1296
> Project: Hadoop
> Issue Type: Improvement
> Components: fs
> Reporter: Owen O'Malley
> Assigned To: dhruba borthakur
>
> The FileSystem interface provides a very limited interface for finding the
> location of the data. The current method looks like:
> String[][] getFileCacheHints(Path file, long start, long len) throws
> IOException
> which returns a list of "block info" where the block info consists of a list
> host names. Because the hints don't include the information about where the
> block boundaries are, map/reduce is required to call the name node for each
> split. I'd propose that we fix the naming a bit and make it:
> public class BlockInfo extends Writable {
> public long getStart();
> public String[] getHosts();
> }
> BlockInfo[] getFileHints(Path file, long start, long len) throws IOException;
> So that map/reduce can query about the entire file and get the locations in a
> single call.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
