Re: HoD and locality of TaskTrackers to data (on DataNodes)

2008-03-24 Thread Jiaqi Tan
Hi Hemanth, More design questions I'm wondering about: So what determines the spread/location of data blocks that are uploaded/added to HDFS outside of the Map/Reduce framework? For instance, if I use a dfs -put to upload files to the HDFS, does the dfs system try to spread the blocks out across

HoD and locality of TaskTrackers to data (on DataNodes)

2008-03-23 Thread Jiaqi Tan
Hi, I have a question about using HoD and the locality of the assigned TaskTrackers to the data. Suppose I have a long-running HDFS installation with TaskTrackers/JobTracker nodes dynamically allocated by HoD, and I uploaded my data to HDFS prior to running my job/allocating nodes using dfs

Re: HoD and locality of TaskTrackers to data (on DataNodes)

2008-03-23 Thread Hemanth Yamijala
Jiaqi, Hi, I have a question about using HoD and the locality of the assigned TaskTrackers to the data. Suppose I have a long-running HDFS installation with TaskTrackers/JobTracker nodes dynamically allocated by HoD, and I uploaded my data to HDFS prior to running my job/allocating nodes using