Hi Vincent- On Thu, Oct 06, 2011 at 11:19:20AM +0200, Vincent Boucher wrote: > We are wondering if it is possible to require that the namenode redirects the > blocks of the files of a given directory to a particular set of datanodes? > > Our case is the following: > > - Servers > 10 x mass storage servers of each 50TB, RAID6 > -> 500TB available for hdfs > 30 x working nodes with 6TB (no RAID) > -> 180TB available for hdfs > > We'd like that the files stored in > /hdfs/ms > be hosted on the mass storage (ms) servers, > while the files in > /hdfs/wn > be hosted on the working nodes (wn).
Why do you want to partition your datanodes this way? In our cluster, datanode size ranges from ~1 TB (no RAID) up to ~80TB (RAID). While in an ideal world all datanodes would be similar in configuration, we have not observed any issues with this arrangement in production. -- Will Maier - UW High Energy Physics cel: 608.438.6162 tel: 608.263.9692 web: http://www.hep.wisc.edu/~wcmaier/
smime.p7s
Description: S/MIME cryptographic signature
