More drives will certainly help in lots of ways but no new drive should fail within a week. I'd assume you have either power or heating issues.
Dejan On Mon, Jul 20, 2009 at 10:44 PM, Tim Nelson < [email protected]> wrote: > I have a question that I think I already know the answer to but I would > like verification. I have a demo cluster comprised of two master nodes and > eight slaves (all 1x1.2 Ghz cpu / 1 Gig Ram / 1x250 Gig Sata 7200 rpm hard > drives). I'm running small MR processes, about 100-200Gigs of total data > that take about 1-2 hours to process. These small processes seem to work > fine. However, I'm starting to run larger processes on them (5-8 hour > processes with 200-300Gigs of data) and the hard drives keep dieing. I know > I'm not running out of space, the hard drives really are crashing under the > load. I don't think it's over heating because the server room temp is a > constant 68-72 degrees. I'm running under the default configuration, 2 maps > + 2 reduces per node. I suspect that since I only have one hard drive per > node it is almost continuously writing to at least four different files (on > the same HD) and its just thrashing the read/write head and the motor. I > assumed my first batch of HD's were just bad but I've just had 4 more brand > new drives fail within a week so I think I'm pushing it too hard. > > If this is the problem, do I need to be sure and place my dfs data and > mapred data directories onto separate physical drives? If I upgrade to some > dell servers with dual core and 4-5HD's per node will hadoop take care of > balancing the load between hard drives (dfs/mapred/scratch storage)? > > Regards, > Tim Nelson >
