Unless you are getting all local IO, and or you have better than GigE nic interfaces 100MB/sec is your cap.
For local IO the bound is going to be your storage subsystem. Decent drives in a raid 0 interface are going to cap out on those machines about 400MB/sec, which is the buffer cache bandwidth on those processors/memory. realistically you are going to see a quite a bit less, but 200 should be doable. On Wed, Mar 31, 2010 at 2:55 PM, Sagar Naik <[email protected]> wrote: > Hi Edson, > > usual commodity machine : > 8GB Ram, 2.5GHZ Intel Xeon, > > 6 Disks : 2 drives RAID 1 , 4 RAID 0 using PERC 6 > > Centos , ext4 > > Datanodes configured to use 4 RAID 0 drives > OS and hadoop installation on RAID 1 > > -Sagar > On Mar 30, 2010, at 4:21 PM, Edson Ramiro wrote: > >> Hi Sagar, >> >> What hardware did you run it on ? >> >> Edson Ramiro >> >> >> On 30 March 2010 19:41, sagar naik <[email protected]> wrote: >> >>> Hi All, >>> >>> I am trying to get DFS IO performance. >>> I used TestDFSIO from hadoop jars. >>> The results were abt 100Mbps read and write . >>> I think it should be more than this >>> >>> Pl share some stats to compare >>> >>> Either I am missing something like config params or something else >>> >>> >>> -Sagar >>> > > -- Pro Hadoop, a book to guide you from beginner to hadoop mastery, http://www.amazon.com/dp/1430219424?tag=jewlerymall www.prohadoopbook.com a community for Hadoop Professionals
