Can you be more specific about your benchmark? You're almost certainly being limited by either outgoing network from your single node, or by disk throughput on that node (writes will go local in addition to remote replicas if you are running a DN on the same host)
-Todd On Mon, Apr 12, 2010 at 7:21 AM, Gokulakannan M <gok...@huawei.com> wrote: > > > Hi, > > > > When I tested the performance of a 11 node hadoop > cluster(private nw) for the write scenario with several threads, I noticed > that the time for each write increases linearly(what ever the file size may > be). > > > > Actually I'm running these threads as a single process in the > system where namenode is running. I'm getting inconsistent results when I > run this suite each time. The results are fluctuating. What might be the > bottlenecks? > > > > I think running this in parallel by spawning multiple shells > will give consistent result. But running these in the same system will give > exact results?? > > > > *Any* *ways to measure the performance correctly?* > > * * > > * *Appreciated any help in this. > > > > Thanks, > > Gokul > > > > > -- Todd Lipcon Software Engineer, Cloudera
