How does Hadoop handle multi-core CPUs? Does each core run a distinct copy of the mapped app? Is this automatic, or need some configuration, or what?
I'm in the market to buy a few machines to set up a small cluster and am wondering what I should consider. Or should I just spread Hadoop over some friendly machines already in my College, buying nothing? - Bob Ted Dunning-3 wrote: > > > We have an oddball collection of machines. Most are in the class you > mention, although some have single dual core CPU's and some have 12GB of > memory. We plan to use developer workstations at night (real soon now) > which typically have 1-3GB of memory + single CPU. > > Our name node is very lightly used because the files we analyze are pretty > good sized (we produce only a few consolidated files per hour). > > > On 9/10/07 4:54 PM, "John Heidemann" <[EMAIL PROTECTED]> wrote: > >> >> What are reasonable hardware specifications for a Hadoop node? >> Can we document this somewhere (maybe in the wiki as >> HowToConfigureHardware?) >> >> Obviously this will be a moving target, but some guidance about how much >> CPU vs. memory vs. disk space is typical would be helpful. >> >> As one datapoint, we are running some boxes that are 4 core, 64-bit @ >> 2GHz machines with 4GB of memory with [I think] 2 x 750GB disks. I >> think if I could I'd put 4 x 750GB disks in this box. I believe this >> configuration is basically the same as what came up in Yahoo!'s recent >> sort benchmark. >> >> Other datapoints anyone? >> >> And what about, say on the namenode? People talk about it being a >> memory bottleneck, but ours is underutilized. >> >> Should we start a wiki page about this? >> >> -John Heidemann >> > > > -- View this message in context: http://www.nabble.com/hardware-specs-for-hadoop-nodes-tf4419439.html#a12883435 Sent from the Hadoop Users mailing list archive at Nabble.com.
