gentuxx wrote:
The main purpose of this box is going to be log crunching and
archival. We have logs that range from tens of MBs to a GB a piece
(uncompressed). The scripts running on it will be transferring (over
the network), decompressing, grepping, normalizing, recompressing, and
inserting into a local database from dozens of sources
simultaneously. In a prior situation, I had a Sun e4500 with 8
UltraSPARC IIIs, 12GB of RAM, and about a TB of disk attached (SCSI
and FC). There were times it wasn't enough. I'm hoping to at least
match that functionality (preferably better it).
Single boxes don't scale. :-)
That's probably simplifying too much, but there is some truth there as
well. I'd seriously think about setting up a preprocessing farm of 1U
boxes that do most of the crunching and then doing inserts into a
smaller db box. I've even worked on systems where logs go directly into
a db, then are pulled and processed by another farm, then inserted into
more of a data warehouse system for reporting later.
You'll have to decide if the application can be broken up into separate
processing units or whatever. If you do go that route it's usually an
easy sell to management.
"We have one $15k box, but it's too small. We could buy one $30k box to
do everything and completely replace the $15k box that we haven't fully
depreciated yet or I can buy three $4k boxes to sit in front of our
existing server which will share the load. And next year when we start
to slow down again instead of buying an even bigger $60k box we just buy
three more $4k boxes. Can I have some programmer time to make some
architecture changes so I can save you around $65k over the next two years?"
The more comparisons and reviews I read are leaning me in that
direction. However, it doesn't look like HP offers a 4-way Opteron
box. I'll have to ask the vendor.
from hp.com the DL585's appear to be configurable for 4-way once you get
into their config tool.
kashani
--
gentoo-user@gentoo.org mailing list
