On Wed, 19 Jan 2011, Frank DiMitri wrote:
I am about to order the parts for the next machine, also to run xen,
an opteron x6 2.6GHZ 4180 with the supermicro motherboard.
$210 cpu, $283 mbd, and a cooler master case for $80 or $90.
The case can have up to 8 sata drives, e.g. 8 x 2TB = 16TB.
Why not a G34/8 core system? The CPU is barely any more expensive (6128),
should be a little faster for heavily threaded stuff (ie multiple VMs,
over-assigning VMs). It's only 2.0GHz x 8core - but it does have quad channel
memory where the 4100 series only has dual channel.
SATA drives? Yuck!
I already have a Phenom X4 2.5GHZ and an X4 3.4GHZ, but they are both
running other applications until I can move them to a system that can run
many virtual machines.
Somehow these albatrosses collect along the way, where everthing works on
a specific install of a specific OS and level, together with all of the
subsequent installs to complete the system. However, I sometimes need to
be able to use it when the customer has a problem. Some apps need to
be up 24/7 in order to check them out. As it is, I can only run as many
of these concurrently as I have physical machines, with each one running
on only one of the partitions.
Why are you limited to *physical* machines with the X4 systems but able to
run many on an Opteron 4100 series?
Those machines are running 24/7, so I want to install xen without any
impact on the existing applications.
Also I will need 64G of memory for the main application coming up,
and the existing machines are maxed out at 16G of memory. At the time
I put them together 16G was all I could get for a reasonable price,
and that was enough for the projected use.
One application will be sequencing the human genome and others, and
doing common subsequence amalysis.
At 21 bytes per word, 2.8 billion words takes 58.8G to hold it.
The suffix tree in this case takes 20 bytes per entry, and 1 byte for
each code makes 21. 3 billion words @ 21 bytes/word = 63GB.
The goal is to be able to build the suffix tree for the 2.8 B words
in 10 to 15 minuutes.
Another application will be queries against a set of files.
The suffix tree finds every occurrence of every string that occurs
more than once, which is very useful for queries.
Another application is data compression. By abbreviating common strings
the data can be made to take a fraction of the space. Obviously,
the more files are includedf in the set, the more efficient abbreviating
will be. This approach is different from other compression methods in
that it deals with many files at once, and will be more efficient as a
result. The abbreviator is actually a very difficult program to write,
as it uses both inserts and deletes on the suffix tree.
In past experiments with relational tables, I used this idea in a simpler
form, and I was able to get compression ratios up to 35 to 1.
The encoding for the data permits 3 billion words of data.
At 10K words per file, it could hold 300,000 files.
Another application is computing similarity measures between files.
For that application, 4G of memory can hold 3,000 files from the Gutenburg
collection, and it computes the 1500 x 2999 coefficients.
With xen I'm planing to collect them all into their own partitions and
be able to run them concurrently at any time. As long as they do not need
the same non sharable resource at the same time, like if the resource
is the incoming port 80 traffic from the internet, which depends on
how I can configure the router, it all looks like it will work OK.
The difference between the Phenom X series and the Opterons is that
Opteron supports registered ecc memory, whereas the Phenom series does not.
This may not be important, but if I am running a whole bunch of stuff,
and it actually DOES get a memory error, then I will know about it.
Otherwise, it will just go along until things crash.
And it might even correct the error via crc.
If you go with X6, then you need non ecc memory. I think I found it
for $3xx for 16GB Of ddr3 1066. You definitely want ddr3. It is
much faster and more energy efficient than ddr2.
The ecc ddr3 memory is not much different in cost. In fact, sometimes
it is actually less. So it may be cheaper to buy an opteron + memory
than an X6 + memory.
The Phenom DOES support ECC (though I don't think it supports registered).
Most MOTHERBOARD BIOS's don't enable ECC. You can download a script to enable
ECC after you're booted up - it uses setpci to flip NB registers. After the
script is run you could load up the EDAC module, and there you go. This is
what I do on my home fileserver. ECC fully enabled.
You're always going to be using DDR3 with a modern system. ECC is never
cheaper than non-ECC memory at equivalent speed/voltage ratings, it requires
1 extra chip per 8! Registered memory is yet again more expensive (I run
ECC/unbuffered with my consumer board + Athlon II/Propus X4)
Hmm. The newegg price for 4 x 8G ddr3 reg ecc dimms was $1,130,
whereas 4 x 8g of non ecc unbuffered was $1,056 at memory america.
I also found 4 x 8g of ecc reg for $770.
So the G34 ($413), two 6128s (2 x $276), case ($100) and 8 x 8G ddr3
($1,640)
comes to $2,595, about $2,100 cheaper than using the 4180 with 4 16G
dimms. Plus, it is 16 cores.
So the difference between a regular X6 with 16G of memory and this 64G
system is about $1,500 or so.
I see your point. It's $2,000 cheaper and better stuff to go with the
G34.
Lex
The only drawback is the motherboard. $283 is a lot for a motherboard.
The MSI motherboard for the X6 is only $85, last time I checked.
$85 (mdbd) + $200 (X6) + $80 (case) + $315 (16GB 1066ddr3) = $780.
+ $140 (one 2TB sata 2Gb/s disk) = $920.
The Opteron starts out about $200 more for the mbd, but the memory is
definitely more reliable, and sometimes actually faster.
The memory is the same - although IIRC the X6 is "officially" rated to faster
speeds than the 4100.
Fully buffered ecc memory can sometimes be faster for sequential access,
and sometimes slower for random access, because of the crc checking
overhead. If I remember right, that can make it 0.5% to 1.5% slower
for random access.
With unbuffered, non ecc DDR3 1066 memory on the 3.4Ghz X4, a program that
stores randomly a billion times in two billion locations takes 135 ns per
operation.
Sequential stores take 0.91 ns per byte.
Random reads and update of each byte takes 175 ns per byte.
This includes the time to generate the psuedo random numbers, which
is 20 ns per number.
The time to access them sequentially and add them up is 0.35 seconds,
0.35 nanoseconds each.
If you only need 4 or 8GB of memory, then it will be really cheap.
64GB of 4 x 16GB ddr3 dimms for the opteron is more than $4000.
16GB DIMMs? Why not 8 8GB DIMMs?
Dual CPU board lets you do 16/4.. and a quad will let you do 32/2.
Dual G34 motherboards are available for $400. . that gives you 16 DIMM slots,
and 8 core 6128s are only $280.
Quad G34 around $800 - 32 DIMMs, 2GB DIMMs are a lot cheaper! That gets you
32 2.0GHz cores, 48MB of L3 cache and a motherboard for under $2000. Throw in
a motherboard, storage, and a power supply.. and you have a fairly cheap
powerhouse.
Maybe this is a little off of the SPAM 6.0 topic. Start another?
Lex
On Wed, 19 Jan 2011, Gary Mort wrote:
With time to learn some new skills I've been ramping up my Amazon EC2
skills.....but I really want to play a bit further without paying for some
things. With my laptop finally dead, I'm replacing it with a desktop....
a refurb system with an AMD Phenom II X6 processor[and BIOS support for
AMD's virtualization technology].
I figured I'd reach out to see if anyone else in the Duchess/Ulster area
is mucking about with Xen and wants to exchange emails/get together to go
through stuff.
-Gary
_______________________________________________
Mid-Hudson Valley Linux Users Group http://mhvlug.org
http://mhvlug.org/cgi-bin/mailman/listinfo/mhvlug
Upcoming Meetings (6pm - 8pm) MHVLS Auditorium
Feb 2 - Zimbra
Mar 2 - MHVLUG 8th Anniversary - Show and Tell
Apr 6 - Introduction to IPv6
_______________________________________________
Mid-Hudson Valley Linux Users Group http://mhvlug.org
http://mhvlug.org/cgi-bin/mailman/listinfo/mhvlug
Upcoming Meetings (6pm - 8pm) MHVLS Auditorium
Feb 2 - Zimbra
Mar 2 - MHVLUG 8th Anniversary - Show and Tell
Apr 6 - Introduction to IPv6
