The last time I looked at the Violin (~9 months ago), it was crazy
expensive, especially given how easy and cheap (relatively) it is to
make a 1TB+ memcached pool.
You sound especially concerned with rack units, so if you're
super-constrained for space, I suppose that is a concern. But generally
most datacenters are power-constrained these days, rather than space
constrained. If that's the case for you, I doubt the possible power
savings from having less CPUs on a Violin setup offsets the extremely
high price of the device.
One upside to having 4-16 nodes in your pool is better failure
resiliency. You'll fail more often, but failures will be less catastrophic.
If you are looking to use block devices for some reason other than space
savings, I've seen a few things from various vendors that are coming in
the next few months based on SSD that will be massively better from a
price-point comparison, so you may want to wait and compare. They're
not going to give you RAM performance, but thousands of random IOPS per
device multiplied by a handful of devices might work well for your use case.
Don
Brad Fitzpatrick wrote:
Ah! I understand now.
Sorry, my bad .... so many people on this list trying to use memcached
in really perverted ways and often not understanding how it works or how
it should be used... so I've grown a bit of a knee-jerk reaction.
On Sun, Oct 26, 2008 at 12:21 AM, Dan Simoes <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
Replying only to you so I don't clutter the list.
Did you read my original inquiry? No one said I wanted to shard a
block device.
I specifically asked about memcache on a block device, and
specifically the Violin.
We're already using memcache, with excellent results (so thanks for
open sourcing it!)
The Violin device appears as a block device to the system. Memcache
lives in RAM. So in order to make it work, someone has to make
memcache work against a block device. According to the Violin
people, they did this themselves, but someone has since written a
community driver.
I did search the archives but the only mention of the Violin device
is from the company themselves, back in July 07.
As I said, we're primarily interested in it for improving database
IO, but it was originally presented to us as the ideal platform for
a large memcache farm. I have my doubts about that, hence the inquiry.
Thanks.
Dan
On Sun, Oct 26, 2008 at 12:06 AM, Brad Fitzpatrick <[EMAIL PROTECTED]
<mailto:[EMAIL PROTECTED]>> wrote:
This mailing list's about memcached, not using memory or using
caches in general.
You do not want to use memcached for what you're talking about
it. If you want to shard a block device over a bunch of
machine's memory, LVM and NBD and ram block devices are your
friend. But that's not for this list.
On Sat, Oct 25, 2008 at 11:58 PM, Dan Simoes
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
For database, the motivation is increased disk i/o and IOPS
beyond what current solutions allow. In addition to the
Violin device, we're looking at other flash, ram and SSD
technologies, but these are admittedly bleeding edge.
We've already done all the other stuff to get better
performance out of our db, and if we could get the dbs
running on this super-fast storage, we think it's worth
examining.
For memcache, it's examining how to scale a large memcache
farm. Our current needs are for 512Gb-1Tb of memcache.
Right now that looks like 8-16U of servers, vs 3U for the
Violin. The Violin people say that when you factor in the
operational costs (space, power, etc) their solution saves
money.
Why do you think I would lose my disk blocks randomly on a
fast block device? According to their specs, it's got a
RAID5-like design so you can lose memory boards without
dropping data.
Lest anyone think this is an ad for the device, it's not. I
have yet to see one, but I'm sure someone here has looked at
it before?
Thanks.
Dan
On Sat, Oct 25, 2008 at 11:28 PM, Brad Fitzpatrick
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
That sounds like a terrible idea.
I was going to leave it at that, but I'll elaborate:
* memcached is a cache. do you want to lose your disk
blocks randomly?
* if you just want fast database slave replicas and
don't care about reliability, there are already
in-kernel memory filesystems and memory block devices.
you'll get much more memory efficiency over those, as
there won't be per-item overhead and internal
fragmentation. (admittedly you could tune memcached's
slab sizes to the size of your blocks, but you'd still
have per-item overhead, both memory and CPU wise
maintaining the LRUs, etc)
So uh, yeah.... I don't get your motivation.
On Sat, Oct 25, 2008 at 8:08 PM, Dan Simoes
<[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
I'm curious as to what results people have had with
the block device driver that is in the community
branch (or so I've been told). We're looking at the
Violin Memory device, primarily for DB storage, but
it may have interesting applications for memcache as
well.
Thanks.
Dan
