Perhaps using tmpfs may be an option. Benefit of using tmpfs is that you can create a filesystem that is larger than physical memory. This has the benefit that virtual memory manager will swap out unused items to disk. You can then perhaps NFS export the file system or do something else. Difficult to say without additional details.
Vladimir Jay Paroline wrote:
We already rely on memcached quite heavily to minimize load on our DB with stunning success, but as a music streaming service, we also serve up lots and lots of 5-6MB files, and right now we don't have a distributed cache of any kind, just lots and lots of really fast disks. Due to the nature of our content, we have some files that are insanely popular, and a lot of long tail content that gets played infrequently. I don't remember the exact numbers, but I'd guesstimate that the top 50GB of our many TB of files accounts for 40-60% of our streams on any given day. What I'd love to do is get those popular files served from memory, which should alleviate load on the disks considerably. Obviously the file system cache does some of this already, but since it's not distributed it uses the space a lot less efficiently than a distributed cache would (say one popular file lives on 3 stream nodes, it's going to be cached in memory 3 separate times instead of just once). We have multiple stream servers, obviously, and between them we could probably scrounge up 50GB or more for memcached, theoretically removing the disk load for all of the most popular content.
