Peter Grandi wrote:
> Ha! that probably means some RAID arrangement. I would be happy
> to know how long a full 'jfs_fsck -f' takes on that filesystem,
> if you have ever had to do it.
Indeed; we're using a 3ware SATA RAID card.
As for the fsck time, I haven't had to do it yet, but I'm sure it will
take a while. (On some other machines with similar size filesystems
using reiserfs, an fsck takes upwards of 12 hours and frequently
exhausts 4GB of RAM.) The images are mirrored to two different servers
as they're uploaded, so losing data on one is not the end of the world.
> To me this seems a very strange concern. You have a cache that
> is at best 0.1% (1/1000th) the size of the volume of data it is
> supposed to cache and you are «concerned» about the difference
> between 1/1,000th and 1/10,000th? Uhm, a bit strange.
If the images were equal in popularity, this would be true.
In my case, the images are user-uploaded photos for a social networking
site, and popularity is generally a factor of how recently the image was
uploaded and the "connectedness" of the user who uploaded it. Likewise,
the relatively tiny thumbnails are served quite frequently, and the
relatively huge originals are *never* served (they're kept just in case
we redesign the site and have to re-resize things). My guess is that
0.1% of our photo storage makes up a double-digit percentage of the load
- cutting that by a factor of 10 makes a huge difference.
> File system caching except for the top levels of the tree and
> some metadata is usually pointless anyhow.
This is interesting to me and deserves further study. Given the slow
seek time of big 7200RPM SATA drives, it seems like being able to serve
the file from memory would be a huge win.
(We're advertiser-supported - barring a surge in people willing to pay
$20CPM rates for banner ads, the photos aren't going to be stored on 15K
SCSI drives any time soon. <g>)
> Conversely one could explore storing the thumbnails in a
> GDBM/Berkeley DB/... hash database and probably the 50-100KB
> range images too.
A clever idea, probably worth prototyping to get some benchmarks. As
with some discussions we've had internally about using Apache's
mod_mem_cache to cache things instead of letting the VFS layer do it,
the hope is that with some proper tweaking the OS will be able to "do
the right thing" most of the time.
> Good ideas, but also I would try the 'deadline' elevator and the
> 'noop' one too. They may be more appropriate (especially 'noop').
This I will definitely try and report back to the list.
--
Dave Pifke, [EMAIL PROTECTED]
Sr. System Administrator, www.bebo.com
-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Jfs-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/jfs-discussion
