XFS might be a good change for us in the future, thanks for the suggestion. For me, it won't be an option I can introduce immediately.
I've created a patch that basically starts a persistent thread to take care of 'unlink' requests. This is patched against the 2.6.3 release. The unlink now renames the file (puts it in the stranded-bstreams dir), adds it to a shared queue, signals the unlink thread to delete it, and returns.. There were a several side effects that the block on deletion caused us. 1) the pvfs2-check-server called via heartbeat would timeout, getting caught behind the remove, and would send heartbeat STONITH'ing away. After STONITH on a 'rm' command, that sent our 3+TB devices into fsck for a bit, and client jobs canceled away. 2) Other accesses, like 'ls' and reads/writes also queued behind deletes, and then they started canceling as well. 3) Once the BMI jobs started timing out and retries sent, there were some cases where canceling those jobs didn't quite stop accesses to the jobs memory. Phil Carns sent me a patch to fix this. I've ran a small sample of tests on a single PVFS2 server with a separate machine for the PVFS2 client. Both machines are running RHEL4 U7 i386 WS. The short version is that there was little affect on current performance for small files, and a large performance increase from a clients perspective for large files. +-----------+-----------+--------------+----------------+ + Num Files + File Size + Time (stock) + Time (threaded)+ +-----------+-----------+--------------+----------------+ + 10 + 0 bytes + 0.55 (secs) + 0.51 (secs) + +-----------+-----------+--------------+----------------+ + 500 + 100 bytes + 35 (secs) + 33 (secs) + +-----------+-----------+--------------+----------------+ + 10,000 + 100 bytes + 728 (secs) + 702 (secs) + +-----------+-----------+--------------+----------------+ + 10 + 500 MB + 7 (secs) + 0.73 (secs) + +-----------+-----------+--------------+----------------+ -----Original Message----- From: Emmanuel Florac [mailto:[EMAIL PROTECTED] Sent: Friday, October 03, 2008 1:50 AM To: [EMAIL PROTECTED] Cc: [email protected] Subject: Re: [Pvfs2-developers] PVFS2 Removal of large files Le Thu, 2 Oct 2008 16:57:19 -0500 vous écriviez: > What are you thinking along the lines of tuning the client side? I'd strongly encourage you NOT to use ext3. ext3 is by far the slowest filesystem under Linux, and it behaves very poorly on big filesystems. I'd definetely go for XFS ( or JFS ) on the underlying device. -- -------------------------------------------------- Emmanuel Florac www.intellique.com --------------------------------------------------
lazy-delete.patch
Description: Binary data
_______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
