Gregory S. Youngblood writes: > I have two IDE drives (on different controllers), using LVM striped across > both drives, and ReiserFS 3.6.25. > > All of this has been installed from a stock SuSE 7.2 professional > installation, using a Linux 2.4.4 kernel. > > First thing, on boot up, there is a report that I'm running ResierFS > 3.6.25, but that I'm using the 3.5 disk image format. Is this normal? Or is > this something SuSE did on purpose? This system had the drives wiped prior > to installation of SuSE and LVM, so all partitions were created with the > SuSE 7.2 software. > > The big problem is performance. > > On Friday I placed almost 700,000 files (all between 0 and 1200 bytes, most > being 800 to 1000 bytes) in a single directory that was exported via Samba > to a windows box. This box was running a process that analyzed each > individual file, did some processing with the data, and then deleted the > file. The process runs at a very leisurely pace, maybe hitting 6 to 10 > files a second. Doesn't hit CPU, network or IO too hard. Dir command on the > Windows box returned files in a reasonable time frame. > > On Saturday we took an unexpected power failure and my box went down hard. > ReiserFS recovered, replaying several transactions and then the system came > up OK, or so it appeared. > > The problem is that ls -U (for unsorted list), or just place ls or du, on > this specific directory does not return anything and just thrashes the hard > drive ( and CPU to a lesser degree ). I left ls running for between 60 and > 90 minutes before killing it. The directory listing from the Windows box > never returned either. can you try ls -Uf? > > I still haven't been able to get into this directory, but I was able to > move it out of the way and recreate the original data. Unfortunately this > means I had to restart the master process so I effectively lost 2 days of > processing since I had to start over. > > Is there a limit on files that can be in a single directory? What about > practical limits, where the number of files in a directory brings > performance down to a point that it should be avoided? Performance is degrading due to fragmentation: directory entries are sorted according to their names (actually, hash of name, but default r5 hash locally preserve lexicographic order) and "inodes" and file bodies are sorted according to their object-ids. When directory is first populated, those orders roughly coincide, because object-ids are allocated sequentially. Problem with ls and du is that they do both readdir and stat. While each of those operations is fast by itself their interleaving causes many disk seeks on fragmented directory. > > In this particular case we would normally put these files on NTFS, since it > handles these quite effortlessly. I needed to evaluate ReiserFS and LVM, > and so far it isn't looking that good. ReiserFS has performed flawlessly up > to now, but I don't think I ever went over 50,000 files in a directory > before this, so I can't say for sure. > > Any advice on this situation or configuration? I couldn't find anything > particularly useful on Google or the FAQ for this. > > Thanks, > Greg Nikita.
