Re: [Declude.JunkMail] OT: Max number of files in directory?
On NTFS systems, this is most likely app-related such as Explorerer where they have to deal with slogging through all the extra files, as noted by another poster. An App opening a specific file will see almost no degradation because the NTFS uses a tree structure to maintain fast access to a file by name. Very true. Getting that one file is very quick. The reason that Explorer is so slow is that it has to at the very least get the name of every single file. In the DOS days, that was relatively quick and easy (a directory of 20,000 files would take up about 320K). With NTFS, though, each file typically uses 8K, so 20,000 files would take up 150MB. So doing a directory listing of 20,000 files is like loading a 150MB files. Worse, if the directory is fragmented (which is very common), it takes even longer. With millions of files, it can take hours just to do directory listings. As someone else pointed out, disabling the last access time can help; also, disabling 8.3 can help too (only on computers with no programs that need 8.3, so you have to be careful with that). Disabling 8.3 can help a lot if the first 5-6 characters of the filenames are often the same. -Scott --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] OT: Max number of files in directory?
I am not sure if this will help on this issue but is worth a shot. There are some other setting we change on the folder view settings that we set when we have remote users connecting remotely with a VPN. This seems to help with the speed when they are browsing the folder structure. First change the folder view to classic the XP view tries to obtain a lot of info with this view. Next Folder Optionsviewadvanced setting set the following. Not checked Not checked Checked Not Checked Checked Checked Second option Not checked Not checked Not checked First option Checked Not checked Not checked Checked Not checked Not checked Then apply to all folders Hopefully this will help Rick -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of R. Scott Perry Sent: Thursday, April 13, 2006 4:35 AM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? On NTFS systems, this is most likely app-related such as Explorerer where they have to deal with slogging through all the extra files, as noted by another poster. An App opening a specific file will see almost no degradation because the NTFS uses a tree structure to maintain fast access to a file by name. Very true. Getting that one file is very quick. The reason that Explorer is so slow is that it has to at the very least get the name of every single file. In the DOS days, that was relatively quick and easy (a directory of 20,000 files would take up about 320K). With NTFS, though, each file typically uses 8K, so 20,000 files would take up 150MB. So doing a directory listing of 20,000 files is like loading a 150MB files. Worse, if the directory is fragmented (which is very common), it takes even longer. With millions of files, it can take hours just to do directory listings. As someone else pointed out, disabling the last access time can help; also, disabling 8.3 can help too (only on computers with no programs that need 8.3, so you have to be careful with that). Disabling 8.3 can help a lot if the first 5-6 characters of the filenames are often the same. -Scott --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] OT: Max number of files in directory?
I agree with the changes advised in this thread for registry settings. Let me point out that these are meant to be applied on the server where the files are, and not, for example, to a workstation that is trying to manipulate a folder over the network. I've been trying out Servant Salamander from http://www.altap.cz/ which I investigated because I was looking for a good modern implementation of Midnight Commander (which is in turn a good modern implementation of NC or Norton Commander). It gets my highest praise: It Just Works. I can use it over my WAN to my Declude spam folder and wait less than 90 seconds to get a complete directory listing with over 380,000 files in it (and that's a busy server with none of the NTFS performance reg hacks). Tip: If you really work interactively with folders this big, turn off the automatic refresh in the configuration options. There's a free older version and a current version that is cheap. Andrew. P.s. Ever used Sysinternals.com tools to monitor explorer.exe with filemon.exe or regmon.exe? It's incredible how much extra work Explorer is doing behind the scenes, so it's no wonder that a light file manager can do the job quicker. In Mark Russinovich's blog, he pointed out just how bad that is (which has since been optimized in Vista): http://www.sysinternals.com/blog/2005/06/explosion-of-audit-records.html -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dave Doherty Sent: Tuesday, April 11, 2006 2:15 PM To: Declude.JunkMail@declude.com Subject: [Declude.JunkMail] OT: Max number of files in directory? Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Max number of files in directory?
One other thing that helps out a lot with this is to use some form of RAID that spans. A single drive or mirrored drives that try to read a bunch of files while also doing regular tasks would be impossible with that many files, in fact Explorer is prone to crashing under those circumstances. I find that RAID 5 with 5 drives can easily handle 100,000 files in a matter of seconds, but it seems to get worse with more files in a sort of exponential rate. Matt Colbeck, Andrew wrote: I agree with the changes advised in this thread for registry settings. Let me point out that these are meant to be applied on the server where the files are, and not, for example, to a workstation that is trying to manipulate a folder over the network. I've been trying out Servant Salamander from http://www.altap.cz/ which I investigated because I was looking for a good modern implementation of Midnight Commander (which is in turn a "good modern implementation" of NC or Norton Commander). It gets my highest praise: It Just Works. I can use it over my WAN to my Declude spam folder and wait less than 90 seconds to get a complete directory listing with over 380,000 files in it (and that's a busy server with none of the NTFS performance reg hacks). Tip: If you really work interactively with folders this big, turn off the automatic refresh in the configuration options. There's a free older version and a current version that is cheap. Andrew. P.s. Ever used Sysinternals.com tools to monitor explorer.exe with filemon.exe or regmon.exe? It's incredible how much extra work Explorer is doing behind the scenes, so it's no wonder that a light file manager can do the job quicker. In Mark Russinovich's blog, he pointed out just how bad that is (which has since been optimized in Vista): http://www.sysinternals.com/blog/2005/06/explosion-of-audit-records.html -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Dave Doherty Sent: Tuesday, April 11, 2006 2:15 PM To: Declude.JunkMail@declude.com Subject: [Declude.JunkMail] OT: Max number of files in directory? Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Max number of files in directory?
I don't know what the limit is, but there is definitely degradation in performance after a few thousand files. On NTFS systems, this is most likely app-related such as Explorerer where they have to deal with slogging through all the extra files, as noted by another poster. An App opening a specific file will see almost no degradation because the NTFS uses a tree structure to maintain fast access to a file by name. There is a separate issue where NTFS doesn't recover unused directory space after deleting many files. This only becomes an issue with files numbering in the millions --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] OT: Max number of files in directory?
I don't know what the limit is, but there is definitely degradation in performance after a few thousand files. For what it's worth, I've found some of the tips for improving NTFS performance found in this article to be useful, especially disabling the Last Access Time if you don't need it. http://www.windowsdevcenter.com/pub/a/windows/2005/02/08/NTFS_Hacks.html - Andy --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Max number of files in directory?
At 04:14 PM 4/11/2006, you wrote: Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. What file system are you using? I don't think there is a file limit per directory in NTFS like there was under FAT and FAT32. I had originally thought there was a file limit in the root directory in NTFS, but I could not find any source stating this. This article my be helpful: http://www.windowsitpro.com/Articles/Index.cfm?ArticleID=27253DisplayTab=Article Mike --- [This E-mail scanned for viruses by Declude Virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Max number of files in directory?
I don't know what the limit is, but there is definitely degradation in performance after a few thousand files. Darin. - Original Message - From: Michael Graveen [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Tuesday, April 11, 2006 7:51 PM Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? At 04:14 PM 4/11/2006, you wrote: Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. What file system are you using? I don't think there is a file limit per directory in NTFS like there was under FAT and FAT32. I had originally thought there was a file limit in the root directory in NTFS, but I could not find any source stating this. This article my be helpful: http://www.windowsitpro.com/Articles/Index.cfm?ArticleID=27253DisplayTab=Article Mike --- [This E-mail scanned for viruses by Declude Virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] OT: Max number of files in directory?
The degeradation comes from using explorer. We have an imaging application the gets upwards of 250,000 file in it. We has issues with explorer taking hours to display the files to manually archive them. We had to write a program to archive them to avoid the pain of explorer. What took all day manually, because of waiting for the listing to appear, takes a few minutes with the .net program we wrote to archive the files to another disk. The degradation would depend on the application and how you are listing the files. Kevin Bilbee -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Darin Cox Sent: Tuesday, April 11, 2006 5:09 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? I don't know what the limit is, but there is definitely degradation in performance after a few thousand files. Darin. - Original Message - From: Michael Graveen [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Tuesday, April 11, 2006 7:51 PM Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? At 04:14 PM 4/11/2006, you wrote: Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. What file system are you using? I don't think there is a file limit per directory in NTFS like there was under FAT and FAT32. I had originally thought there was a file limit in the root directory in NTFS, but I could not find any source stating this. This article my be helpful: http://www.windowsitpro.com/Articles/Index.cfm?ArticleID=27253Dis playTab=Article Mike --- [This E-mail scanned for viruses by Declude Virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Max number of files in directory?
It's NTFS. Thanks for the referral. That's just what I needed. -d - Original Message - From: Michael Graveen [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Tuesday, April 11, 2006 7:51 PM Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? At 04:14 PM 4/11/2006, you wrote: Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. What file system are you using? I don't think there is a file limit per directory in NTFS like there was under FAT and FAT32. I had originally thought there was a file limit in the root directory in NTFS, but I could not find any source stating this. This article my be helpful: http://www.windowsitpro.com/Articles/Index.cfm?ArticleID=27253DisplayTab=Article Mike --- [This E-mail scanned for viruses by Declude Virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Max number of files in directory?
It's just straight archiving. Thanks for the tips. We may zip them into bloocks by day or month to prevent the Explorer probelm. -d - Original Message - From: Kevin Bilbee [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Tuesday, April 11, 2006 8:42 PM Subject: RE: [Declude.JunkMail] OT: Max number of files in directory? The degeradation comes from using explorer. We have an imaging application the gets upwards of 250,000 file in it. We has issues with explorer taking hours to display the files to manually archive them. We had to write a program to archive them to avoid the pain of explorer. What took all day manually, because of waiting for the listing to appear, takes a few minutes with the .net program we wrote to archive the files to another disk. The degradation would depend on the application and how you are listing the files. Kevin Bilbee -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Darin Cox Sent: Tuesday, April 11, 2006 5:09 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? I don't know what the limit is, but there is definitely degradation in performance after a few thousand files. Darin. - Original Message - From: Michael Graveen [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Tuesday, April 11, 2006 7:51 PM Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? At 04:14 PM 4/11/2006, you wrote: Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. What file system are you using? I don't think there is a file limit per directory in NTFS like there was under FAT and FAT32. I had originally thought there was a file limit in the root directory in NTFS, but I could not find any source stating this. This article my be helpful: http://www.windowsitpro.com/Articles/Index.cfm?ArticleID=27253Dis playTab=Article Mike --- [This E-mail scanned for viruses by Declude Virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] OT: Max number of files in directory?
Good to know. Thanks, Kevin. Explorer definitely keeps getting heavier and slower. Looks like it's about time for a simple explorer-like app that doesn't have all of the extras like thumbnail viewing, metadata, application hooks, etc simple file listing and registry-based association. Darin. - Original Message - From: Kevin Bilbee [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Tuesday, April 11, 2006 8:42 PM Subject: RE: [Declude.JunkMail] OT: Max number of files in directory? The degeradation comes from using explorer. We have an imaging application the gets upwards of 250,000 file in it. We has issues with explorer taking hours to display the files to manually archive them. We had to write a program to archive them to avoid the pain of explorer. What took all day manually, because of waiting for the listing to appear, takes a few minutes with the .net program we wrote to archive the files to another disk. The degradation would depend on the application and how you are listing the files. Kevin Bilbee -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Darin Cox Sent: Tuesday, April 11, 2006 5:09 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? I don't know what the limit is, but there is definitely degradation in performance after a few thousand files. Darin. - Original Message - From: Michael Graveen [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Tuesday, April 11, 2006 7:51 PM Subject: Re: [Declude.JunkMail] OT: Max number of files in directory? At 04:14 PM 4/11/2006, you wrote: Does anyone know of either a hard or practical limit on the number of files that a directory may contain under Windows XP? -Dave Doherty Skywaves, Inc. What file system are you using? I don't think there is a file limit per directory in NTFS like there was under FAT and FAT32. I had originally thought there was a file limit in the root directory in NTFS, but I could not find any source stating this. This article my be helpful: http://www.windowsitpro.com/Articles/Index.cfm?ArticleID=27253Dis playTab=Article Mike --- [This E-mail scanned for viruses by Declude Virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
