So, to fix this, you could: -create more volumes and split the load between the drive, which is limited by the number of drive letters that can be assigned -create more file servers and spread the load over all of them
Do other operating systems handle the issue of too many small files better than Windows? -----Original Message----- From: Michael B. Smith [mailto:mich...@smithcons.com] Sent: Wednesday, September 07, 2011 6:00 AM To: NT System Admin Issues Subject: RE: # of files on Windows server, is 4 million too many? When performance starts to suffer? :-) Regards, Michael B. Smith Consultant and Exchange MVP http://TheEssentialExchange.com -----Original Message----- From: Kennedy, Jim [mailto:kennedy...@elyriaschools.org] Sent: Tuesday, September 06, 2011 9:46 PM To: NT System Admin Issues Subject: RE: # of files on Windows server, is 4 million too many? Going along the same lines, when do you start to worry about the number of concurrent client connections to said SAN attached to a 2008 R2 server, simple document connections.... ________________________________________ From: Michael B. Smith [mich...@smithcons.com] Sent: Tuesday, September 06, 2011 6:11 PM To: NT System Admin Issues Subject: RE: # of files on Windows server, is 4 million too many? No, it matters in file open and file extension as well – but not to as great a degree. 2008 R2 doesn’t improve this to any degree EXCEPT that the Server service no longer has any limit to the amount of directory cache that it can allocate. This means that LOCAL enumeration (after the first) will probably be faster than the initial enumeration. Across the network (SMB / CIFS) that is not assured. Regards, Michael B. Smith Consultant and Exchange MVP http://TheEssentialExchange.com From: David Lum [mailto:david....@nwea.org] Sent: Tuesday, September 06, 2011 2:53 PM To: NT System Admin Issues Subject: RE: # of files on Windows server, is 4 million too many? Would there be any worthwhile gains to attach this SAN drive to a 2008 R2 server vs. 2003 32-bit? Enumeration is about the only time raw number of files matter, right? Feeling like a n00b asking that one. Dave From: Michael B. Smith [mailto:mich...@smithcons.com]<mailto:[mailto:mich...@smithcons.com]> Sent: Tuesday, September 06, 2011 11:19 AM To: NT System Admin Issues Subject: RE: # of files on Windows server, is 4 million too many? I have a customer with a 2003 server that has a couple million files in a single folder. I didn’t say it didn’t work. I said it wasn’t performant. And I stand by that. ☺ Regards, Michael B. Smith Consultant and Exchange MVP http://TheEssentialExchange.com From: Cameron [mailto:cameron.orl...@gmail.com]<mailto:[mailto:cameron.orl...@gmail.com]> Sent: Tuesday, September 06, 2011 2:15 PM To: NT System Admin Issues Subject: Re: # of files on Windows server, is 4 million too many? Damn...then I'm in trouble! One Win 2K server currently has 7,981,328 files at 60GB On Tue, Sep 6, 2011 at 1:38 PM, David Lum <david....@nwea.org<mailto:david....@nwea.org>> wrote: 2003 Server, 32-bit From: Andrew S. Baker [mailto:asbz...@gmail.com<mailto:asbz...@gmail.com>] Sent: Tuesday, September 06, 2011 10:32 AM To: NT System Admin Issues Subject: Re: # of files on Windows server, is 4 million too many? What OS, btw? ASB http://XeeMe.com/AndrewBaker<http://xeeme.com/AndrewBaker> Harnessing the Advantages of Technology for the SMB market… On Tue, Sep 6, 2011 at 1:15 PM, David Lum <david....@nwea.org<mailto:david....@nwea.org>> wrote: Here’s the results from a single logical (SAN) drive on one of our file servers: Total Files Listed: 4661023 File(s) 681,427,607,680 bytes On this logical drive are our primary shares for users and shared data, but if no single folder has more than 2000 files does it matter? I’m thinking still yes. And yes, it takes FOREVER to do any kid of file maintenance on this. Dave From: Michael B. Smith [mailto:mich...@smithcons.com<mailto:mich...@smithcons.com>] Sent: Tuesday, September 06, 2011 8:57 AM To: NT System Admin Issues Subject: RE: # of files on Windows server Consider this: NTFS is a type of database. And not a very efficient one either. And, for very small files, the files are actually stored within the file system, while with larger files, NTFS has pointers to the actual file data stored in the FS. Storage size has exploded in the last 10 years. However, performance has not matched the size expansion. If I’m going to simply OPEN or CREATE a small file – number of files has little impact. NTFS is just going to create a new entry in the database. If I need to find, list, or extend a file – well, it’s going to take longer. If I need to enumerate files (that is, get a directory listing) the more files I’ve got, the longer it’s going to take. It “can be shown” that having more than about 1K files in a directory will effect enumeration. It becomes really noticeable (IMHO) around 10K and heads rapidly downhill after that. Regards, Michael B. Smith Consultant and Exchange MVP http://TheEssentialExchange.com<http://theessentialexchange.com/> From: David Lum [mailto:david....@nwea.org<mailto:david....@nwea.org>] Sent: Tuesday, September 06, 2011 11:46 AM To: NT System Admin Issues Subject: # of files on Windows server Recently we had a thread about how many files get to be too many for reasonable performance. Would this be just per folder, or possibly logical drive in general? Links/documents would work too. David Lum Systems Engineer // NWEATM Office 503.548.5229<tel:503.548.5229> // Cell (voice/text) 503.267.9764<tel:503.267.9764> ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com<mailto:listmana...@lyris.sunbeltsoftware.com> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to listmana...@lyris.sunbeltsoftware.com with the body: unsubscribe ntsysadmin
