I've created 1 billion files (or more) in my MD tests, and one major thing that you do run into is the OST allocation issue I mentioned in my presentation at LUG. You can alleviate this by increasing the number of OSTs you have in your FS (I wouldn't exceed 32M files per OST), or by changing the OST object layout.
-----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Sam Bigger Sent: Friday, May 20, 2011 12:17 PM To: 'Daniel Mayfield'; [email protected] Subject: Re: [Lustre-discuss] Huge numbers of inodes I doubt it. The largest lustre systems are a little over 300 million files that are really in existence. However, the limits, in terms of just maximum inodes allowed, using 4K per inode and a 2TB max MDT size (mirrored 2 copies), only allows for a little over 500 million files max. -Sam Whamcloud -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Daniel Mayfield Sent: Friday, May 20, 2011 9:53 AM To: [email protected] Subject: [Lustre-discuss] Huge numbers of inodes Does anybody have experience with Lustre filesystems in the 5-10B inode range? What kind of gotchas do you find at that scale? What are your stripe settings like? daniel --------------------------------------------------------------- This email, along with any attachments, is confidential. If you believe you received this message in error, please contact the sender immediately and delete all copies of the message. Thank you. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
