We have quite large mogile storage setup running in production. Our current mogile cluster has 44T of storage where 19T used. Total number of stored files is ~330 millions (x3 copies ~= 1 billion objects)
We customized mogile cluster next way: - 8 federated db boxes with backup mysql replication - 2 trackers per db group (16 total) - 23 storage nodes, each has 8 250-320G drives (one for each group) Our performance is - 50 writes per second per whole cluster (tested up to 500 wps) - 120+ reads per second per storage node (~ 3000 reads per second on a whole cluster) - 80% full device recovery (re-replication) usually takes 30 hours with no affect to production performance. We developed custom web gui tool and init.d scripts to manage multiple (8) instances of storage daemons (mogstored) on a given storage node. GUI tool also comes handy with graphical display of usage pattern and device/storage node availability. In addition, we designed custom nagios checks that trigger notifications about tracker/storage node availability along with single drive (device) failure. I hope you find this helpful. Oleg -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Justin Huff Sent: Thursday, December 13, 2007 9:42 AM To: Clinton Goudie-Nice Cc: mogilefs Subject: Re: Questions about MogileFS in a production environment I'm not sure we'd count as 'large scale' -- but we use it at Picnik. We have about 5.7TB, using about 2.7TB. That's about 6.6million files with a replcount of 2. --Justin Clinton Goudie-Nice wrote: > Dormando was kind enough to point me at some info in the mailing list > from a few weeks ago where people talked about the scale of the MogileFS > stores they're currently using. > > Could anyone mention some names of some reasonably large scale web > sites/services which I might recognize that are using MogileFS? > > Also, I've read a bit about trouble serving files greater than 100mb. Is > there some limit on file size that I should be aware of? > > Thanks again! > > Clint > > > Twelve Horses <http://web.twelvehorses.com/> Mobile > <http://web.twelvehorses.com/info_center/whitepaper/> Social > <http://web.twelvehorses.com/solutions/blogging_for_business/> Web > <http://web.twelvehorses.com/solutions/website_design/> Email > <http://web.twelvehorses.com/technology/> > > > *Clinton Goudie-Nice* > Architect / Senior Software Engineer > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > Phone: +1.801.571.2665 ext 3264 > Mobile: +1.801.915.0629 > Fax: +1.801.571.2669 > LinkedIn: http://www.linkedin.com/in/cgoudie > > > *Twelve Horses* > 13961 Minuteman Drive > Suite 125 > Draper, UT 84020 > *www.twelvehorses.com <http://www.twelvehorses.com>* > > > > > > On Wed, 2007-12-12 at 16:06 -0700, Clinton Goudie-Nice wrote: >> Is anyone on the list using mogile in a large scale production >> environment? >> >> I'm considering running MogileFS in a production environment; sort of >> a giant disk in the sky, for >> an assortment of Internet apps. It will potentially be serving up >> terabytes of data, mostly in small image files, probably behind a >> squid caching proxy. >> >> Is there anything people on the list could comment on? How stable is >> it, how big are your various production stores, etc... >> >> Thanks! >> >> Clint >
