Oleg, What kind of hardware makes up each machine if you don't mind my asking? Also, any other advice you can give would be most appreciated. That would be helpful for anyone like us that is looking at the best way to implement mogilefs.
Warmest regards, Andre' -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Oleg Tyaglo Sent: Tuesday, December 25, 2007 6:17 PM To: Justin Huff; Clinton Goudie-Nice Cc: mogilefs Subject: RE: Questions about MogileFS in a production environment We have quite large mogile storage setup running in production. Our current mogile cluster has 44T of storage where 19T used. Total number of stored files is ~330 millions (x3 copies ~= 1 billion objects) We customized mogile cluster next way: - 8 federated db boxes with backup mysql replication - 2 trackers per db group (16 total) - 23 storage nodes, each has 8 250-320G drives (one for each group) Our performance is - 50 writes per second per whole cluster (tested up to 500 wps) - 120+ reads per second per storage node (~ 3000 reads per second on a whole cluster) - 80% full device recovery (re-replication) usually takes 30 hours with no affect to production performance. We developed custom web gui tool and init.d scripts to manage multiple (8) instances of storage daemons (mogstored) on a given storage node. GUI tool also comes handy with graphical display of usage pattern and device/storage node availability. In addition, we designed custom nagios checks that trigger notifications about tracker/storage node availability along with single drive (device) failure. I hope you find this helpful. Oleg -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Justin Huff Sent: Thursday, December 13, 2007 9:42 AM To: Clinton Goudie-Nice Cc: mogilefs Subject: Re: Questions about MogileFS in a production environment I'm not sure we'd count as 'large scale' -- but we use it at Picnik. We have about 5.7TB, using about 2.7TB. That's about 6.6million files with a replcount of 2. --Justin Clinton Goudie-Nice wrote: > Dormando was kind enough to point me at some info in the mailing list > from a few weeks ago where people talked about the scale of the MogileFS > stores they're currently using. > > Could anyone mention some names of some reasonably large scale web > sites/services which I might recognize that are using MogileFS? > > Also, I've read a bit about trouble serving files greater than 100mb. Is > there some limit on file size that I should be aware of? > > Thanks again! > > Clint > > > Twelve Horses <http://web.twelvehorses.com/> Mobile > <http://web.twelvehorses.com/info_center/whitepaper/> Social > <http://web.twelvehorses.com/solutions/blogging_for_business/> Web > <http://web.twelvehorses.com/solutions/website_design/> Email > <http://web.twelvehorses.com/technology/> > > > *Clinton Goudie-Nice* > Architect / Senior Software Engineer > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > Phone: +1.801.571.2665 ext 3264 > Mobile: +1.801.915.0629 > Fax: +1.801.571.2669 > LinkedIn: http://www.linkedin.com/in/cgoudie > > > *Twelve Horses* > 13961 Minuteman Drive > Suite 125 > Draper, UT 84020 > *www.twelvehorses.com <http://www.twelvehorses.com>* > > > > > > On Wed, 2007-12-12 at 16:06 -0700, Clinton Goudie-Nice wrote: >> Is anyone on the list using mogile in a large scale production >> environment? >> >> I'm considering running MogileFS in a production environment; sort of >> a giant disk in the sky, for >> an assortment of Internet apps. It will potentially be serving up >> terabytes of data, mostly in small image files, probably behind a >> squid caching proxy. >> >> Is there anything people on the list could comment on? How stable is >> it, how big are your various production stores, etc... >> >> Thanks! >> >> Clint >
