My first (*sniff*) MogileFS cluster was built on the 1.x series and was
built and data migrating to it in four days. It had ten million files
hosted by the fourth day of operation after that. It grew to many times
that over two years. Note that we saved time because gaia had already
been running on perlbal for a while, which was from another four day
project a while earlier.
MogileFS is definitely a bit of work up front. The effort to get it in
place at gaia was a combined effort with two members of the developer
staff and myself. Then there were tweaks/fixes along the way. A day
here, a day there.
That was before mogilefs started having options (lighttpd, apache, dav
backends) and we didn't have a choice over the hardware it went on.
So if you want to research the best storage hosts, the best database
setup etc, factor that all in.
Every couple months I'd spend a few days on it to fix some bugs.
Occasionally something will break and you will have to focus on it, but
it doesn't require constant babysitting.
That said, please use the mail archives, submit bugs, patches,
questions, etc if you decide to further research mogilefs and dig more
into it.
-Dormando
Lance Reed wrote:
I am working out the Total Cost for a large (er, scalable...) mogilefs
deployment. The numbers for hardware are relatively easy depending on
design. However, like all opensource initiatives, calculating the time
/ resources for administration by humans is tricky until you do it for
awhile. Our company has plenty of experience using OpenSource tools for
the majority of our needs, with the proprietary solution thrown in where
applicable. So I was wondering if anyone who is attempting to setup a
mogilefs storage setup, or who is using one now could shed some light on
the subject. I figured initially, on a new deployment, about 60% of a
single Administrators time would be taken up to maintain a large
mogilefs cluster. I figure design and implementation would take much
more than that (maybe 2 or 3 people to save time). I am comparing this
to a couple proprietary solutions we have been looking at that are known
to scale well into multi TB or even PB setup that are basically Scalable
NAS solutions that use a "brick" mentality. The theory being that you
pay upfront for stability or support if needed. But I am wondering if
say 60% for the first year of learning how to maintain and grow a
production mogilefs is close or not. Too high / low?
Any thoughts or real work data would be great.
Thanks all!!
