On 26 Oct 2009, at 17:08, Chris Anderson wrote:
On Mon, Oct 26, 2009 at 7:58 AM, till <[email protected]> wrote:
Hey guys,
I've tested this a couple times with smaller database and so far, I
always had to rebuild my views when I updated a server from 0.9.x to
0.10.0. And I also had that confirmed (by some of you). ;-)
Now my problem is the following -- last time I build the view on the
database, it took a week to finish, literally 7.5 days. The view code
itself is pretty simple, but the database contains a 160,000,000
documents and totals to ~200 GB in size. With a set of this
magnitude,
I'd like to avoid another week. I'd even like to avoid 3-4 days (even
though I'm a fan, I don't think it's twice as fast).
So my question is if there is *any* way to avoid view rebuilding and
still updating to 0.10.x.
The principle difference between the 0.9.x code and the 0.10.x code is
the view index filename. When you update to 0.10.0, and start couchdb
building the view, it will create a file named after a hash of the
ddoc.views contents. If you restart the new couch, and rename the 0.9
view file to the hash name, and then start 0.10.0, there's a chance it
could work.
Obviously this should be tested, and perhaps Jan already tried that.
There may be differences is the header format between versions that
should accounted for. This part of the patch (if it is needed) will be
more challenging.
I did a little patch for Till that looks for 0.9-filenames when it
couldn't
find the 0.10 md5 file. That worked until 0.10 found out that the header
didn't match the 0.10 format and reset the index.
I stopped short of diving into the update_db_header() function.
Someone with a little time on his/her hands and some erlang-fu
can pull this off pretty easily. My code so far is on GitHub:
http://github.com/janl/couchdb/tree/zero-nine-indexes-for-oh-ten
I'd be in favour of adding this to 0.10.1.
Cheers
Jan
--
All in all, I'm guessing this is doable for 0.10.1, but it's not
guaranteed to be easy.
We can' allow this kinda thing between beta releases, and 0.10.1 is a
good chance to get the 0.9 update process solid.
Hope that helps,
Chris
ps. Damien did the 4-5x map/reduce speedup on 0.11 (current trunk), so
if you're gonna do a big rebuild, that's worth looking at.
Devs - what we need is a callback that modules can use to do things
like rename files on upgrade.
Thanks for listening,
Till
--
Chris Anderson
http://jchrisa.net
http://couch.io
