For a 100,000 document DB that I was trying to work on, I spent a little time tweaking main.js o reduce object creation (especially arrays) as well as toJSON which creates an inner closure object on each invocation. I'm definitely getting some speed-ups here though I haven't benchmarked to get exact numbers.
Some of the tweaks I did were: - Cache all the regex's created in toJSON - Create a const map object that toJSON uses to minimize object creation - Instead of reinitializing map_results to [], just set the length to zero - This last one, I looked at the spidermonkey's code to see what happens when you do this and it seems to be pretty efficient in truncating arrays. - One another possibility that I've been thinking is to move sum into couch_js.c so it's native C. This is a very commonly used function that it deserves to be fast. Intuitively, no matter what language, object creation has a price to pay. Anything we can do to cache objects and not have to create them repeatedly helps the GC tremendously (million documents * #objects created to map/reduce them). I'm attaching what I have (not a diff, just the final main.js) and I didn't spend the time to try each change and measure the results. YMMV and glad that one change helped. K. On Wed, Apr 1, 2009 at 12:30 PM, Paul Davis <[email protected]> wrote: > On Wed, Apr 1, 2009 at 3:23 PM, Manjunath Somashekhar > <[email protected]> wrote: >> >> hi All, >> >> Would like to thank you all for such fantastic participation. Got to learn a >> lot apart from the question i asked :). >> >> I did make a change as kowsik suggested - although aware of the include_docs >> feature nvr thought it would make such a big difference. The results are >> pretty amazing - the view generation completes in about 25 mins - just one >> execution. Will need to perform a few more runs before getting a good >> average time. >> > > Wow, that's way beyond what I would've expected. Do please try and get > some solid numbers on this. Perhaps we should start a wiki section to > try and gather some number reports for reference? > >> Thanks a lot. Do let me know if any more optimizations can be done. >> >> Thanks >> Manju >> >> >> >> >
main.js
Description: JavaScript source
