On Mon, Oct 27, 2008 at 11:53 AM, Ben Nevile <[EMAIL PROTECTED]> wrote:
> > > > First off to alay your main concern, view indexes are not completely > > regenerated on each update. Its only a diff. > > > > Presumably reduce operations have to operate on the entire set every time? > > Ben > I may be completely off base here, but that's where rereduce comes. I'd done a little profiling on reduce a while back, and it looks as though it lumps reductions into manageable chunks of approximately 20 or so emitted map records. I don't know what kind of magic it does behind the scenes to determine when to tease apart these chunks, but something's going on where only affected records get broken apart, otherwise the whole chunk gets passed through as an object. So yes, a reduce operates on the whole set, but in very large summary chunks. Someone correct me if I'm wrong (please!) -- I've been scratching my head about the internals of reduce for a while now, and still haven't found a very illuminating description of the process.
