It might be possible to work around this, depending on how typically
out-of-order the documents are;
Have the reduce object be initially:
{
result: "",
version: 0,
unapplied_deltas: [] //(a sorted list)
}
where each element of unapplied_deltas is:
{
from:M,
to:N,
delta:"..."
}
and the delta documents contain a delta definition, the version number
they expect to apply to, and the resulting version (usually from+1).
So the reduce function would look something like:
//Put the delta in the list, maintaining the sort order
obj.unapplied_deltas = insert_delta( obj.unapplied_deltas, doc );
//Can any deltas be applied?
while(
obj.unapplied_deltas.length > 0 &&
obj.version === obj.unapplied_deltas[0].from
) {
d = obj.unapplied_deltas.shift();
obj.result = apply_delta(obj.result, d);
obj.version = d.to;
}
What do you think?
-Patrick
On 19/06/2011 7:45 AM, Charles Lehner wrote:
Thanks for addressing my question, Dave and Sean. It looks like I can't do what
I was trying to do, which is to store deltas as documents and then use a
time-ordered view to apply the deltas and see the current state of the system,
or its state at a point in the past.
I could do this with a list function, but that would require iterating through
all the documents for each request. More likely I'll make something custom in
node.js.
Charles
On Jun 17, 2011, at 10:18 PM, Dave Cottlehuber<[email protected]> wrote:
On 18 June 2011 14:07, Sean Copenhaver<[email protected]> wrote:
Hmmm... I don't think it's a good idea to have an order sensitive reduce
function. Also I think you are thinking about the actual view index which is
ordered by the key you emit from the map function.
I believe couch will reduce each b-tree node worth of the view and store those,
then rereduce those and parts of nodes as needed so your reduced queries only
do part of the work.
I hope that made some sense.
On Jun 17, 2011, at 9:17 PM, Charles Lehner<[email protected]> wrote:
Hi,
I have a reduce function that is sensitive to row order. My understanding is
that reduces are supposed to get their rows sorted by key. But this one is
getting them out of order; in fact, the order changes inconsistently between
replications.
I'm seeing this on both 1.0.2 and trunk. Is this the expected behavior?
Thanks,
Charles
Sean's spot on; MR should not be sensitive to row order; that's kinda
the point of being able to divide and conquer arbitrarily is that the
result is identical and independent of the order of processing.
http://wiki.apache.org/couchdb/Introduction_to_CouchDB_views#Restrictions_on_map_and_reduce_functions
I'm not sure what you meant by between replications, but after more
data is added to the couch, impacted b-tree leaf and intermediate
nodes get re-written. As its append-only, this continues right to the
top of the tree. So your "row order" changes each time more data is
added, potentially.
Perhaps a list might be more appropriate for your post-processing
sorting? or client side?
A+
Dave
