Thanks Kevin. Makes sense.
For the record, this is how I have it now:
var reduce = function(v) {
return v.filter(function(n) { return !n.not_found })
}
db.link({ bucket: "highlights", keep: false })
.map({name: 'Riak.mapValuesJson'})
.reduce({source: reduce})
.run([["user_chapters", id]])()
2010/6/15 Kevin Smith <[email protected]>:
> Francisco -
>
> You have perfect timing. This same question came up yesterday in #riak.
>
> The core of the issue is Riak runs map functions on the node hosting the
> relevant data. In the case of a 'not found', there is no hosting node -- the
> data doesn't exist -- so the map function is never invoked for the missing
> entry. The 'not found' shows up in the reduce phase because reduce phases
> execute on the node coordinating the entire map/reduce job. Riak converts
> link walks into map phases which is why broken links also show up as 'not
> found' responses being passed to your reduce phase.
>
> Riak.mapValuesJson contains filtering logic since it could be used in either
> map OR reduce phases. I'd recommend performing 'not found' filtering in your
> reduce phases since that'll be the first opportunity you have to deal with
> the missing data.
>
> --Kevin
> On Jun 15, 2010, at 4:30 AM, francisco treacy wrote:
>
>> Hi all,
>>
>> I have a document with some broken links (i.e. that point to a
>> non-existent key), and want to apply a map/reduce there.
>>
>> var map = function(v, k, args) { ejsLog("/tmp/mr", "hello from map");
>> return [{from: "map"}] }
>>
>> db.link({ bucket: "highlights", keep: false })
>> .map({source: map})
>> .run([["user_chapters", id]])()
>>
>> (That is, apply link+map to users_chapters/id)
>>
>> Result:
>> [{"not_found":{"bucket":"highlights","key":"f542e13e0b5cece1f637be384e685667"}}]
>>
>> Makes sense because that is true. But why don't I get [{from: "map"}],
>> and why is nothing logged?
>>
>> I presume it is just failing at the link phase, which then outputs no
>> keys for the following phases. The "not found" objects should be get
>> rid of at a node level, so I don't think they belong to a reduce
>> phase. Or is that the case?
>>
>> (FWIW i got inspiration from Riak.mapValuesJson which seems to deal
>> with not_founds, but I never actually seem to reach the map phase).
>>
>> What is the recommended approach to gracefully handle broken links? It
>> should be fairly easy to exclude not found results from the resulting
>> map/reduce array.
>>
>> Francisco
>>
>> _______________________________________________
>> riak-users mailing list
>> [email protected]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
