Hi Jon, If the map function is only returning '1' for each object there is no need to distinguish between map inputs and reduce inputs.
For example, imagine we have 5 objects in our "test" bucket and the initial batch of results to reduceSum only includes 3 of the objects: [1,1,1] -> reduceSum -> [3] The next time reduceSum is run it is given the remaining 2 map results as well as the previously calculated reduceSum result: [1,1,3] -> reduceSum -> [5] Thanks, Dan Daniel Reverri Developer Advocate Basho Technologies, Inc. [email protected] On Mon, Nov 29, 2010 at 10:53 AM, Jon Brisbin < [email protected]> wrote: > > On Nov 29, 2010, at 12:15 PM, Dan Reverri wrote: > > Note the following from the article: > "The important thing to understand is that the function defining the reduce > phase may be evaluated multiple times, and the input of later evaluations > will include the input of earlier evaluations." > > Based on your defined reduce function it looks like you are trying to count > the number of objects in the "test" bucket. Rather than return the length of > the incoming list it would be better to sum the values of the incoming list > using "reduceSum": > https://github.com/basho/riak_kv/blob/master/priv/mapred_builtins.js#L66 > > > I logged the incoming data and since I'm getting a mixture of objects and > sums (so far) in the reduce input, I did this to account for both kinds of > data coming into the function in a single parameter: > > function(v){ > var count=0; > for(i in v){ > if(typeof v[i] === 'object'){ > count += 1; > } else { > count += v[i]; > } > } > ejsLog('/tmp/mapred.log','r: '+JSON.stringify(v)); > return [count]; > } > > I'm not sure I understand how I can separate the objects from the reduce > return values using reduceSum. > > jb > > > > > Thanks, > Dan > > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > [email protected] > > > On Mon, Nov 29, 2010 at 7:08 AM, Jon Brisbin < > [email protected]> wrote: > >> >> On Nov 29, 2010, at 8:59 AM, David Smith wrote: >> >> > On Mon, Nov 29, 2010 at 7:43 AM, Jon Brisbin >> > <[email protected]> wrote: >> >> I'm working on the Spring Data and Grails Gorm support for Riak and I'm >> seeing some problems running my tests against Riak 0.13. I don't see these >> problems when running 0.12. >> > >> > Can you elaborate on the problems you're seeing, please? >> >> There's two problems I'm seeing, one is with map/reduce, the other is that >> the server will crash under the load the tests put on it (I'll send a >> separate email with steps to reproduce). >> >> Here's an example of a problem I'm seeing in M/R (from an earlier >> message): >> >> I'm seeing some weird behavior with map/reduce in 0.13 versus 0.12 and I'm >> not sure what's going on here. I'm hanging out in the chat room right now >> trying to find out, but basically, if I run this test case against 0.13 I >> get a completely different result than what I get in 0.12: >> >> http://pastie.org/1320874 >> >> In 0.12, I get [1] in 0.13 I get [2]. >> >> The output I'm logging looks like this for 0.12: >> >> 11/23/2010 (13:39:47): >> map={"bucket":"test","key":"test","vclock":"a85hYGBgzGDKBVIsTE+lb2UwJTLmsTLUTEk4xpcFAA==","values":[{"metadata":{"Links":[],"X-Riak-VTag":"5qaBBTXfdEQUyCBTDRRIQ7","content-type":"application/json","X-Riak-Last-Modified":"Tue, >> 23 Nov 2010 19:39:40 >> GMT","X-Riak-Meta":[]},"data":"{\"test\":\"value\",\"integer\":12}\n"}]} >> 11/23/2010 (13:39:47): reduce=[1] >> >> And this in 0.13: >> >> 11/23/2010 (13:46:52): >> map={"bucket":"test","key":"test","vclock":"a85hYGDgzmDKBVIsjDI+7RlMiYx5rAz9ExKO8UGE2ZqTWGZL10AlRCfBJVgYLplNxhQGqmeMbn4EldBEUs/k7LodUxionils9l2oRBGy+cHJ7zGFgeqZk3faQCVOT0aoZzG1PohFmNWr8CxUWGUaijFv+lYiS2QBAA==","values":[{"metadata":{"Links":[],"X-Riak-VTag":"If4qIgutNTerbEmxiKLhR","content-type":"application/json","X-Riak-Last-Modified":"Tue, >> 23 Nov 2010 19:46:44 >> GMT","X-Riak-Meta":[]},"data":"{\"test\":\"value\",\"integer\":12}\n"}]} >> 11/23/2010 (13:46:52): reduce=[] >> 11/23/2010 (13:46:52): reduce=[1,0] >> >> I tried Homebrew (OS X 10.6) and the OS X download packages of both >> versions. >> >> Jon Brisbin >> Portal Webmaster >> NPC International, Inc. >> >> >> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > > > > Jon Brisbin > Portal Webmaster > NPC International, Inc. > > > >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
