Hi Daniel,
Sorry for the slow reply.
On 12 Mar 2012, at 11:36, ivenhov wrote:
> Excuse my doubel-posting and replying to my own thread.
> I found this in PBClientAdapter class in fetchIndex method
>
> final MapReduce mr = new IndexMapReduce(this, indexQuery);
>
> mr.addReducePhase(NamedErlangFunction.REDUCE_IDENTITY,
> Args.REDUCE_PHASE_ONLY_1);
> // only return the key, to match the http rest api
> mr.addReducePhase(new JSSourceFunction("function(v) { return
> v.map(function(e) { return e[1]; }); }"), Args.REDUCE_PHASE_ONLY_1);
>
> This means it uses map reduce and JavaScript for protocol buffers.
> Would that be the reason of slow performance with thousands of keys?
> In my case it more performant to list all the keys than do 2i query.
You can measure if this *is* the problem if you are using Riak 1.1.*+ as an
index MapReduce no longer requires a reduce identity call in Riak 1.1 onwards.
Using the java client you can create an Index MapReduce without the reduce
phases:
IndexQuery = new BinIndexQuery(BinIndex.named("idx_key"), BUCKET_NAME,
filter.toString());
client.mapReduce(indexQuery).execute();
Your result will be a list of bucket/key pairs (different from the HTTP API's
list of keys.) If that executes faster the problem is with the reduce phases,
you could even try with just the first reduce phase, and then try with the
second to discover if the JS phase is the issue.
The reason for the extra JS reduce phase was a (perhaps poor?) choice I made.
In order to provide parity between the HTTP and PB API I made the choice to
have the PB call return a list of Keys only (like the HTTP API.) I guess I
could have iterated on the client to reformat the result, or had Jackson do it,
or even just returned different output between the APIs…it is a hard choice and
really the problem is that Riak has a special API for HTTP returning results in
one format, and requires a MapReduce to get at indexes over protocol buffers.
Please let me know if executing an index MapReduce without reduce phases
performs more in line with your expectations.
Cheers
Russell
>
> I'm using java client 1.0.4 and riak cluster 1.1.1
> Any feedback appreciated.
>
> Daniel
>
> --
> View this message in context:
> http://riak-users.197444.n3.nabble.com/Slow-performance-on-getting-via-2i-tp3812664p3818984.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [email protected]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
