I've been working on this. I have it working with an anonymous javascript
function. I was hoping to move it to using the "map_object_value" built-in
erlang function. However, when I attempt to use this function, I get
failures if any of my keys don't exist. Is there a way to construct my map
phase so that it gracefully handles not founds and just returns what it
finds?
Also, using this function, is there a way to return the bucket and key
names? Or can one assume the response order is identical to the order of
inputs?
I have my current constructions below.
Thanks for any help.
Jacques
*Working Function Definition (including not founds)*
private static final OtpErlangAtom FUNCTION_TYPE = new
OtpErlangAtom("jsanon");
private static final OtpErlangBinary FUNCTION_VALUE = new
OtpErlangBinary("function(v){ return [v]; }".getBytes());
private static final OtpErlangTuple MAP_REDUCE_FUNCTION = new
OtpErlangTuple(new OtpErlangObject[] { FUNCTION_TYPE, FUNCTION_VALUE });
*Non-working Function Definition (works fine if all input list exists.
Fails on not founds.)*
private static final OtpErlangAtom FUNCTION_TYPE = new
OtpErlangAtom("modfun");
private static final OtpErlangAtom FUNCTION_MODULE = new
OtpErlangAtom("riak_kv_mapreduce");
private static final OtpErlangAtom FUNCTION_NAME = new
OtpErlangAtom("map_object_value");
private static final OtpErlangTuple MAP_REDUCE_FUNCTION = new
OtpErlangTuple(new OtpErlangObject[] { FUNCTION_TYPE, FUNCTION_MODULE,
FUNCTION_NAME });
*Final construction of Query object for submission (using either of above).*
private static final OtpErlangTuple MAP_REDUCE_PHASE = new
OtpErlangTuple(new OtpErlangObject[]{ATOM_MAP, MAP_REDUCE_FUNCTION,
ATOM_NONE, KEEP_TRUE});
private static final OtpErlangList MAP_REDUCE_PHASES = new
OtpErlangList(new OtpErlangTuple[]{MAP_REDUCE_PHASE});
private static final OtpErlangTuple FULL_QUERY = new OtpErlangTuple(new
OtpErlangObject[] { ATOM_QUERY, MAP_REDUCE_PHASES });
On Tue, Jun 7, 2011 at 7:57 AM, Russell Brown <[email protected]> wrote:
>
> On 7 Jun 2011, at 15:29, Jacques wrote:
>
> Have you had any success with reading the server response?
>
>
> Yes. Sorry I didn't post a reply.
>
> It was trivial (but fraught, see below), I just used the OtpInputStream to
> deserialize the byte array returned from pbc.MapReduceResponse.
>
> Like:
>
> ByteString bs = resp.getContent();
> OtpInputStream is = new OtpInputStream(bs.toByteArray());
> OtpErlangObject result = is.read_any();
> // and then all sorts of looping, sniffing types, unpacking etc
>
>
> Caveats:
>
> You have a pain unpacking any reasonably complex result.
> The ETF specification drops this doozy about "strings":
> http://www.erlang.org/doc/apps/erts/erl_ext_dist.html#id85596<http://www.erlang.org/doc/apps/erts/erl_ext_dist.html>.
> My first test actually returned a list of [0,1,2,3...200] and Jinterface
> helpfully turned that into a string for me.
>
> That aside it is certainly feasible to use Jinterface to
> serialize/deserialize Map/Reduce jobs/results.
>
> Cheers
>
> Russell
>
>
> Thanks,
> Jacques
>
> On Sat, Jun 4, 2011 at 1:19 PM, Russell Brown <[email protected]>wrote:
>
>>
>> On 4 Jun 2011, at 18:22, Jacques wrote:
>>
>> I like the sound of option 3 also. I'll have a look at it this weekend
>>> and get back to you.
>>>
>>
>> Awesome! Thanks. If you can give me a point in the right direction
>> regarding the correct typing approach and what not, I'm up for giving it a
>> shot as well.
>>
>>
>> Ok, I have a half working hack. It isn't pretty 'cos the Jinterface API is
>> verbose. I've hacked the pbc.MapReduceBuilder to encode the job as
>> "application/x-erlang-binary" and submit that, but really this code should
>> be in a separate class, maybe using the output from
>> MapReduceBuilder.getJSON() as input. That way you can get the feature
>> without patching the client.
>>
>> What I haven't done is decode the response from Riak yet. If you want a
>> pointer here is a gist of the (unclean) hack. It could use a lot of work,
>> but it proves the concept:
>>
>> https://gist.github.com/1008293
>>
>> The gist is just the diff so you can apply it as a patch
>> to src/main/java/com/basho/riak/pbc/mapreduce/MapReduceBuilder.java if you
>> want to play with it.
>>
>> You'll have to add Jinterface to your pom too.
>>
>> <!-- jinterface -->
>> <dependency>
>> <groupId>org.erlang.otp</groupId>
>> <artifactId>jinterface</artifactId>
>> <version>1.5.4</version>
>> </dependency>
>>
>> I think it is best to put the result decoding outside the library too. I'm
>> going to hack up a poc for that now, but I thought I'd post what I have thus
>> far.
>>
>> Cheers
>>
>> Russell
>>
>>
>> Thanks again,
>> Jacques
>>
>>
> _______________________________________________
> riak-users mailing list
> [email protected]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
