Yes, I am able to do that, but I feel like this completely defeats the purpose of a link by having to do two different calls. I might as well just store the user id in the data for user_email instead and not use a link at all with your method. What advantage does a link offer at that point?
On Thu, Jun 23, 2011 at 8:55 AM, Jeremiah Peschka <[email protected]> wrote: > HTTP link walking will get you back the data in the way that you'd expect. > It's a two-step process using PBC. MR link phases will give you a list of > [bucket, key, tag] that you can then use to pull back the records from > Riak. > --- > Jeremiah Peschka > Founder, Brent Ozar PLF, LLC > > On Thursday, June 23, 2011 at 8:52 AM, Andrew Berman wrote: > > Ah, ok, that makes sense. One more question, when I use the HTTP link > walking, I do get the data back as expected, so is there a way to > replicate this in a Map-Reduce job or using the Erlang PBC (which I > forgot to mention is what I'm using and the reason I'm not using the > HTTP link walking method)? > > --Andrew > > On Thu, Jun 23, 2011 at 8:39 AM, Mathias Meyer <[email protected]> wrote: > > Andrew, > > the data looks like JSON, but it's not valid JSON. Have a look at the list > that's in the data section (which is your BERT encoded data), the first > character in that list is 131, which is not a valid UTF-8 character, and > JSON only allows valid UTF-8 characters. With a binary-encoded format, > there's always a chance for a control character like that to blow up the > JSON generated before and after the MapReduce code is executed. With JSON, > content agnosticism only goes as far as the set of legal characters allows. > > On a side note, if the data were a valid representation of a string, you > would see it as a string in the log file as well, not just as a list of > numbers. > > Mathias Meyer > Developer Advocate, Basho Technologies > > > On Donnerstag, 23. Juni 2011 at 17:31, Andrew Berman wrote: > > But isn't the value itself JSON? Meaning this part: > > {struct, > [{<<"bucket">>,<<"user">>}, > {<<"key">>,<<"LikiWUPJSFuxtrhCYpsPfg">>}, > {<<"vclock">>, > > <<"a85hYGBgzGDKBVIsLKaZdzOYEhnzWBmes6Yd58sCAA==">>}, > {<<"values">>, > [{struct, > [{<<"metadata">>, > {struct, > [{<<"X-Riak-VTag">>, > <<"1KnL9Dlma9Yg4eMhRuhwtx">>}, > {<<"X-Riak-Last-Modified">>, > <<"Fri, 10 Jun 2011 03:05:11 GMT">>}]}}, > {<<"data">>, > > <<131,108,0,0,0,18,104,2,100,0,6,114,...>>}]}]} > > So the only thing that is not JSON is the data itself, but when I get > the value, shouldn't I be getting the all the info above which is JSON > encoded? > > Thank you all for your help, > > Andrew > > On Thu, Jun 23, 2011 at 8:17 AM, Sean Cribbs <[email protected] > (mailto:[email protected])> wrote: > > The object has to be JSON-encoded to be marshalled into the Javascript VM, > and also on the way out if the Accept header indicates application/json. So > you have two places where it needs to be encodable into JSON. > On Thu, Jun 23, 2011 at 11:14 AM, Andrew Berman <[email protected] > (mailto:[email protected])> wrote: > > Mathias, > > I thought Riak was content agnostic when it came to the data being > stored? The map phase is not running Riak.mapValuesJson, so why is > the data itself going through the JSON parser? The JSON value > returned by v with all the info is valid and I see the struct atom in > there so mochijson2 can parse it properly, but I'm not clear why > mochijson2 would be coughing at the data part. > > --Andrew > > On Thu, Jun 23, 2011 at 5:32 AM, Mathias Meyer <[email protected] > (mailto:[email protected])> wrote: > > Andrew, > > you're indeed hitting a JSON encoding problem here. BERT is binary data, > and won't make the JSON parser happy when trying to deserialize it, before > handing it into the map phase. You have two options here, and none of them > will involve JavaScript as the MapReduce language. > > 1.) Use the Protobuff API, use Erlang functions to return the value or > object (e.g. riak_mapreduce:map_object_value or > riak_kv_mapreduce:map_identity), and then run MapReduce queries with the > content type 'application/x-erlang-binary'. However, you're constrained by > client libraries here, e.g. Ruby and Python don't support this content type > for MapReduce on the Protobuffs interface yet, so you'd either implement > something custom, or resort to a client that does, riak-erlang-client comes > to mind, though it was proven to be possible using the Java client too, > thanks to Russell. See [1] and [2] > > 2.) Convert the result from BERT into a JSON-parseable structure inside > an Erlang map function, before it's returned to the client. > > The second approach certainly is less restrictive in terms of API usage, > but certainly involves some overhead inside of the MapReduce request itself, > but is less prone to encoding/decoding issues with JSON. > > Mathias Meyer > Developer Advocate, Basho Technologies > > [1] > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004447.html > [2] > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004485.html > > On Donnerstag, 23. Juni 2011 at 07:59, Andrew Berman wrote: > > Hey Ryan, > > Here is the error from the sasl log. It looks like some sort of > encoding error. Any thoughts on how to fix this? I am storing the > data as BERT encoded binary and I set the content-type as > application/octet-stream. > > Thanks for your help! > > Andrew > > ERROR REPORT==== 9-Jun-2011::21:37:05 === > ** Generic server <0.5996.21> terminating > ** Last message in was {batch_dispatch, > {map, > {jsanon,<<"function(value) {return [value];}">>}, > [{struct, > [{<<"bucket">>,<<"user">>}, > {<<"key">>,<<"LikiWUPJSFuxtrhCYpsPfg">>}, > {<<"vclock">>, > > <<"a85hYGBgzGDKBVIsLKaZdzOYEhnzWBmes6Yd58sCAA==">>}, > {<<"values">>, > [{struct, > [{<<"metadata">>, > {struct, > [{<<"X-Riak-VTag">>, > <<"1KnL9Dlma9Yg4eMhRuhwtx">>}, > {<<"X-Riak-Last-Modified">>, > <<"Fri, 10 Jun 2011 03:05:11 GMT">>}]}}, > {<<"data">>, > > <<131,108,0,0,0,18,104,2,100,0,6,114,...>>}]}]}]}, > <<"user">>,none]}} > ** When Server state == > {state,<0.143.0>,riak_kv_js_map,#Port<0.92614>,true} > ** Reason for termination == > ** {function_clause,[{js_driver,eval_js, > [#Port<0.92614>,{error,bad_encoding},5000]}, > {riak_kv_js_vm,invoke_js,2}, > {riak_kv_js_vm,define_invoke_anon_js,3}, > {riak_kv_js_vm,handle_call,3}, > {gen_server,handle_msg,5}, > {proc_lib,init_p_do_apply,3}]} > > =CRASH REPORT==== 9-Jun-2011::21:37:05 === > crasher: > initial call: riak_kv_js_vm:init/1 > pid: <0.5996.21> > registered_name: [] > exception exit: > > {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]} > in function gen_server:terminate/6 > in call from proc_lib:init_p_do_apply/3 > ancestors: [riak_kv_js_sup,riak_kv_sup,<0.128.0>] > messages: [] > links: [<0.142.0>,<0.6009.21>] > dictionary: [] > trap_exit: false > status: running > heap_size: 4181 > stack_size: 24 > reductions: 2586 > neighbours: > neighbour: > [{pid,<0.6009.21>},{registered_name,[]},{initial_call,{riak_kv_mapper,init,[Argument__1]}},{current_function,{gen,do_call,4}},{ancestors,[riak_kv_mapper_sup,riak_kv_sup,<0.128.0>]},{messages,[]},{links,[<0.5996.21>,<12337.6227.21>,<0.162.0>]},{dictionary,[]},{trap_exit,false},{status,waiting},{heap_size,987},{stack_size,53},{reductions,1043}] > =SUPERVISOR REPORT==== 9-Jun-2011::21:37:05 === > Supervisor: {local,riak_kv_js_sup} > Context: child_terminated > Reason: > > {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]} > Offender: > > [{pid,<0.5996.21>},{name,undefined},{mfargs,{riak_kv_js_vm,start_link,undefined}},{restart_type,temporary},{shutdown,2000},{child_type,worker}] > > On Wed, Jun 22, 2011 at 6:10 PM, Ryan Zezeski <[email protected] > (mailto:[email protected]) > (mailto:[email protected])> wrote: > > Andrew, > Maybe you could elaborate on the error? I tested this against master > (commit below) just now with success. > 2b1a474f836d962fa035f48c05452e22fc6c2193 Change dependency to allow > for R14B03 as well as R14B02 > -Ryan > On Wed, Jun 22, 2011 at 7:03 PM, Andrew Berman <[email protected] > (mailto:[email protected]) > (mailto:[email protected])> wrote: > > Hello, > I'm having issues link walking using the Map Reduce link function. > I am using HEAD from Git, so it's possible that's the issue, but here is > what is happening. > I've got two buckets, user and user_email where user_email contains > a link to the user. > When I run this: > { > "inputs": [ > [ > "user_email", > "[email protected] (mailto:[email protected])" > ] > ], > "query": [ > { > "link": { > "bucket": "user", > "tag": "user" > } > } > ] > } > I only get [["user","LikiWUPJSFuxtrhCYpsPfg","user"]] returned. The > second I add a map function, even the simplest one (function(v) { [v] } I > get a "map_reduce error": > { > "inputs": [ > [ > "user_email", > "[email protected] (mailto:[email protected])" > ] > ], > "query": [ > { > "link": {"bucket":"user", "tag":"user"} > } > ,{ > "map": { > "language": "javascript", > "source": "function(v) { return[v]; }" > } > } > ] > } > Is this functionality broken? I am following what it says on the > Wiki for the MapRed version of link walking. When I use HTTP link walking, > it works correctly. > Thanks, > Andrew > _______________________________________________ > riak-users mailing list > [email protected] (mailto:[email protected]) > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ > riak-users mailing list > [email protected] (mailto:[email protected]) > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ > riak-users mailing list > [email protected] (mailto:[email protected]) > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > -- > Sean Cribbs <[email protected] (mailto:[email protected])> > Developer Advocate > Basho Technologies, Inc. > http://www.basho.com/ > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
