The object has to be JSON-encoded to be marshalled into the Javascript VM, and also on the way out if the Accept header indicates application/json. So you have two places where it needs to be encodable into JSON.
On Thu, Jun 23, 2011 at 11:14 AM, Andrew Berman <[email protected]> wrote: > Mathias, > > I thought Riak was content agnostic when it came to the data being > stored? The map phase is not running Riak.mapValuesJson, so why is > the data itself going through the JSON parser? The JSON value > returned by v with all the info is valid and I see the struct atom in > there so mochijson2 can parse it properly, but I'm not clear why > mochijson2 would be coughing at the data part. > > --Andrew > > On Thu, Jun 23, 2011 at 5:32 AM, Mathias Meyer <[email protected]> wrote: > > Andrew, > > > > you're indeed hitting a JSON encoding problem here. BERT is binary data, > and won't make the JSON parser happy when trying to deserialize it, before > handing it into the map phase. You have two options here, and none of them > will involve JavaScript as the MapReduce language. > > > > 1.) Use the Protobuff API, use Erlang functions to return the value or > object (e.g. riak_mapreduce:map_object_value or > riak_kv_mapreduce:map_identity), and then run MapReduce queries with the > content type 'application/x-erlang-binary'. However, you're constrained by > client libraries here, e.g. Ruby and Python don't support this content type > for MapReduce on the Protobuffs interface yet, so you'd either implement > something custom, or resort to a client that does, riak-erlang-client comes > to mind, though it was proven to be possible using the Java client too, > thanks to Russell. See [1] and [2] > > > > 2.) Convert the result from BERT into a JSON-parseable structure inside > an Erlang map function, before it's returned to the client. > > > > The second approach certainly is less restrictive in terms of API usage, > but certainly involves some overhead inside of the MapReduce request itself, > but is less prone to encoding/decoding issues with JSON. > > > > Mathias Meyer > > Developer Advocate, Basho Technologies > > > > [1] > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004447.html > > [2] > http://lists.basho.com/pipermail/riak-users_lists.basho.com/2011-June/004485.html > > > > On Donnerstag, 23. Juni 2011 at 07:59, Andrew Berman wrote: > > > >> Hey Ryan, > >> > >> Here is the error from the sasl log. It looks like some sort of > >> encoding error. Any thoughts on how to fix this? I am storing the > >> data as BERT encoded binary and I set the content-type as > >> application/octet-stream. > >> > >> Thanks for your help! > >> > >> Andrew > >> > >> ERROR REPORT==== 9-Jun-2011::21:37:05 === > >> ** Generic server <0.5996.21> terminating > >> ** Last message in was {batch_dispatch, > >> {map, > >> {jsanon,<<"function(value) {return [value];}">>}, > >> [{struct, > >> [{<<"bucket">>,<<"user">>}, > >> {<<"key">>,<<"LikiWUPJSFuxtrhCYpsPfg">>}, > >> {<<"vclock">>, > >> > >> <<"a85hYGBgzGDKBVIsLKaZdzOYEhnzWBmes6Yd58sCAA==">>}, > >> {<<"values">>, > >> [{struct, > >> [{<<"metadata">>, > >> {struct, > >> [{<<"X-Riak-VTag">>, > >> <<"1KnL9Dlma9Yg4eMhRuhwtx">>}, > >> {<<"X-Riak-Last-Modified">>, > >> <<"Fri, 10 Jun 2011 03:05:11 GMT">>}]}}, > >> {<<"data">>, > >> > >> <<131,108,0,0,0,18,104,2,100,0,6,114,...>>}]}]}]}, > >> <<"user">>,none]}} > >> ** When Server state == > {state,<0.143.0>,riak_kv_js_map,#Port<0.92614>,true} > >> ** Reason for termination == > >> ** {function_clause,[{js_driver,eval_js, > >> [#Port<0.92614>,{error,bad_encoding},5000]}, > >> {riak_kv_js_vm,invoke_js,2}, > >> {riak_kv_js_vm,define_invoke_anon_js,3}, > >> {riak_kv_js_vm,handle_call,3}, > >> {gen_server,handle_msg,5}, > >> {proc_lib,init_p_do_apply,3}]} > >> > >> =CRASH REPORT==== 9-Jun-2011::21:37:05 === > >> crasher: > >> initial call: riak_kv_js_vm:init/1 > >> pid: <0.5996.21> > >> registered_name: [] > >> exception exit: > >> > {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]} > >> in function gen_server:terminate/6 > >> in call from proc_lib:init_p_do_apply/3 > >> ancestors: [riak_kv_js_sup,riak_kv_sup,<0.128.0>] > >> messages: [] > >> links: [<0.142.0>,<0.6009.21>] > >> dictionary: [] > >> trap_exit: false > >> status: running > >> heap_size: 4181 > >> stack_size: 24 > >> reductions: 2586 > >> neighbours: > >> neighbour: > [{pid,<0.6009.21>},{registered_name,[]},{initial_call,{riak_kv_mapper,init,[Argument__1]}},{current_function,{gen,do_call,4}},{ancestors,[riak_kv_mapper_sup,riak_kv_sup,<0.128.0>]},{messages,[]},{links,[<0.5996.21>,<12337.6227.21>,<0.162.0>]},{dictionary,[]},{trap_exit,false},{status,waiting},{heap_size,987},{stack_size,53},{reductions,1043}] > >> =SUPERVISOR REPORT==== 9-Jun-2011::21:37:05 === > >> Supervisor: {local,riak_kv_js_sup} > >> Context: child_terminated > >> Reason: > >> > {function_clause,[{js_driver,eval_js,[#Port<0.92614>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]} > >> Offender: > >> > [{pid,<0.5996.21>},{name,undefined},{mfargs,{riak_kv_js_vm,start_link,undefined}},{restart_type,temporary},{shutdown,2000},{child_type,worker}] > >> > >> On Wed, Jun 22, 2011 at 6:10 PM, Ryan Zezeski <[email protected](mailto: > [email protected])> wrote: > >> > > >> > Andrew, > >> > Maybe you could elaborate on the error? I tested this against master > (commit below) just now with success. > >> > 2b1a474f836d962fa035f48c05452e22fc6c2193 Change dependency to allow > for R14B03 as well as R14B02 > >> > -Ryan > >> > On Wed, Jun 22, 2011 at 7:03 PM, Andrew Berman <[email protected](mailto: > [email protected])> wrote: > >> > > > >> > > Hello, > >> > > I'm having issues link walking using the Map Reduce link function. I > am using HEAD from Git, so it's possible that's the issue, but here is what > is happening. > >> > > I've got two buckets, user and user_email where user_email contains > a link to the user. > >> > > When I run this: > >> > > { > >> > > "inputs": [ > >> > > [ > >> > > "user_email", > >> > > "[email protected] (mailto:[email protected])" > >> > > ] > >> > > ], > >> > > "query": [ > >> > > { > >> > > "link": { > >> > > "bucket": "user", > >> > > "tag": "user" > >> > > } > >> > > } > >> > > ] > >> > > } > >> > > I only get [["user","LikiWUPJSFuxtrhCYpsPfg","user"]] returned. The > second I add a map function, even the simplest one (function(v) { [v] } I > get a "map_reduce error": > >> > > { > >> > > "inputs": [ > >> > > [ > >> > > "user_email", > >> > > "[email protected] (mailto:[email protected])" > >> > > ] > >> > > ], > >> > > "query": [ > >> > > { > >> > > "link": {"bucket":"user", "tag":"user"} > >> > > } > >> > > ,{ > >> > > "map": { > >> > > "language": "javascript", > >> > > "source": "function(v) { return[v]; }" > >> > > } > >> > > } > >> > > ] > >> > > } > >> > > Is this functionality broken? I am following what it says on the > Wiki for the MapRed version of link walking. When I use HTTP link walking, > it works correctly. > >> > > Thanks, > >> > > Andrew > >> > > _______________________________________________ > >> > > riak-users mailing list > >> > > [email protected] (mailto:[email protected]) > >> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >> > >> _______________________________________________ > >> riak-users mailing list > >> [email protected] (mailto:[email protected]) > >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > -- Sean Cribbs <[email protected]> Developer Advocate Basho Technologies, Inc. http://www.basho.com/
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
