I am observing an error while running a MR job that I think is related to my
attempt as using binary keys. The error results in a crash. I upgraded to
1.0.0pre2 in the hope that the better support for binary keys would resolve
the problem, but it has not.
The crash and error reports are below. The MP job in question is a simple
proof of concept. It uses a query as input, has a JS inline map function
that maps each object to 1, and uses the builtin Riak.reduceSum function to
generate a count of matching objects.
{
"timeout": 600000,
"inputs": {
"module" : "riak_search",
"function" : "mapred_search",
"arg" : [ "events", "data_fnd:foo"]
},
"query": [
{
"map":
{
"language": "javascript",
"source": "function(obj) { return [1]; }"
},
},
{
"reduce":
{
"language": "javascript",
"name": "Riak.reduceSum"
}
}
]
}
As you can see from the error report, the key is binary, while the data is
encoded in JSON and the char type is 8-bit US ASCII. The JSON is valid JSON.
The error claims the problem js_driver, eval_js and that the problem is
bad encoding.
I am guessing that something is attempting to interpret the binary key as a
UTF8 string or some such thing, which will generate an error as the keys are
not valid in the encoding.
Is this what is happening? And if so, it there any way to use binary keys
with JS functions?
2011-09-14 00:10:58 =CRASH REPORT====
crasher:
initial call: riak_pipe_vnode_worker:init/1
pid: <0.3821.0>
registered_name: []
exception exit: processing_error
in function gen_fsm:terminate/7
in call from proc_lib:init_p_do_apply/3
ancestors:
[<0.202.0>,<0.201.0>,riak_core_vnode_sup,riak_core_sup,<0.91.0>]
messages: []
links: [<0.201.0>,<0.202.0>]
dictionary:
[{eunit,[{module,riak_pipe_vnode_worker},{partition,68507889249886074290797726533575766546371837952},{<0.201.0>,<0.201.0>},{details,{fitting_details,{fitting,<0.3715.0>,#Ref<0.0.0.65882>,follow,1},{xform_map,0},riak_kv_mrc_map,
{{jsanon,<<"function(obj) { return [1];
}">>},none},{fitting,<0.3714.0>,#Ref<0.0.0.65882>,#Fun<riak_kv_mrc_pipe.2.111096700>,1},[{sink,{fitting,<0.110.0>,#Ref<0.0.0.65882>,sink,undefined}},{log,sink},{trace,{set,1,16,16,8,80,48,{[],[],[],[],[]
,[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[error],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}}],64}}]}]
trap_exit: false
status: running
heap_size: 987
stack_size: 24
reductions: 2002
neighbours:
2011-09-14 00:10:58 =SUPERVISOR REPORT====
Supervisor: {<0.202.0>,riak_pipe_vnode_worker_sup}
Context: child_terminated
Reason: processing_error
Offender:
[{pid,<0.3821.0>},{name,undefined},{mfargs,{riak_pipe_vnode_worker,start_link,undefined}},{restart_type,temporary},{shutdown,2000},{child_type,worker}]
2011-09-14 00:10:58 =SUPERVISOR REPORT====
Supervisor: {local,riak_kv_js_sup}
Context: child_terminated
Reason:
{function_clause,[{js_driver,eval_js,[#Port<0.7099>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_ap
ply,3}]}
Offender:
[{pid,<0.3812.0>},{name,undefined},{mfargs,{riak_kv_js_vm,start_link,undefined}},{restart_type,temporary},{shutdown,2000},{child_type,worker}]
2011-09-14 00:10:58 =ERROR REPORT====
** Generic server <0.3822.0> terminating
** Last message in was
{dispatch,{<0.3822.0>,#Ref<0.0.0.67797>},{{jsanon,<<"function(obj) { return
[1]; }">>},[{struct,[{<<"bucket">>,<<"events">>},{
<<"key">>,<<"ND"[132,133,109,165,135,33,59,119,71,77,148,101,73,183,107,167,50,239,3,185]>>}
,{
<<"vclock">>,<<"a85hYGBgzmDKBVIsjMej2jOYEhnzWBnuv3xwnA8qzKwhuhwq3HnnIUyYrTmJVSR1M7JEFgA=">>},{<<"values">>,[{struct,[{<<"metadata">>,{struct,[{<<"X-Riak-VTag">>,<<"1AO1uqa9gpEdppAdAqMdyA">>},{
<<"content-type">>,<<"application/json">>},{<<"X-Ri
ak-Last-Modified">>,<<"Mon, 12 Sep 2011 01:29:45 GMT">>},{
<<"charset">>,<<"ASCII-8BIT">>
}]}},{<<"data">>,<<"{"id":953,"tso":1313113221,"data":{"fnd":"foo"}}">>}]}]}]},{struct,[{<<"data_fnd">>,<<"foo">>},{p,[3]},{score,[1.0]}]},none]}}
** When Server state == {state,<0.253.0>,riak_kv_js_map,#Port<0.7112>,false}
** Reason for termination ==
**
{function_clause,[{js_driver,eval_js,[#Port<0.7112>,{error,bad_encoding},5000]},{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do_apply,3}]}
2011-09-14 00:10:58 =CRASH REPORT====
crasher:
initial call: riak_kv_js_vm:init/1
pid: <0.3822.0>
registered_name: []
exception
exit:{function_clause,[{js_driver,eval_js,[#Port<0.7112>,{error,bad_encoding},5000]}
,{riak_kv_js_vm,invoke_js,2},{riak_kv_js_vm,define_invoke_anon_js,3},{riak_kv_js_vm,handle_call,3},{gen_server,handle_msg,5},{proc_lib,init_p_do
_apply,3}]}
in function gen_server:terminate/6
in call from proc_lib:init_p_do_apply/3
ancestors: [riak_kv_js_sup,riak_kv_sup,<0.172.0>]
messages: []
links: [<0.252.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 10946
stack_size: 24
reductions: 8678
neighbours:
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
