Unfortunately you've run into some unintuitive behavior Erlang has with
anonymous functions. When Erlang sends an anonymous function to a remote VM it
sends a pointer to the function instead of the function itself. This means the
code which creates the anonymous functions needs to be on the server-side
codepath so the VM receiving the pointer can resolve it to runnable code. This
dramatically reduces the utility of anonymous functions in map/reduce jobs, but
it's a limitation of how Erlang handles these types of functions.
If you're just experimenting with the map/reduce API AND you have a single Riak
server (no cluster) then you can get this to work by following these steps:
1) Start riak via 'bin/riak console'
2) Inside the console, obtain a Riak client instance like so: {ok, Client} =
riak:local_client()
3) Build and execute your map/reduce job inside the riak console
Alternatively, you'll need to define your map/reduce functions in a proper
Erlang module and distribute it to all the Riak nodes and insure it's on the
codepath. If you decide to go this route you can use Erlang's hot code loading
to load updated versions of the functions without needing to reboot the Riak
nodes.
--Kevin
On Apr 13, 2010, at 11:58 PM, ben six wrote:
>
> on macos 10.6.3 using riak-0.9.1 following instructions at
> https://wiki.basho.com/display/RIAK/Installing+on+Mac+OS+X
> install and test passed without issue.
> then tried the erland mapreduce example at
> https://wiki.basho.com/display/RIAK/MapReduce#MapReduce-MapReduceviatheErlangAPI
>
> ~/FromFtp/riak/riak-0.9.1/erts-5.7.5/bin/erl -name [email protected]
> -setcookie riak -pa `pwd`/lib/riak-0.9.1.ez/riak-0.9.1/ebin
> Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe]
> [kernel-poll:false]
>
> Eshell V5.7.5 (abort with ^G)
> ([email protected])1> code:which(riak).
> "~/FromFtp/riak/riak-0.9.1/lib/riak-0.9.1.ez/riak-0.9.1/ebin/riak.beam"
> ([email protected])2> {ok, Client} = riak:client_connect('[email protected]').
> {ok,{riak_client,'[email protected]',<<6,202,197,138>>}}
> ([email protected])4> Count = fun(G, undefined, none) ->
> ([email protected])4> [dict:from_list([{I, 1} || I <-
> riak_object:get_value(G)])]
> ([email protected])4> end.
> #Fun<erl_eval.18.105910772>
> ([email protected])5> Merge = fun(Gcounts, none) ->
> ([email protected])5> [lists:foldl(fun(G, Acc) ->
> ([email protected])5> dict:merge(fun(_, X, Y)
> -> X+Y end,
> ([email protected])5> G, Acc)
> ([email protected])5> end,
> ([email protected])5> dict:new(),
> ([email protected])5> Gcounts)]
> ([email protected])5> end.
> #Fun<erl_eval.12.113037538>
> ([email protected])6> {ok, [R]} = Client:mapred([{<<"groceries">>,
> <<"mine">>},
> ([email protected])6> {<<"groceries">>,
> <<"yours">>}],
> ([email protected])6> [{map, {qfun, Count},
> none, false},
> ([email protected])6> {reduce, {qfun,
> Merge}, none, true}]).
> ** exception error: no match of right hand side value "all nodes failed"
>
> above happened when starting riak with console or plain ./bin/riak start
> (riak's erl shell : Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2]
> [async-threads:0] [hipe] [kernel-poll:false])
>
> also error when using erlang shell (from port : Erlang R13B03 (erts-5.7.4)
> [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe]
> [kernel-poll:false]) :
> ::erl -name [email protected] -setcookie riak -pa
> `pwd`/lib/riak-0.9.1.ez/riak-0.9.1/ebin
> Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:2:2] [rq:2]
> [async-threads:0] [hipe] [kernel-poll:false]
>
> Eshell V5.7.4 (abort with ^G)
> ([email protected])1> code:which(riak).
> "~/FromFtp/riak/riak-0.9.1/lib/riak-0.9.1.ez/riak-0.9.1/ebin/riak.beam"
> ([email protected])2> {ok, Client} = riak:client_connect('[email protected]').
> {ok,{riak_client,'[email protected]',<<4,181,78,207>>}}
> ([email protected])4> Count = fun(G, undefined, none) ->
> ([email protected])4> [dict:from_list([{I, 1} || I <-
> riak_object:get_value(G)])]
> ([email protected])4> end.
> #Fun<erl_eval.18.105910772>
> ([email protected])5> Merge = fun(Gcounts, none) ->
> ([email protected])5> [lists:foldl(fun(G, Acc) ->
> ([email protected])5> dict:merge(fun(_, X, Y)
> -> X+Y end,
> ([email protected])5> G, Acc)
> ([email protected])5> end,
> ([email protected])5> dict:new(),
> ([email protected])5> Gcounts)]
> ([email protected])5> end.
> #Fun<erl_eval.12.113037538>
> ([email protected])6> {ok, [R]} = Client:mapred([{<<"groceries">>,
> <<"mine">>},
> ([email protected])6> {<<"groceries">>,
> <<"yours">>}],
> ([email protected])6> [{map, {qfun, Count},
> none, false},
> ([email protected])6> {reduce, {qfun,
> Merge}, none, true}]).
> ** exception error: undefined function luke:new_flow/5
> in function riak_client:mapred_stream/4
> in call from riak_client:mapred/4
>
> i used the default config files so the cookie is also consistant.
> new to this so not sure where to get from there.
> _______________________________________________
> riak-users mailing list
> [email protected]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
