I'm try fix this bug.
1. Add fucntion for generating ConsHashCookie in riak_kv_mrc_pipe
const_hash_cookie()->
Random = chash:key_of(now()),
{ok, Ring} = riak_core_ring_manager:get_my_ring(),
NodeCount = length(riak_core_ring:all_members(Ring)),
case riak_core_apl:get_primary_apl(Random,NodeCount,riak_pipe) of
[{{Partition, _Node},_}|_]->
riak_pipe_vnode:hash_for_partition(Partition);
_->
Random
end.
It's taking account of which nodes are up.
2. Remove Hash = chash:key_of(ConstHashCookie) in reduce2pipe function.
After that all work fine.
May be I must write const_hash_cookie more accurate and push this changed to
github?
----- Исходное сообщение -----
От: [email protected]
Кому: "Bryan Fink" <[email protected]>
Копия: "Riak-Users" <[email protected]>
Отправленные: Четверг, 31 Январь 2013 г 21:51:21
Тема: Re: Differences between riak_client and riak_kv_mrc_pipe MapReduce
when one node is down.
Hello Bryan.
I'm detect problem.
Problem is in reduce phase.
1. See riak_kv_mrc_pipe:mr2pipe_phases implementation. It convert MapReduce job
spec to riak_pipe spec.
In this fun created ConstHashCookie as Now = now(), and use it as chashfun
value for fitting in reduce phase.
This generated value actually used in riak_kv_w_reduce:done function, you try
make prereduce not reduced data and send to output.
But output vnode in that case is
preflist for ConstHashCookie,i.e. some random value and n_val for this phase is
always 1, that why sometimes calculated perflist for this phase is empty.
Do you have any suggestion how we can fix it?
Thanks,
Alexander Gunin.
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
