Hi Dan, And thanks for the reply. But if we want to get all the keys from the bucket then is there any other way to do that?? I am using Erlang client.
//Wilson On Mon, May 2, 2011 at 6:42 AM, Dan Reverri <[email protected]> wrote: > The error message you posted indicates the list keys operation is timing > out. The list keys operation is an expensive operation and should only be > used during development or for diagnostic purposes. It should not be used > during a load test. > > Thanks, > Dan > > Daniel Reverri > Developer Advocate > Basho Technologies, Inc. > [email protected] > > > > On Fri, Apr 29, 2011 at 5:57 AM, Wilson Tuladhar > <[email protected]>wrote: > >> Hi, >> >> Here the error that get logged into sasl-error.log file. >> >> =ERROR REPORT==== 29-Apr-2011::14:45:37 === >> ** State machine <0.3117.0> terminating >> ** Last event in was timeout >> ** When State == initialize >> ** Data == >> {state,<0.3101.0>,plain,<<>>,undefined,undefined,undefined, >> undefined,<<"Games">>, >> {<<"Games">>,[]}, >> 60000,69622590, >> {chstate,'[email protected]', >> [{'[email protected]',{4,63470357906}}], >> {64, >> [{0,'[email protected]'}, >> {22835963083295358096932575511191922182123945984, >> '[email protected]'}, >> {45671926166590716193865151022383844364247891968, >> '[email protected]'}, >> {68507889249886074290797726533575766546371837952, >> '[email protected]'}, >> {91343852333181432387730302044767688728495783936, >> '[email protected]'}, >> {114179815416476790484662877555959610910619729920, >> '[email protected]'}, >> {137015778499772148581595453067151533092743675904, >> '[email protected]'}, >> {159851741583067506678528028578343455274867621888, >> '[email protected]'}, >> {182687704666362864775460604089535377456991567872, >> '[email protected]'}, >> {205523667749658222872393179600727299639115513856, >> '[email protected]'}, >> {228359630832953580969325755111919221821239459840, >> '[email protected]'}, >> {251195593916248939066258330623111144003363405824, >> '[email protected]'}, >> {274031556999544297163190906134303066185487351808, >> '[email protected]'}, >> {296867520082839655260123481645494988367611297792, >> '[email protected]'}, >> {319703483166135013357056057156686910549735243776, >> '[email protected]'}, >> {342539446249430371453988632667878832731859189760, >> '[email protected]'}, >> {365375409332725729550921208179070754913983135744, >> '[email protected]'}, >> {388211372416021087647853783690262677096107081728, >> '[email protected]'}, >> {411047335499316445744786359201454599278231027712, >> '[email protected]'}, >> {433883298582611803841718934712646521460354973696, >> '[email protected]'}, >> {456719261665907161938651510223838443642478919680, >> '[email protected]'}, >> {479555224749202520035584085735030365824602865664, >> '[email protected]'}, >> {502391187832497878132516661246222288006726811648, >> '[email protected]'}, >> {525227150915793236229449236757414210188850757632, >> '[email protected]'}, >> {548063113999088594326381812268606132370974703616, >> '[email protected]'}, >> {570899077082383952423314387779798054553098649600, >> '[email protected]'}, >> {593735040165679310520246963290989976735222595584, >> '[email protected]'}, >> {616571003248974668617179538802181898917346541568, >> '[email protected]'}, >> {639406966332270026714112114313373821099470487552, >> '[email protected]'}, >> {662242929415565384811044689824565743281594433536, >> '[email protected]'}, >> {685078892498860742907977265335757665463718379520, >> '[email protected]'}, >> {707914855582156101004909840846949587645842325504, >> '[email protected]'}, >> {730750818665451459101842416358141509827966271488, >> '[email protected]'}, >> {753586781748746817198774991869333432010090217472, >> '[email protected]'}, >> {776422744832042175295707567380525354192214163456, >> '[email protected]'}, >> {799258707915337533392640142891717276374338109440, >> '[email protected]'}, >> {822094670998632891489572718402909198556462055424, >> '[email protected]'}, >> {844930634081928249586505293914101120738586001408, >> '[email protected]'}, >> {867766597165223607683437869425293042920709947392, >> '[email protected]'}, >> {890602560248518965780370444936484965102833893376, >> '[email protected]'}, >> {913438523331814323877303020447676887284957839360, >> '[email protected]'}, >> {936274486415109681974235595958868809467081785344, >> '[email protected]'}, >> {959110449498405040071168171470060731649205731328, >> '[email protected]'}, >> {981946412581700398168100746981252653831329677312, >> '[email protected]'}, >> {1004782375664995756265033322492444576013453623296, >> '[email protected]'}, >> {1027618338748291114361965898003636498195577569280, >> '[email protected]'}, >> {1050454301831586472458898473514828420377701515264, >> '[email protected]'}, >> {1073290264914881830555831049026020342559825461248, >> '[email protected]'}, >> {1096126227998177188652763624537212264741949407232, >> '[email protected]'}, >> {1118962191081472546749696200048404186924073353216, >> '[email protected]'}, >> {1141798154164767904846628775559596109106197299200, >> '[email protected]'}, >> {1164634117248063262943561351070788031288321245184, >> '[email protected]'}, >> {1187470080331358621040493926581979953470445191168, >> '[email protected]'}, >> {1210306043414653979137426502093171875652569137152, >> '[email protected]'}, >> {1233142006497949337234359077604363797834693083136, >> '[email protected]'}, >> {1255977969581244695331291653115555720016817029120, >> '[email protected]'}, >> {1278813932664540053428224228626747642198940975104, >> '[email protected]'}, >> {1301649895747835411525156804137939564381064921088, >> '[email protected]'}, >> {1324485858831130769622089379649131486563188867072, >> '[email protected]'}, >> {1347321821914426127719021955160323408745312813056, >> '[email protected]'}, >> {1370157784997721485815954530671515330927436759040, >> '[email protected]'}, >> {1392993748081016843912887106182707253109560705024, >> '[email protected]'}, >> {1415829711164312202009819681693899175291684651008, >> '[email protected]'}, >> {1438665674247607560106752257205091097473808596992, >> '[email protected]'}]}, >> {dict,4,16,16,8,80,48, >> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}, >> {{[],[],[],[], >> [[{bucket,<<"Games">>}| >> {meta_entry, >> [{name,<<"Games">>}, >> {n_val,3}, >> {allow_mult,false}, >> {last_write_wins,false}, >> {precommit, >> [{struct, >> [{<<"mod">>,<<"riak_search_kv_hook">>}, >> {<<"fun">>,<<"precommit">>}]}]}, >> {postcommit,[]}, >> >> {chash_keyfun,{riak_core_util,chash_std_keyfun}}, >> {linkfun, >> >> {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}}, >> {old_vclock,86400}, >> {young_vclock,20}, >> {big_vclock,50}, >> {small_vclock,10}, >> {r,quorum}, >> {w,quorum}, >> {dw,quorum}, >> {rw,quorum}], >> 63470357892}]], >> [],[], >> [[{bucket,<<"User">>}| >> {meta_entry, >> [{name,<<"User">>}, >> {n_val,3}, >> {allow_mult,false}, >> {last_write_wins,false}, >> {precommit, >> [{struct, >> [{<<"mod">>,<<"riak_search_kv_hook">>}, >> {<<"fun">>,<<"precommit">>}]}]}, >> {postcommit,[]}, >> >> {chash_keyfun,{riak_core_util,chash_std_keyfun}}, >> {linkfun, >> >> {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}}, >> {old_vclock,86400}, >> {young_vclock,20}, >> {big_vclock,50}, >> {small_vclock,10}, >> {r,quorum}, >> {w,quorum}, >> {dw,quorum}, >> {rw,quorum}], >> 63470357885}], >> [{bucket,<<"Game_instances">>}| >> {meta_entry, >> [{name,<<"Game_instances">>}, >> {n_val,3}, >> {allow_mult,false}, >> {last_write_wins,false}, >> {precommit, >> [{struct, >> [{<<"mod">>,<<"riak_search_kv_hook">>}, >> {<<"fun">>,<<"precommit">>}]}]}, >> {postcommit,[]}, >> >> {chash_keyfun,{riak_core_util,chash_std_keyfun}}, >> {linkfun, >> >> {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}}, >> {old_vclock,86400}, >> {young_vclock,20}, >> {big_vclock,50}, >> {small_vclock,10}, >> {r,quorum}, >> {w,quorum}, >> {dw,quorum}, >> {rw,quorum}], >> 63470357900}]], >> [],[],[],[],[],[],[], >> [[{bucket,<<"Game_instances_run">>}| >> {meta_entry, >> [{name,<<"Game_instances_run">>}, >> {n_val,3}, >> {allow_mult,false}, >> {last_write_wins,false}, >> {precommit, >> [{struct, >> [{<<"mod">>,<<"riak_search_kv_hook">>}, >> {<<"fun">>,<<"precommit">>}]}]}, >> {postcommit,[]}, >> >> {chash_keyfun,{riak_core_util,chash_std_keyfun}}, >> {linkfun, >> >> {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}}, >> {old_vclock,86400}, >> {young_vclock,20}, >> {big_vclock,50}, >> {small_vclock,10}, >> {r,quorum}, >> {w,quorum}, >> {dw,quorum}, >> {rw,quorum}], >> 63470357906}]]}}}}, >> undefined} >> ** Reason for termination = >> ** {timeout,{gen_server,call, >> [{riak_kv_keylister_master,'[email protected] >> '}, >> >> {start_kl,69622590,<0.3117.0>,{<<"Games">>,[]}}]}} >> >> =CRASH REPORT==== 29-Apr-2011::14:45:38 === >> crasher: >> initial call: riak_kv_keys_fsm:init/1 >> pid: <0.3117.0> >> registered_name: [] >> exception exit: {timeout,{gen_server,call,[{riak_kv_keylister_master, >> [email protected]},{start_kl,69622590,<0.3117.0>,{<<"Games">>,[]}}]}} >> in function gen_fsm:terminate/7 >> in call from proc_lib:init_p_do_apply/3 >> ancestors: [<0.3110.0>] >> messages: [] >> links: [] >> dictionary: [] >> trap_exit: true >> status: running >> heap_size: 610 >> stack_size: 24 >> reductions: 9906 >> neighbours: >> >> >> And the functions that i have made are >> %% gets all the keys in the bucket >> get_bucket_keys(Bucket) -> >> {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087), >> {ok, List} = riakc_pb_socket:list_keys(Pid, Bucket,infinity), >> riakc_pb_socket:stop(Pid), >> {ok,lists:foldr(fun(Key, Acc) -> >> [binary_to_list(Key)|Acc] >> end, [], List)}. >> >> %% select the Key and get the values to return >> select(DB, Key, Values_to_return) -> >> {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087), >> Ret = >> case riakc_pb_socket:get(Pid, list_to_binary(DB), >> list_to_binary(Key)) of >> {ok, FromDBObject} -> >> Val = to_list(binary_to_term(riakc_obj:get_value(FromDBObject))), >> riakc_pb_socket:stop(Pid), >> Return = >> lists:foldr( >> fun(Key_for_return, Acc)-> >> case proplists:get_value(Key_for_return, Val) of >> undefined -> Acc; >> Sth -> [{Key_for_return,Sth} | Acc] >> end >> end, [], Values_to_return), >> {ok, Return}; >> {error, Msg} -> >> {error, Msg} >> end, >> Ret. >> >> And I do the following to get all the keys in the bucket and then find >> some field from that key and return the value. >> {ok, Key_list} = db_api:get_bucket_keys("Games"), >> lists:foldl( >> fun(Key, Acc) -> >> case db_api:select("Games", Key, ["game_name"]) of >> {ok, []} -> >> Acc; >> {ok, List} -> [[{"game_id", Key} | List]|Acc] >> end >> end,[], Key_list). >> >> >> It works for like around 2-3 seconds then it starts to timeout and >> eventually crash.. >> >> //Wilson >> >> >> On Fri, Apr 29, 2011 at 2:01 PM, Rusty Klophaus <[email protected]> wrote: >> >>> Hi Wilson, >>> >>> Do you see any errors or otherwise abnormal looking messages in the logs? >>> That will help diagnose the issue. >>> >>> The max requests per second depends very much on your hardware, size of >>> data, and read/write patterns. >>> >>> Thanks, >>> Rusty >>> >>> On Fri, Apr 29, 2011 at 6:44 AM, Wilson Tuladhar < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> I am using riaksearch with erlang client and i have made an API that >>>> does for the select, insert and other such stuffs. I have built an web >>>> application on top of it and everything works fine when i do the manual >>>> testing. >>>> >>>> But now when i am using the automated load testing through Tsung by >>>> sending 100 req/sec, the riak shuts down after few seconds. Does anybody >>>> know why this is happening or how much requests can riak handle in a >>>> second. >>>> I am using only single node currently. >>>> >>>> //Wilson >>>> >>>> _______________________________________________ >>>> riak-users mailing list >>>> [email protected] >>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>>> >>> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
