Hi, We are facing 500 Internal Server Error for a search queries and amazingly it fails only in cluster mode (i.e. with more than one riak node running).
We have following single node setup to start with: $ ./bin/riak-admin member-status ================================= Membership ================================== Status Ring Pending Node ------------------------------------------------------------------------------- valid 100.0% -- '[email protected]' ------------------------------------------------------------------------------- Valid:1 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 At this point following query works perfectly: http://172.17.3.82:8098/solr/documents/select?wt=json&q=to:e30cff3f4a500fe5826a14019f268f1147fe6993 AND tags:(conversation_message) AND ts_num:([1354872576 TO 1354872578]) We proceed to add another node to setup a cluster: $ ./bin/riak-admin member-status ================================= Membership ================================== Status Ring Pending Node ------------------------------------------------------------------------------- valid 50.0% -- '[email protected]' valid 50.0% -- '[email protected]' ------------------------------------------------------------------------------- Valid:2 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 At this point the above query starts to fail with 500 error code. We checked upon the logs and found following inside 172.17.3.63 error.log file: 2012-12-10 10:05:01.076 [error] <0.3968.0>@mi_server:handle_info:524 lookup/range failure: {{badfun,#Fun<riak_search_client.9.6802141>},[{mi_server,iterate,6,[{file,"src/mi_server.erl"},{line,657}]},{mi_server,lookup,8,[{file,"src/mi_server.erl"},{line,632}]}]} 2012-12-10 10:05:01.082 [error] emulator Error in process <0.4290.0> on node '[email protected]' with exit value: {{badfun,#Fun<riak_search_client.9.6802141>},[{mi_server,iterate,6,[{file,"src/mi_server.erl"},{line,657}]},{mi_server,lookup,8,[{file,"src/mi_server.erl"},{line,632}]}]} looks like search merge index backend is failing with reason badfun. I tried looking around the mi_server.erl module but could get much out of it. Inside 172.17.3.82 error.log file we see following logs which gets propagated as query response: 2012-12-10 10:06:00.987 [error] <0.4622.0> webmachine error: path="/solr/documents/select" {error,{error,{case_clause,timeout},[{riak_search_client,search_doc,8},{riak_search_utils,run_query,7},{riak_solr_searcher_wm,to_json,2},{webmachine_resource,resource_call,3},{webmachine_resource,do,3},{webmachine_decision_core,resource_call,1},{webmachine_decision_core,decision,1},{webmachine_decision_core,handle_request,2}]}} Can we do something about this? Is this related to our configuration and environment setup? Has someone experienced this before? I will be happy to share more info if required. We are using riak-1.2.1 stable release. Thanks in advance. -- Abhinav Singh http://abhinavsingh.com/
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
