Hi Spike, I think the reason the query is timing out is because of some other failure that is occurring. In your error gist there is a supervisor report @ 19:16:07 about a riak_pipe_vnode_worker dying. There isn't much in the way of explanation, just "fitting_died." Unfortunately, it looks like pipe just swallows the reason the fitting died [1]. The next supervisor report (for riak_core_vode_sup) actually occurred _before_ the previous one according to the timestamp (@ 19:14:27). It looks as if the ETS table riak_core_node_watcher doesn't exist which would be a sign that it's owning process crashed at some point. The question is what caused the riak_core_vnode_watcher to crash? Do you have any other errors in your crash/error.log that might help shed some light on this?
In summary, I believe the search timeout is a symptom of something else. -Ryan [1]: https://github.com/basho/riak_pipe/blob/master/src/riak_pipe_vnode.erl#L625 On Wed, Nov 9, 2011 at 2:46 PM, Spike Gronim <[email protected]> wrote: > Hello, > > My Riak search cluster is timing out very often. I am indexing text > content extracted from web pages containing news articles. My articles > range in size from a few KB to tens of KB. I have put about 4.4 million > articles into Riak for an average article size of 15 KB. The keys are MD5 > ASCII hex hashes and the values are JSON. When I set this system up I > loaded it with 1GB or so of data and played with the search system. > Everything was kosher, it responded quickly and the search relevance was > fine. Now that I've imported 100x as much data I am getting timeouts. For > example the query "steve jobs died" times out. When I put in extremely > specific conjunctive queries like "+steve +jobs +died +cupertino +apple" I > get no results but it runs quickly. While the system is running a query > that will time out I see the coordinator Riak node consuming between one > and two cores worth of CPU. > > How can I configure Riak to stop timing out searches? I am open to > changing my schema and query pattern if that's what I need to do. > > app.config - https://gist.github.com/1352608 > schema - https://gist.github.com/1352616 > selected errors - https://gist.github.com/1c0976ced0f05ef0d5d6 > > Nodes in the cluster: 4 > Hardware: EC2 m1.large with two disks in a RAID-0 on /mnt > Operating system: Linux ip-XXXX 2.6.38-11-virtual #50-Ubuntu SMP Mon Sep > 12 21:51:23 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux > Disk space consumed: > > 66G /mnt/riak/leveldb > 36G /mnt/riak/merge_index > > Disk space available: 800G > > > Spike Gronim > [email protected] > > > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
