Nope, no delta indexing. The high load is just searches, no updates. Only reads. There are a few updates, but they're not related to the index. Mostly stuff like sessions.
The concurrency we threw at it was about 50 Users. The results are something like this: connections 50 requests 182 replies 154 test-duration 27.187 s And many more runs like the above one, which average out out to about 70 replies in 14 seconds. After a lot of investigation we were advised to use persistent connections to sphinx ... because our main bottleneck is IO contention. i.e. one connection per query. Unfortunately we couldn't get riddle to start up a persistent connection. We tried caching the socket it was spawning in a class variable, but the second time Riddle tries to contact sphinx we get a "sphinx is not responding error" which I think is sphinx killing the connection on it's end. Thanks, Sid. On Jul 16, 1:25 pm, James Healy <[email protected]> wrote: > I don't have any immediate suggestions, but just to complete the > picture: > > - are you running delta indexing? if so, which type? > - is this high load from requests doing sphinx searches, updates to > indexed models, or both? > > Sphinx does fork itself to respond to queries, but that it itself is not > unusual and I wouldn't expect it to drive up load on your server *that* > much. What sort of concurrency are we talking about? How many requests/sec? > > -- James Healy <jimmy-at-deefa-dot-com> Thu, 16 Jul 2009 22:20:55 +1000 > > sid wrote: > > > Hi, > > > We're seeing a situation on our servers, wherein under high > > concurrency, multiple > > instances of searchd are spawned and immediately killed, almost like > > an instance per > > query. Due to this process, we're observing a high load on the server > > 4.something and > > delays in our reply times of over 1/2 seconds. Even though sphinx > > takes only 0.something > > seconds to reply and rails takes 0.0something to process the request. > > > Here's the generated config file we're using. > > > indexer > > { > > } > > > searchd > > { > > address = 127.0.0.1 > > port = 3312 > > log = /path/to/log/searchd.log > > query_log = /path/to/log/searchd.query.log > > pid_file = /path/to/log/sphinx.pid > > max_matches = 500 > > } > > > source deal_core_0 > > { > > ... > > } > > > index deal_core > > { > > source = deal_core_0 > > path = /path/to/db/sphinx/production/deal_core > > morphology = stem_en > > charset_type = utf-8 > > } > > > index deal > > { > > type = distributed > > local = deal_core > > } > > > Any help would be appreciated. > > > Thanks, > > Sid. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
