Hi,
To start I wanted to thank Pat for Thinking Sphinx - wonderful bit of open
source.
I have a fairly large pool of items - 2387868 - that are each processed by
1 or more queries. Given that each query takes about .1 seconds to run I'm
using a threaded approach to run them (thread pool style architecture with
each item being popped off a redis queue).
What I'm seeing is a large number of ThinkingSphinx::SphinxError: Too many
connections. Here's a full stack trace:
SELECT * FROM `unique_link_core` WHERE MATCH('@site_id 37 @url_no_www
*\"debit ATM\"*') AND `sphinx_deleted` = 0 LIMIT 0, 20
ThinkingSphinx::SphinxError: Too many connections - SELECT * FROM
`unique_link_core` WHERE MATCH('@site_id 37 @url_no_www *\"debit ATM\"*')
AND `sphinx_deleted` = 0 LIMIT 0, 20; SHOW META
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:93:in
`rescue in query'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:96:in
`query'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:77:in
`query_all'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search/batch_inquirer.rb:17:in
`block in results'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:39:in
`block in take'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/innertube-1.1.0/lib/innertube.rb:138:in
`take'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/connection.rb:37:in
`take'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search/batch_inquirer.rb:16:in
`results'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/inquirer.rb:9:in
`block in call'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in
`block in instrument'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/activesupport-4.1.4/lib/active_support/notifications/instrumenter.rb:20:in
`instrument'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/activesupport-4.1.4/lib/active_support/notifications.rb:159:in
`instrument'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/logger.rb:3:in
`log'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/inquirer.rb:8:in
`call'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/geographer.rb:11:in
`call'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/middlewares/sphinxql.rb:14:in
`call'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/middleware-0.1.0/lib/middleware/runner.rb:31:in
`call'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/middleware-0.1.0/lib/middleware/builder.rb:102:in
`call'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search.rb:77:in
`populate'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search.rb:56:in
`meta'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/masks/pagination_mask.rb:43:in
`total_entries'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search.rb:124:in
`block in method_missing'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search.rb:123:in
`each'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/search.rb:123:in
`method_missing'
from
/home/banks/srv/shared/bundle/ruby/1.9.1/gems/thinking-sphinx-3.2.0/lib/thinking_sphinx/active_record/base.rb:24:in
`search_count'
My db connection pool is set to 1000 in database.yml but I haven't found
any option for setting connections for sphinx itself. Is there some option
I missed?
I'm only running with 10 threads at present (down from 20 when I first saw
this).
Any thoughts? Is Thinking Sphinx threadsafe or do I have a configuration
setting wrong perhaps ?
Appreciated. I'm glad to see your recent trip the states was enjoyable.
Scott
--
You received this message because you are subscribed to the Google Groups
"Thinking Sphinx" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/thinking-sphinx.
For more options, visit https://groups.google.com/d/optout.
