You are awesome! Thanks! I've deployed to production and everything works great. I'll be keeping an eye out for my original problem of errors on deployment.
Can't wait for 2.1.1 :-) On Monday, August 12, 2013 5:08:54 AM UTC-7, Pat Allan wrote: > > The change was pretty simple: > > https://github.com/pat/thinking-sphinx/commit/d5249ea0215128d5d1f916ee42882932ddb86ba7 > > > To use it (for the 2.x releases of Thinking Sphinx): > > gem 'thinking-sphinx', '~> 2.1.0', > :git => 'git://github.com/pat/thinking-sphinx.git', > :branch => 'v2', > :ref => '64da4dc7ff' > > And then in an initialiser: > > ThinkingSphinx.persistence_enabled = false > > Will be keen to hear if everything returns to normal for you! > > On 12/08/2013, at 12:19 AM, Ngan Pham wrote: > > > I see... > > > > So I've tested it out and it's a bit of a disaster for my situation. > Some search requests are getting dropped and the server that runs sphinx is > hitting memory limits. I could, in theory, solve the problem by increasing > ram, how ever we have fallback servers...and it would suck to have to > increase those as well. I'll do some research on how people handle large > sphinx requests...like airbnb. > > > > As for my 2 cents: I never felt that there was that much of an issue > with the cost of opening a connection to sphinx for every request. It is > very fast for us currently, and shaving 50-80% off something that already > only takes milliseconds while trading for more memory usage and less > scalability isn't worth for me specifically. It would awesome if you added > a configuration option to disable persistent connections! I will buy you a > beer! :) > > > >> On Aug 10, 2013, at 10:26 PM, Pat Allan > >> <[email protected]<javascript:>> > wrote: > >> > >> And the benefit is faster query times - the socket isn't being > setup/cleaned up on every single search request. From the quick testing I > did back when I made this change, this made a noticeable difference with > Sphinx query times (no numbers at hand, but I think it was at least a 50% > improvement, if not closer to 80%). > >> > >> -- > >> Pat > >> > >>> On 11/08/2013, at 3:15 PM, Ngan Pham wrote: > >>> > >>> Hm…I was afraid of that… > >>> So, we have 9 application servers with 10-20 processes each. At high > traffic times, we'd be seeing 181 processes for searchd? Won't that blow > memory up like crazy? Is this normally something that everyone deals with? > >>> > >>> Just curious…what are the benefits of persisted connection pools? > >>> > >>> As always, thanks for the quick response Pat! > >>> > >>> - Ngan > >>>> On Saturday, August 10, 2013 at 10:11 PM, Pat Allan wrote: > >>>> > >>>> Hi Ngan > >>>> > >>>> There's not really any documentation around the changes, I'm afraid… > but what you're seeing is a process per thread of your app, plus a master > daemon process, due to the persisted connection pool. > >>>> > >>>> All the logic for this connection pool can be found in > ThinkingSphinx::Connection: > >>>> > https://github.com/pat/thinking-sphinx/blob/v2/lib/thinking_sphinx/connection.rb > > >>>> > >>>> The HISTORY file has a list of all relevant changes though - here's > the list of what's changed between 2.0.14 and 2.1.0 - but this and some > delta refactoring are the biggest items: > >>>> > >>>> * Removed plugin support - Thinking Sphinx is now gem-only across all > branches. > >>>> * ThinkingSphinx::Version and the thinking_sphinx:version task have > been removed - it's a gem, it has a version number. > >>>> * Updating Riddle to 1.5.6 or newer. > >>>> * Requires ActiveRecord ~> 2.1 for TS 1.x releases (earlier versions > were considered unsupported a few releases ago). > >>>> * Allow custom Riddle controllers - useful for Flying Sphinx to take > over management of Sphinx daemon/indexing actions. > >>>> * Rejigged delta support to be generic, with local job classes that > provide a clean, simple interface for third-party libraries. > >>>> * Add hooks for anything that needs to happen before indexing (such > as clearing out existing delta jobs). > >>>> * Connection pool for all Sphinx client communication, with new > connections built if there's any connection-related (as opposed to syntax) > issues. > >>>> * Multiple-field search conditions can be done with arrays of field > names as keys in the :conditions hash (Alex Dowad). > >>>> * Removed named capture in regular expressions to maintain MRI 1.8 > support (Michael Wintrant). > >>>> * Support new JDBC configuration style (Kyle Stevens). > >>>> > >>>> -- > >>>> Pat > >>>> > >>>>> On 11/08/2013, at 2:39 PM, Ngan wrote: > >>>>> > >>>>> Hi Pat, > >>>>> > >>>>> Thanks for the help. I tried upgrading to 2.1.0...and I'm noticing > multiple instance of searched running now. Is that normal? Would you be > able to point me to documentation of major changes with 2.1.0? > >>>>> > >>>>> Thanks, > >>>>> Ngan > >>>>> > >>>>> On Friday, August 9, 2013 7:57:51 AM UTC-7, Pat Allan wrote: > >>>>> Hi Ngan > >>>>> > >>>>> In 2.1.0 there's been some patches that deal with these kinds of > errors - TS will now retry searches if an error crops up on the client > connection (which is also persisted per thread in a connection pool, saving > socket setup/teardown time). If an error crops up, a new connection is > made, but if the error persists, it'll still get raised… > >>>>> > >>>>> Give 2.1.0 a spin, see if that helps matters. > >>>>> > >>>>> -- > >>>>> Pat > >>>>> > >>>>>> On 09/08/2013, at 10:46 PM, Ngan wrote: > >>>>>> > >>>>>> Forgot to add the trace: > >>>>>> > >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:438:in > `block in populate' > >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:606:in > `call' > >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:606:in > `retry_on_stale_index' > >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:426:in > `populate' > >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:104:in > `to_a' > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Friday, August 9, 2013 5:45:55 AM UTC-7, Ngan wrote: > >>>>>> Hi, > >>>>>> > >>>>>> We reindex our entire index pretty often (once every 3 minutes) > because we have a pretty small data collection and we don't want to use > delayed delta. I notice however, that every once in a while, when we deploy > our application and it happens to be the same time the reindexing is about > to rotate, we'll get this error "no enabled local indexes to search" every > time we hit sphinx there afterwards. When this happens, we have to restart > our app so that it picks up the new indexes. We are reindexing with rotate > so the existing index should still be there and the rotation should be > seamless. Any ideas on why this happens? And if there's anything to do > about it? > >>>>>> > >>>>>> Thanks, > >>>>>> Ngan > >>>>>> > >>>>>> -- > >>>>>> You received this message because you are subscribed to the Google > Groups "Thinking Sphinx" group. > >>>>>> To unsubscribe from this group and stop receiving emails from it, > send an email to [email protected]. > >>>>>> To post to this group, send email to [email protected]. > >>>>>> Visit this group at http://groups.google.com/group/thinking-sphinx. > > >>>>>> For more options, visit https://groups.google.com/groups/opt_out. > >>>>> > >>>>> > >>>>> -- > >>>>> You received this message because you are subscribed to the Google > Groups "Thinking Sphinx" group. > >>>>> To unsubscribe from this group and stop receiving emails from it, > send an email to [email protected] <javascript:>. > >>>>> To post to this group, send email to > >>>>> [email protected]<javascript:>. > > >>>>> Visit this group at http://groups.google.com/group/thinking-sphinx. > >>>>> For more options, visit https://groups.google.com/groups/opt_out. > >>>> > >>>> -- > >>>> You received this message because you are subscribed to a topic in > the Google Groups "Thinking Sphinx" group. > >>>> To unsubscribe from this topic, visit > https://groups.google.com/d/topic/thinking-sphinx/rPsXVX8wlHU/unsubscribe. > > >>>> To unsubscribe from this group and all its topics, send an email to > [email protected] <javascript:>. > >>>> To post to this group, send email to > >>>> [email protected]<javascript:>. > > >>>> Visit this group at http://groups.google.com/group/thinking-sphinx. > >>>> For more options, visit https://groups.google.com/groups/opt_out. > >>> > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups "Thinking Sphinx" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected] <javascript:>. > >>> To post to this group, send email to > >>> [email protected]<javascript:>. > > >>> Visit this group at http://groups.google.com/group/thinking-sphinx. > >>> For more options, visit https://groups.google.com/groups/opt_out. > >> > >> -- > >> You received this message because you are subscribed to a topic in the > Google Groups "Thinking Sphinx" group. > >> To unsubscribe from this topic, visit > https://groups.google.com/d/topic/thinking-sphinx/rPsXVX8wlHU/unsubscribe. > > >> To unsubscribe from this group and all its topics, send an email to > [email protected] <javascript:>. > >> To post to this group, send email to > >> [email protected]<javascript:>. > > >> Visit this group at http://groups.google.com/group/thinking-sphinx. > >> For more options, visit https://groups.google.com/groups/opt_out. > >> > >> > > > > -- > > You received this message because you are subscribed to the Google > Groups "Thinking Sphinx" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected] <javascript:>. > > To post to this group, send email to > > [email protected]<javascript:>. > > > Visit this group at http://groups.google.com/group/thinking-sphinx. > > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/thinking-sphinx. For more options, visit https://groups.google.com/groups/opt_out.
