Hi KIR I appreciate your willingness to fork and patch, so here's my recommended approach:
- for the method_missing, I'd shift that method into a separate module, then include it only when a sphinx scope is added to a model (otherwise, there's no need for it). You'll also want to make sure it doesn't get included more than once. - for the delta callbacks, perhaps add a check to ThinkingSphinx.deltas_enabled? - keep in mind this can be set and unset within a normal request (don't treat it as a constant that may be set when a Rails app is starting up, then never changed). In other words, you'll need to add the normal after_callback, but within the index_delta method (what the callback points to), do nothing if ThinkingSphinx.deltas_enabled? is false. Does this all sound reasonable? Give it a try, would love to know how it impacts the speed. Cheers -- Pat On 22/01/2011, at 5:33 AM, KIR wrote: > Hello, > > First of all, thanks a lot for the library. Great work! > > Second, sorry for lengthy post. It is kind of bugreport, so I want > to provide maximum details. > > I'm adding search support to my project (Checkvist) and I decided > to try using time-based delta indexes (:delta => :datetime). As I > understand, this should eliminate performance overhead for delta > indexes update when the model is updated. > > The price is the need to run a separate rake task to update delta > indexes periodically. Also, search results will include changed data > only after update of the delta indexes. > > I've added search support, wrote search tests (great Cucumber > support for integration testing, BTW!). Everything worked as expected. > > After that I run my old performance tests for basic operations. And, > I've seen noticeable performance degradation for most operations (not > only for write operations, but also for read ones). > > I've made some investigations with the help of NewRelic (in > development mode/profiling). Here are some results: > > 1. read operations were slow when there was many calls to has_many > association methods. The problem is in method_missing method > ThinkingSphinx::ActiveRecord::HasManyAssociation. When a transaction > has hundreds of calls to association methods (like 'all' or 'detect'), > this method_missing gives ~10% slowdown of operations in my tests. > Given that I'm not using shinx_scopes at all (and search on > associations), it would be great to have a way to disable this > method_missing. I should say, that even if this method has stub > implementation (super), the slow down is still there. I was managed to > remove slowdown only when I removed last &block parameter from > method_missing. BTW, will_paginate plugin has much faster approach to > implement method_missing call for association, and it doesn't use > &block parameter. Personally, I'd be glad to remove association > patching at all, just to remove performance penalty. > > 2. write operations gave me ~20% performance penalty in my tests. > Despite the fact that I use delayed indexes which should be updated > only with rake task, there were some additions to model to support > them explicitly (like various before_save and after_commit hooks). And > looks like these operations add more slowness. > > I'm ready to fork thinksphinx and > - make an option which disables search on associations (or, at least, > support of named scopes for them) > - make changes which would suppress various after_commit hooks when > delta indexes are timestamp-based. > > Is it sound reasonable? Or there is something I miss or don't > understand? > > Thanks! > > KIR > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/thinking-sphinx?hl=en. > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
