Hi Emil As usual, I didn't think about multi-server deployments. Considering that the rest of Thinking Sphinx isn't ideally built for that situation, I think for the moment I'll leave the code as it is. But in the future, you're right, it's definitely something that needs some work.
Thanks for the feedback. Cheers -- Pat On 28/10/2008, at 7:29 PM, Emil Tin wrote: > > Sounds great, thanks for looking into this. > > One thing though - it sounds like you might be checking the pid file > to see if the daemon is running? That will not work in a multi-server > setup, where the daemon is running in a separate machine. The daemon > could also be unreachable due to whatever network issues etc. > > As look at it, the problem is that "cant connect to search daemon" > exceptions can be thrown anywhere in the rails app (since it happens > whenever a model is destroyed or uppdated), This makes it impossible > to handle these exceptions in a meaningful way. > Therefore I would suggest simply never throwing "cant connect to > search deamon" exceptions when trying to update delta indexes or > changing the delete flag, etc. Instead make sure that a simple > reindexing will get everything updated correctly. > > When I perform a search however, throwing the exception still makes > perfect sense - I know I'm doing a search, and can handle the > situation in a meaningful way. > > Deleting a model might have very high importance. Updating the delta > has less importance, and will be fixed by the next reindexing anyways. > Therefore I feel that deletes should never fail just because the > search engine in unavailable. > > > Thanks you for your work! > Emil Tin > > > > > > > On Oct 26, 1:12 pm, Pat Allan <[EMAIL PROTECTED]> wrote: >> HiEmil >> >> Just added something to Thinking Sphinx so it doesn't try to talk to >> Sphinx if it isn't running when a model is destroyed. >> Thanks for your patience with this. >> >> Cheers >> >> -- >> Pat >> >> On 19/10/2008, at 9:05 PM, Pat Allan wrote: >> >> >> >>> HiEmil >> >>> You raise a fair point about problems for model changes while Sphinx >>> is being restarted... I don't have a neat solution for that yet - >>> although I guess the system could check for the existance of the pid >>> file, perhaps? And not try to update to Sphinx if that doesn't >>> exist... Hmm. I'll see what I can do. >> >>> Cheers >> >>> -- >>> Pat >> >>> On 19/10/2008, at 7:04 PM,EmilTin wrote: >> >>>> during restart of the sphinx daemon i will have to take the site >>>> offline to avoid potential problems in case destroying a models >>>> fail >>>> because the deamon can't be contacted. >> >>>> what would happpen if the delete flag is not set when a model is >>>> destroyed? would a later search then cause errors, or simply return >>>> empty records that could be filtered out? >> >>>> what would happen if the delta index is not updated when a model is >>>> changed? nothing except search results would be a bit out of date, >>>> right? as soon as the db is reindexed, things would be fine again? >> >>>> i'm considering patching my copy of ts so i would get get warnings >>>> (perhaps by email) instead of exceptions if the search daemon can't >>>> be >>>> reached when deleting/changing models. >> >>>> On Oct 18, 4:41 pm, Pat Allan <[EMAIL PROTECTED]> wrote: >>>>> That's correct - any change of structure (as opposed to data) >>>>> requires >>>>> Sphinx to be restarted. >> >>>>> -- >>>>> Pat >> >>>>> On 17/10/2008, at 12:51 AM,EmilTin wrote: >> >>>>>> hi pat, >>>>>> thanks you for reply. >>>>>> i trust that the daemon is very stable, but as far as i >>>>>> understood >>>>>> it, >>>>>> whenever i change the define_index'es i have to stop, >>>>>> reconfigure, >>>>>> reindex and restart sphinx? >> >>>>>> On Oct 16, 6:13 am, Pat Allan <[EMAIL PROTECTED]> wrote: >>>>>>> HiEmil >> >>>>>>> Thinking Sphinx doesn't just talk to Sphinx to get deltas >>>>>>> indexed. >>>>>>> For >>>>>>> example, when a model instance is destroyed, the following >>>>>>> things >>>>>>> happen: >> >>>>>>> - Check if the instance exists in the core index >>>>>>> - Flags it as deleted in the core index if it does >>>>>>> - If deltas are enabled and the instance had delta set to true, >>>>>>> flag >>>>>>> it as deleted in the delta index >> >>>>>>> The first two steps happen every time, whether you're using >>>>>>> delta >>>>>>> indexes or not. This is to make sure those deleted records don't >>>>>>> get >>>>>>> returned in search queries. >> >>>>>>> And ideally, the Sphinx daemon should be running constantly. >>>>>>> From >>>>>>> what >>>>>>> I've found (and heard from others), the daemon is very stable, >>>>>>> and >>>>>>> rarely crashes. And when it does, that's where process tracking >>>>>>> tools >>>>>>> like God and Monit can be useful. >> >>>>>>> Hope this clears things up a bit. >> >>>>>>> -- >>>>>>> Pat >>>>>>> e: [EMAIL PROTECTED] || m: +614 1327 3337 >>>>>>> w:http://freelancing-gods.com|| t: twitter.com/pat >>>>>>> discworld:http://ausdwcon.org|| skype: patallan >> >>>>>>> On 15/10/2008, at 11:27 PM,EmilTin wrote: >> >>>>>>>> thanks for you reply. >> >>>>>>>> i now have sphinx running on a separate ec2 instance, and rails >>>>>>>> accessing it. the key was to edit the production.config.sphinx >>>>>>>> file in >>>>>>>> the rails config folder on the sphinx server (i deploy the rail >>>>>>>> app >>>>>>>> folder, but don't run the app) to 0.0.0.0 isntead of the >>>>>>>> default >>>>>>>> 127.0.0.0, to make sure it listens to all interfaces, not just >>>>>>>> the >>>>>>>> local one. >> >>>>>>>> destroy an indexed model will still causes an exception if the >>>>>>>> sphinx >>>>>>>> server can't be reached. that's really bad, since it could mean >>>>>>>> otherwise perfect cod breaks down in case th sphinx server is >>>>>>>> down. >>>>>>>> it's suddently making my rails app very fragile. and there's no >>>>>>>> need >>>>>>>> to! delta indexing is off! in any case it would be better to >>>>>>>> just >>>>>>>> silently ignore the error, so problems are not multiplied. it's >>>>>>>> much >>>>>>>> worse to have a destroy() call fail than failing to update a >>>>>>>> non- >>>>>>>> existing delta search index. >> >>>>>>>> On Oct 15, 10:06 am, Pat Allan <[EMAIL PROTECTED]> >>>>>>>> wrote: >>>>>>>>> HiEmil >> >>>>>>>>> Sorry there's not been a reply to this, but I think the short >>>>>>>>> answer >>>>>>>>> is: no, there's no easy way to do this. You could overwrite >>>>>>>>> some >>>>>>>>> methods or something like that, but it's definitely a hackish >>>>>>>>> solution, and would be best just to focus on getting the >>>>>>>>> environment >>>>>>>>> working correctly so this is no longer an issue. >> >>>>>>>>> Sorry, I realise that's not really all that helpful. And I've >>>>>>>>> never >>>>>>>>> tried EC2, let alone getting Sphinx running on it. >> >>>>>>>>> -- >>>>>>>>> Pat >> >>>>>>>>> On 08/10/2008, at 4:59 PM,EmilTin wrote: >> >>>>>>>>>> The problem is this: whenever the sphinx server is down, not >>>>>>>>>> only >>>>>>>>>> the >>>>>>>>>> search page stops working. Every page that tries to delete a >>>>>>>>>> model >>>>>>>>>> will fail. This is a problem while I'm trying to get the >>>>>>>>>> sphinx >>>>>>>>>> server >>>>>>>>>> up running. I still haven't managed to make TS contact the >>>>>>>>>> sphinx >>>>>>>>>> server running on a separate EC2 instance. >> >>>>>>>>>> Is there anyway to stop TS from trying to change the >>>>>>>>>> sphinx_deleted >>>>>>>>>> attribute? (Even just temporarily?) >> >>>>>>>>>> On Oct 7, 10:18 am, Pat Allan <[EMAIL PROTECTED]> >>>>>>>>>> wrote: >>>>>>>>>>> HiEmil >> >>>>>>>>>>> Sphinx can update _attributes_ in indexes (but not fields - >>>>>>>>>>> hence >>>>>>>>>>> why >>>>>>>>>>> some people use delta indexes) - and there's an attribute, >>>>>>>>>>> created by >>>>>>>>>>> Thinking Sphinx, called sphinx_deleted. This gets set to >>>>>>>>>>> true >>>>>>>>>>> when a >>>>>>>>>>> model is deleted, so it won't get returned in future results >>>>>>>>>>> (otherwise pagination counts are inaccurate). >> >>>>>>>>>>> Cheers >> >>>>>>>>>>> -- >>>>>>>>>>> Pat >> >>>>>>>>>>> On 06/10/2008, at 9:12 PM,EmilTin wrote: >> >>>>>>>>>>>> Hi! >> >>>>>>>>>>>> I'm gettting the error 'Connection to Sphinx Daemon >>>>>>>>>>>> (searchd) >>>>>>>>>>>> failed' >>>>>>>>>>>> whenever I try to delete any of my indexed models. >> >>>>>>>>>>>> It's true that the sphinx server is not running. But delta >>>>>>>>>>>> indexing is >>>>>>>>>>>> not turned on, so why would TS try to connect to the sphinx >>>>>>>>>>>> server >>>>>>>>>>>> when destroying a model? >> >>>>>>>>>>>> none of my models include :delta => true. They only contain >>>>>>>>>>>> trivial >>>>>>>>>>>> things like: >> >>>>>>>>>>>> define_index do >>>>>>>>>>>> indexes :name >>>>>>>>>>>> indexes :description >>>>>>>>>>>> end >> >>>>>>>>>>>> trace: >>>>>>>>>>>> vendor/plugins/thinking-sphinx/lib/thinking_sphinx/ >>>>>>>>>>>> search.rb: >>>>>>>>>>>> 231:in >>>>>>>>>>>> `search_for_id' >>>>>>>>>>>> vendor/plugins/thinking-sphinx/lib/thinking_sphinx/ >>>>>>>>>>>> active_record/ >>>>>>>>>>>> search.rb:43:in `search_for_id' >>>>>>>>>>>> vendor/plugins/thinking-sphinx/lib/thinking_sphinx/ >>>>>>>>>>>> active_record.rb: >>>>>>>>>>>> 121:in `in_core_index?' >>>>>>>>>>>> vendor/plugins/thinking-sphinx/lib/thinking_sphinx/ >>>>>>>>>>>> active_record.rb: >>>>>>>>>>>> 137:in `toggle_deleted' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 173:in >>>>>>>>>>>> `send' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 173:in >>>>>>>>>>>> `evaluate_method' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 161:in >>>>>>>>>>>> `call' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 93:in >>>>>>>>>>>> `run' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 92:in >>>>>>>>>>>> `each' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 92:in >>>>>>>>>>>> `send' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 92:in >>>>>>>>>>>> `run' >>>>>>>>>>>> vendor/rails/activesupport/lib/active_support/callbacks.rb: >>>>>>>>>>>> 272:in >>>>>>>>>>>> `run_callbacks' >>>>>>>>>>>> vendor/rails/activerecord/lib/active_record/callbacks.rb: >>>>>>>>>>>> 298:in >>>>>>>>>>>> `callback' >>>>>>>>>>>> vendor/rails/activerecord/lib/active_record/callbacks.rb: >>>>>>>>>>>> 290:in >>>>>>>>>>>> `destroy_without_transactions' >>>>>>>>>>>> vendor/rails/activerecord/lib/active_record/ >>>>>>>>>>>> transactions.rb: >>>>>>>>>>>> 102:in >>>>>>>>>>>> `destroy_without_after_commit_callback' >>>>>>>>>>>> vendor/rails/activerecord/lib/active_record/ >>>>>>>>>>>> connection_adapters/ >>>>>>>>>>>> abstract/database_statements.rb:66:in `transaction' >>>>>>>>>>>> vendor/rails/activerecord/lib/active_record/ >>>>>>>>>>>> transactions.rb: >>>>>>>>>>>> 79:in >>>>>>>>>>>> `transaction' >>>>>>>>>>>> vendor/rails/activerecord/lib/active_record/ >>>>>>>>>>>> transactions.rb: >>>>>>>>>>>> 98:in >>>>>>>>>>>> `transaction' >>>>>>>>>>>> vendor/rails/activerecord/lib/active_record/ >>>>>>>>>>>> transactions.rb: >>>>>>>>>>>> 102:in >>>>>>>>>>>> `destroy_without_after_commit_callback' >>>>>>>>>>>> vendor/plugins/thinking-sphinx/lib/thinking_sphinx/ >>>>>>>>>>>> active_record/ >>>>>>>>>>>> delta.rb:58:in `destroy' >>>>>>>>>>>> app/controllers/bands_controller.rb:236:in `destroy' > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en -~----------~----~----~----~------~----~------~--~---
