Hi Graham It's a shame that it's still not working. I guess my one last question is - is there any difference between running script/delayed_job, rake jobs:work and Delayed::Worker.work_off? (Granted, I'm used to the Rails 3 version, so maybe these are slightly different).
It shouldn't be too hard to create a new gem for a different queue system - one exists for Workling: https://github.com/dpickett/workling_delta_indexer And I know others have been created (but maybe not released publicly). Good luck! -- Pat On 01/02/2011, at 9:24 AM, Graham Pengelly wrote: > Hi Pat > > I've spent another age on this and still haven't got a solution. The > issue is definitely down to delayed_job or one of its dependencies. I > can happily run everything from your code outside of delayed_job but > once I start to queue it. > > There are lots of issues on the delayed_job issues page on github > (https://github.com/collectiveidea/delayed_job/issues) which don't > seem to be getting addressed in the 2.0.* branch. I've had a look but > can't see anything I can contribute myself at this stage and the gem > has moved on now to the rails 3 version. > > In terms of progress, I've actually got it to run now but it hangs on > the second indexing job for some reason and the DJ daemon has to be > restarted, at which point it will run another indexing job and then > hang again. I got to this stage with DJ 2.0.5 and daemons 1.0.10 (1.1 > has known issues with DJ it appears and won't let the daemon start). > I've also tried swapping out the use of daemons with daemon-spawn and > got the same result. First index is fine, subsequent one hangs. > > I'm going to have to give up on this approach and try something else, > probably another queue and write my own custom delta worker. If > there's anything I can contribute back to TS I will do... > > Thanks for your help > > Graham > > > On Jan 30, 2:30 pm, Graham Pengelly <[email protected]> wrote: >> Hi Pat >> >> Yes... I tried system(... but just got the same problem. It did at one >> stage go through but output a "memory can't be allocated" message but >> when I looked into it I'd just run the rake task so many times that >> I'd filled up the RAM on the box. Once I cleared it we were back to >> square one with it hanging. I thought I was on to something with that >> as well but apparently not. >> >> I'm logged in as the user that rails runs under so if I execute the >> index command directly it should be the same user as when I do it >> through rake ts:dd etc. I did think that it may be a permissions issue >> so I changed the permissions on the indexer to 777 but it didn't make >> any difference. The other delayed jobs that ts queues go through fine >> when I re-run the rake task. It is just the indexer one that has the >> problem so it seems that delayed_job is working fine except with the >> sphinx indexer. >> >> I'm going to have another crack at it this evening so I'll post back >> if I make any progress... >> >> Thanks >> >> Graham >> >> On Jan 30, 1:21 pm, Pat Allan <[email protected]> wrote: >> >> >> >> >> >> >> >>> Hi Graham >> >>> What happens if you switch `#{config.bin_path}...` to >>> system("#{config.bin_path}...")? >> >>> No idea why it would be any different, mind you. Are you running the >>> commands manually with the same user that runs delayed job? >> >>> -- >>> Pat >> >>> On 30/01/2011, at 9:04 AM, Graham Pengelly wrote: >> >>>> Hi Pay >> >>>> I still haven't fixed it but it seems that it is hanging in the >>>> perform method on DeltaJob when the indexer gets called. The command >>>> is fine - >> >>>> /usr/bin/indexer --config /home/rails/secure.mapslegal.co.uk/releases/ >>>> 20110128140157/config/production.sphinx.conf --rotate policy_delta >>>> policy_delta >> >>>> ...and if I execute 'manually' it works as expected. Curiously, if I >>>> alter that line and change it to exec "#{config.bin_path}... The >>>> indexer runs successfully without hanging but obviously it never >>>> returns and the rake task exits without updating the delayed_jobs >>>> table. >> >>>> I'll have another go at it tomorrow but thought I'd report back and >>>> see if my 'findings' so far shed any light on the problem. >> >>>> Thanks >> >>>> Graham >> >>>> On Jan 27, 8:22 am, Graham Pengelly <[email protected]> wrote: >>>>> Hi Pat >> >>>>> Yes... thinking_sphinx, ts-delayed-delta and delayed-job are all >>>>> installed as gems. I'm away today so can't work on it but I'll have >>>>> another crack at it tomorrow and post back on here with any progress. >> >>>>> Thanks again for your advice. >> >>>>> Graham >> >>>>> On Jan 27, 1:21 am, Pat Allan <[email protected]> wrote: >> >>>>>> Hi Graham >> >>>>>> Just to confirm - are you using TS as a gem? From memory, I think if >>>>>> ts-delayed-delta is being used, TS needs to be installed as a gem >>>>>> instead of a plugin. >> >>>>>> Beyond that, I'm at a loss as to why this could be happening. I guess, >>>>>> I'd start debugging by editing the delayed job and ts-delayed-delta gems >>>>>> to figure out where it's hanging. Can understand if you don't want to do >>>>>> this, mind you... if you're willing to give me access to the project so >>>>>> I can try reproducing the issue, contact me off-list. >> >>>>>> Cheers >> >>>>>> -- >>>>>> Pat >> >>>>>> On 26/01/2011, at 10:47 PM, Graham Pengelly wrote: >> >>>>>>> Hi Pat >> >>>>>>> Thanks for responding... I'm still stuck with it and was about to >>>>>>> resort to trying to fix it with a hammer :) >> >>>>>>> I thought there might be some version issues myself as the latest >>>>>>> delayed_job looked to be only compatible with rails 3 and the ts- >>>>>>> delayed-job gem seemed to pull in the latest version of delayed_job >>>>>>> when dj wasn't installed. So... I installed the earlier, 2.3.* >>>>>>> compatible version of dj. >> >>>>>>> My app is currently on: >> >>>>>>> Rails 2.3.8 >>>>>>> thinking-sphinx 1.4.1 >>>>>>> ts-delayed-delta 1.1.1 >>>>>>> delayed_job 2.0.4 >> >>>>>>> Thanks >> >>>>>>> Graham >> >>>>>>> On Jan 26, 11:20 am, Pat Allan <[email protected]> wrote: >>>>>>>> Hi Graham >> >>>>>>>> This is definitely something odd. What versions of Rails and Delayed >>>>>>>> Job are you using? >> >>>>>>>> -- >>>>>>>> Pat >> >>>>>>>> On 24/01/2011, at 10:02 PM, Graham Pengelly wrote: >> >>>>>>>>> Hi >> >>>>>>>>> I'm attempting to set up delta indexing using delayed_job via the the >>>>>>>>> ts-delayed-delta (1.1.1) gem. The app is happily queuing jobs to the >>>>>>>>> delayed_job table in the db but when I run the ts:dd rake task (or any >>>>>>>>> other method of getting the delayed_job worker running) it hangs after >>>>>>>>> locking the first delta job with the log - >> >>>>>>>>> Starting job worker >>>>>>>>> acquired lock on ThinkingSphinx::Deltas::DeltaJob for policy_delta, >>>>>>>>> policy_delta >> >>>>>>>>> I can create a ThinkingSphinx::Deltas::DeltaJob and then 'perform' it >>>>>>>>> successfully from the console so that works on the machine. The db >>>>>>>>> table is also getting written to as the 'locked_at' and 'locked_by' >>>>>>>>> field are completed. It isn't specifically the policy model causing >>>>>>>>> the problem as I use delta indexing on a couple of other models and >>>>>>>>> they hang after the lock too. >> >>>>>>>>> Any ideas would be gratefully received. >> >>>>>>>>> Thanks in advance >> >>>>>>>>> Graham >> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Thinking Sphinx" group. >>>>>>>>> To post to this group, send email to [email protected]. >>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>> [email protected]. >>>>>>>>> For more options, visit this group >>>>>>>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. >> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Thinking Sphinx" group. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]. >>>>>>> For more options, visit this group >>>>>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. >> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Thinking Sphinx" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]. >>>> For more options, visit this group >>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/thinking-sphinx?hl=en. > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
