Hi Pat I've spent another age on this and still haven't got a solution. The issue is definitely down to delayed_job or one of its dependencies. I can happily run everything from your code outside of delayed_job but once I start to queue it.
There are lots of issues on the delayed_job issues page on github (https://github.com/collectiveidea/delayed_job/issues) which don't seem to be getting addressed in the 2.0.* branch. I've had a look but can't see anything I can contribute myself at this stage and the gem has moved on now to the rails 3 version. In terms of progress, I've actually got it to run now but it hangs on the second indexing job for some reason and the DJ daemon has to be restarted, at which point it will run another indexing job and then hang again. I got to this stage with DJ 2.0.5 and daemons 1.0.10 (1.1 has known issues with DJ it appears and won't let the daemon start). I've also tried swapping out the use of daemons with daemon-spawn and got the same result. First index is fine, subsequent one hangs. I'm going to have to give up on this approach and try something else, probably another queue and write my own custom delta worker. If there's anything I can contribute back to TS I will do... Thanks for your help Graham On Jan 30, 2:30 pm, Graham Pengelly <[email protected]> wrote: > Hi Pat > > Yes... I tried system(... but just got the same problem. It did at one > stage go through but output a "memory can't be allocated" message but > when I looked into it I'd just run the rake task so many times that > I'd filled up the RAM on the box. Once I cleared it we were back to > square one with it hanging. I thought I was on to something with that > as well but apparently not. > > I'm logged in as the user that rails runs under so if I execute the > index command directly it should be the same user as when I do it > through rake ts:dd etc. I did think that it may be a permissions issue > so I changed the permissions on the indexer to 777 but it didn't make > any difference. The other delayed jobs that ts queues go through fine > when I re-run the rake task. It is just the indexer one that has the > problem so it seems that delayed_job is working fine except with the > sphinx indexer. > > I'm going to have another crack at it this evening so I'll post back > if I make any progress... > > Thanks > > Graham > > On Jan 30, 1:21 pm, Pat Allan <[email protected]> wrote: > > > > > > > > > Hi Graham > > > What happens if you switch `#{config.bin_path}...` to > > system("#{config.bin_path}...")? > > > No idea why it would be any different, mind you. Are you running the > > commands manually with the same user that runs delayed job? > > > -- > > Pat > > > On 30/01/2011, at 9:04 AM, Graham Pengelly wrote: > > > > Hi Pay > > > > I still haven't fixed it but it seems that it is hanging in the > > > perform method on DeltaJob when the indexer gets called. The command > > > is fine - > > > > /usr/bin/indexer --config /home/rails/secure.mapslegal.co.uk/releases/ > > > 20110128140157/config/production.sphinx.conf --rotate policy_delta > > > policy_delta > > > > ...and if I execute 'manually' it works as expected. Curiously, if I > > > alter that line and change it to exec "#{config.bin_path}... The > > > indexer runs successfully without hanging but obviously it never > > > returns and the rake task exits without updating the delayed_jobs > > > table. > > > > I'll have another go at it tomorrow but thought I'd report back and > > > see if my 'findings' so far shed any light on the problem. > > > > Thanks > > > > Graham > > > > On Jan 27, 8:22 am, Graham Pengelly <[email protected]> wrote: > > >> Hi Pat > > > >> Yes... thinking_sphinx, ts-delayed-delta and delayed-job are all > > >> installed as gems. I'm away today so can't work on it but I'll have > > >> another crack at it tomorrow and post back on here with any progress. > > > >> Thanks again for your advice. > > > >> Graham > > > >> On Jan 27, 1:21 am, Pat Allan <[email protected]> wrote: > > > >>> Hi Graham > > > >>> Just to confirm - are you using TS as a gem? From memory, I think if > > >>> ts-delayed-delta is being used, TS needs to be installed as a gem > > >>> instead of a plugin. > > > >>> Beyond that, I'm at a loss as to why this could be happening. I guess, > > >>> I'd start debugging by editing the delayed job and ts-delayed-delta > > >>> gems to figure out where it's hanging. Can understand if you don't want > > >>> to do this, mind you... if you're willing to give me access to the > > >>> project so I can try reproducing the issue, contact me off-list. > > > >>> Cheers > > > >>> -- > > >>> Pat > > > >>> On 26/01/2011, at 10:47 PM, Graham Pengelly wrote: > > > >>>> Hi Pat > > > >>>> Thanks for responding... I'm still stuck with it and was about to > > >>>> resort to trying to fix it with a hammer :) > > > >>>> I thought there might be some version issues myself as the latest > > >>>> delayed_job looked to be only compatible with rails 3 and the ts- > > >>>> delayed-job gem seemed to pull in the latest version of delayed_job > > >>>> when dj wasn't installed. So... I installed the earlier, 2.3.* > > >>>> compatible version of dj. > > > >>>> My app is currently on: > > > >>>> Rails 2.3.8 > > >>>> thinking-sphinx 1.4.1 > > >>>> ts-delayed-delta 1.1.1 > > >>>> delayed_job 2.0.4 > > > >>>> Thanks > > > >>>> Graham > > > >>>> On Jan 26, 11:20 am, Pat Allan <[email protected]> wrote: > > >>>>> Hi Graham > > > >>>>> This is definitely something odd. What versions of Rails and Delayed > > >>>>> Job are you using? > > > >>>>> -- > > >>>>> Pat > > > >>>>> On 24/01/2011, at 10:02 PM, Graham Pengelly wrote: > > > >>>>>> Hi > > > >>>>>> I'm attempting to set up delta indexing using delayed_job via the the > > >>>>>> ts-delayed-delta (1.1.1) gem. The app is happily queuing jobs to the > > >>>>>> delayed_job table in the db but when I run the ts:dd rake task (or > > >>>>>> any > > >>>>>> other method of getting the delayed_job worker running) it hangs > > >>>>>> after > > >>>>>> locking the first delta job with the log - > > > >>>>>> Starting job worker > > >>>>>> acquired lock on ThinkingSphinx::Deltas::DeltaJob for policy_delta, > > >>>>>> policy_delta > > > >>>>>> I can create a ThinkingSphinx::Deltas::DeltaJob and then 'perform' it > > >>>>>> successfully from the console so that works on the machine. The db > > >>>>>> table is also getting written to as the 'locked_at' and 'locked_by' > > >>>>>> field are completed. It isn't specifically the policy model causing > > >>>>>> the problem as I use delta indexing on a couple of other models and > > >>>>>> they hang after the lock too. > > > >>>>>> Any ideas would be gratefully received. > > > >>>>>> Thanks in advance > > > >>>>>> Graham > > > >>>>>> -- > > >>>>>> You received this message because you are subscribed to the Google > > >>>>>> Groups "Thinking Sphinx" group. > > >>>>>> To post to this group, send email to > > >>>>>> [email protected]. > > >>>>>> To unsubscribe from this group, send email to > > >>>>>> [email protected]. > > >>>>>> For more options, visit this group > > >>>>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > >>>> -- > > >>>> You received this message because you are subscribed to the Google > > >>>> Groups "Thinking Sphinx" group. > > >>>> To post to this group, send email to [email protected]. > > >>>> To unsubscribe from this group, send email to > > >>>> [email protected]. > > >>>> For more options, visit this group > > >>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Thinking Sphinx" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]. > > > For more options, visit this group > > > athttp://groups.google.com/group/thinking-sphinx?hl=en. -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
