I got this to work by using @reboot cron. It works great, and super
simple. Cross-platform, needs no external daemons.
The trick was that web2py wasn't shutting down when my script was
running, because I had a try-catch that was catching the
KeyboardInterrupt exception. So here's what I did:
In cron/crontab:
@reboot toomim *applications/init/cron/background_work.py
(Thanks to achipa and massimo for fixing @reboot recently!)
In cron/background_work.py:
import time, sys, commands, datetime
# Lock the database so we don't accidentally run two background
# processes at the same time, e.g. if we run web2py twice by
accident
if not db.executesql('select pg_try_advisory_lock(1);')[0][0]:
logging.debug('CRON: FAILED to get background process lock')
raise Exception('Somebody else is running background
processes')
logging.debug('CRON: got lock')
count = 0
while True:
count += 1
logging.debug('Sleeping for the %dth time' % count)
time.sleep(10)
try:
logging.debug('Processing launch queue')
process_launch_queue()
logging.debug('Processing bonus queue')
process_bonus_queue()
except KeyboardInterrupt as e:
logging.debug('Someone killed us.')
raise
except:
logging.debug("Error in background process: %s" %
sys.exc_info()[0])
Note that I explicitly catch the KeyboardInterrupt and re-raise it so
the script dies. If you kill web2py with some other type of exception
you might need to put it in there too.
The other trick is that print statements and exceptions aren't
displayed at the console because this is run as a separate python
process. So you need to use logging.debug from log.py. I don't
remember where I got that file...
On Jan 3, 1:57 pm, Richard <[email protected]> wrote:
> I was doing that but sometimes the process crashed or was killed by my
> hosting provider, so I need a way to continue it automatically. Also I
> may want to execute a number of background tasks in parallel. And it
> uses extra resources that aren't needed, which is a problem on my
> hosting where I am reaching the limits.
>
> Michael, are you looking for a particular platform solution or cross
> platform?
>
> On Jan 4, 5:23 am, mdipierro <[email protected]> wrote:
>
>
>
> > I normally just start a separate web2py process.
>
> > python web2py.py -S app -M -S thescript
>
> > On Jan 3, 11:58 am, Michael Toomim <[email protected]> wrote:
>
> > > Great! I am also trying to implement this. Richard and Auden, have
> > > you gotten anything working yet? How about we share solutions?
>
> > > My current difficulty is figuring out where to create thepersistent
> > > background thread.
> > > - If I spawn it in a controller or model file will it be limited to
> > > 10 seconds?
> > > - If I run it in a periodic cron there will be a delay and I will
> > > need to handle the case of multiple threads running simultaneously
> > > - If I use @reboot cron, it prevents web2py from shutting down (it
> > > seems to wait for my thread to quit, which does not know to quit)
>
> > > Any ideas on these problems? It would be great to have a general task
> > > queue in web2py, like the one on GAE. Although I've never learned how
> > > to use the GAE one, perhaps what we're making could be easier to use.
>
> > > On Jan 3, 8:06 am, mdipierro <[email protected]> wrote:
>
> > > > This is what I would do:
>
> > > > db.define_table('queue',Field('status'),Field('descr'))
>
> > > > In actions
>
> > > > db.queue.insert(status='PENDING',descr='whatever')
>
> > > > In your own scrips that you can run via CRON or via background process
>
> > > > while True:
> > > > tasks = db(db.queue.status=='PENDING').select(limitby=(0,10)).first
> > > > ()
> > > > if task:
> > > > task.update_record(status='PROCESSING')
> > > > db.commit()
> > > > try:
> > > > #process task
> > > > task.update_record(status='COMPLETED')
> > > > except:
> > > > task.update_record(status='FAILED')
> > > > db.commit()
> > > > else:
> > > > sleep(10) # retry in 10 seconds
>
> > > > On Jan 3, 7:28 am, Richard <[email protected]> wrote:
>
> > > > > I haven't implemented this yet, but my plan on Linux to keep a
> > > > > background process running is:
>
> > > > > - Define a task table, which is checked on every request
> > > > > - If the task table is empty then start the background task with
> > > > > subprocess and store the pid in the task table
> > > > > - If the task table has an entry but its pid is not active (not in /
> > > > > proc) then restart the background task and update the pid
> > > > > - Else the task is running fine
>
> > > > > Perhaps you could adapt that to your needs, assuming you are using
> > > > > Linux.
>
> > > > > Richard
>
> > > > > On Dec 30 2009, 3:26 pm, Auden RovelleQuartz <[email protected]>
> > > > > wrote:
>
> > > > > > any simple example on how to create apersistentbackground process
> > > > > > that runs continuously on the server side no matter what user signs
> > > > > > in
> > > > > > or out, and no matter how many concurrent users are on the system?
>
> > > > > > Here are a couple of examples:
>
> > > > > > when a user performs an "event", it kicks off a server side process
> > > > > > that continues executing even when that user signs off.
>
> > > > > > when a user performs an "event", then after a set period of time
> > > > > > (say
> > > > > > two days) then an email is automatically sent to a specified e-mail
> > > > > > address that happens even if the user that triggers the event no
> > > > > > longer has an active session on the server side.
>
> > > > > > I am attempting to build an auction application in the web2py
> > > > > > framework and would be interested in learning a tecqnique of
> > > > > > starting
> > > > > > (and stopping)persistentserver-side background processes.
>
> > > > > > Much thanks- Hide quoted text -
>
> > > - Show quoted text -
--
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en.
