If you are on GAE you could use Task Queues. Else how about have a table with the URL and a boolean for whether it has been scraped. Then use a cron job to periodically process outstanding scrapes.
- [web2py] two alternatives for running background processes apple
- [web2py] Re: two alternatives for running background processes PhE
- [web2py] Re: two alternatives for running background processes Plumo

