If you are on GAE you could use Task Queues.
Else how about have a table with the URL and a boolean for whether it has 
been scraped. Then use a cron job to periodically process outstanding 
scrapes.

Reply via email to