Re: [PHP] Numerical Recipe - Scheduling Question
2009/5/29 Bastien Koert : > > On Fri, May 29, 2009 at 2:12 PM, Stuart wrote: >> >> Hmm, might have to write that lot up as a blog post with some example >> code. Sometime... > > I would be very interested in that. I am developing a project where a queue > is required to manage load. Your post above has some pointers that I like, > but more information would be greatly appreciated. http://stut.net/blog/2009/05/29/php-job-queue/ Pretty much a brain dump but it does provide a little more info than my earlier post. Comments welcome. -Stuart -- http://stut.net/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] Numerical Recipe - Scheduling Question
On Fri, May 29, 2009 at 2:12 PM, Stuart wrote: > 2009/5/29 kyle.smith : > > I'm confused as to why cron doesn't work for you. It doesn't explicitly > > tell you when the next X occurences will be, but math does. If you > > schedule something to run every 5 minutes starting at 1:45 PM, it's > > simple work to be able to report that the next times would be 1:50 PM, > > 1:55 PM, 2:00 PM etc. > > You can be a lot more intelligent than that. I have a job queue system > running on several sites I maintain that uses a simple run_at > timestamp. A cron job runs every minute and essentially does this... > > * Locks the job queue. > > * Does the equivalent of "select job from job_queue where run_at <= > unix_timestamp() order by run_at asc limit 1". > > * If no jobs need running it simply exits otherwise it locks the job > it got back and unlocks the queue. > > * Runs the job (wrapped in a safe environment that catches output and > errors and does something useful with them). > > * Marks the job as completed or with an error status. > > * If the job is marked as recurring it creates a new job by cloning > the job it just ran, sets run_at based upon the schedule definition > (which can be a time of day, a time of day + a day of week, a time of > day + a day of month or simply a number of seconds) and sets the > status to new. > > * Either removes the completed job from the queue or archives it > complete with errors and output for later inspection depending on the > job config and status. > > * If this processor has been running for > 60 minutes it exits, > otherwise it looks for another job to run. > > This system will automatically scale up to 60 job processors per hour, > but obviously you can modify the cron config to run more or less as > your requirements dictate. Assuming the job queue is on a shared > resource such as a database this can also scale across machines > effectively infinitely. > > There's also a whole bunch of stuff around catching crashed jobs and > doing something useful with them, but I'll leave how to handle those > as an exercise for the reader. > > > Is this running in a web browser, somehow? If not, PHP is not the > > solution. > > Total codswallop! PHP is no more tied to web browsers than a > hovercraft is tied to water. My job queue system is 100% PHP (although > it can run jobs not written in PHP, but that's a topic for another > day) and beyond initial development it's never given me any problems. > > Hmm, might have to write that lot up as a blog post with some example > code. Sometime... > > -Stuart > > -- > http://stut.net/ > > > -Original Message- > > From: bruce [mailto:bedoug...@earthlink.net] > > Sent: Friday, May 29, 2009 1:11 PM > > To: php-general@lists.php.net > > Subject: [PHP] Numerical Recipe - Scheduling Question > > > > Hi.. > > > > Got a need to be able to allow a user to specify the frequency to run > > certain apps/processes.. I need to be able to have the user specify a > > start Time, as well as a periodic frequency (once, hourly, daily, > > weekly...) as well as allow the user to specify every XX minutes... > > > > So i basically need to be able to determine when the future > > events/occurances are, based on the user input. > > > > I've searched the net for alogorithms dealing with scheduling and > > haven't come up with any php based solutions.. I've also looked at > > numerical recipes and some other sources (freshmeat/sourceforge/etc..) > > with no luck.. > > > > I have found an approach in another language that I could port to php.. > > But before I code/recreate this, I figured I'd see if anyone here has > > pointers or suggestions... > > > > Cron doesn't work for me, as it can run a process at a given time.. but > > it doesn't tell me when the next 'X' occurance would be... > > > > Thoughts/Comments.. > > > > Thanks > > > > > > -- > > PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: > > http://www.php.net/unsub.php > > > > > > -- > > PHP General Mailing List (http://www.php.net/) > > To unsubscribe, visit: http://www.php.net/unsub.php > > > > > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > > I would be very interested in that. I am developing a project where a queue is required to manage load. Your post above has some pointers that I like, but more information would be greatly appreciated. -- Bastien Cat, the other other white meat
Re: [PHP] Numerical Recipe - Scheduling Question
2009/5/29 kyle.smith : > I'm confused as to why cron doesn't work for you. It doesn't explicitly > tell you when the next X occurences will be, but math does. If you > schedule something to run every 5 minutes starting at 1:45 PM, it's > simple work to be able to report that the next times would be 1:50 PM, > 1:55 PM, 2:00 PM etc. You can be a lot more intelligent than that. I have a job queue system running on several sites I maintain that uses a simple run_at timestamp. A cron job runs every minute and essentially does this... * Locks the job queue. * Does the equivalent of "select job from job_queue where run_at <= unix_timestamp() order by run_at asc limit 1". * If no jobs need running it simply exits otherwise it locks the job it got back and unlocks the queue. * Runs the job (wrapped in a safe environment that catches output and errors and does something useful with them). * Marks the job as completed or with an error status. * If the job is marked as recurring it creates a new job by cloning the job it just ran, sets run_at based upon the schedule definition (which can be a time of day, a time of day + a day of week, a time of day + a day of month or simply a number of seconds) and sets the status to new. * Either removes the completed job from the queue or archives it complete with errors and output for later inspection depending on the job config and status. * If this processor has been running for > 60 minutes it exits, otherwise it looks for another job to run. This system will automatically scale up to 60 job processors per hour, but obviously you can modify the cron config to run more or less as your requirements dictate. Assuming the job queue is on a shared resource such as a database this can also scale across machines effectively infinitely. There's also a whole bunch of stuff around catching crashed jobs and doing something useful with them, but I'll leave how to handle those as an exercise for the reader. > Is this running in a web browser, somehow? If not, PHP is not the > solution. Total codswallop! PHP is no more tied to web browsers than a hovercraft is tied to water. My job queue system is 100% PHP (although it can run jobs not written in PHP, but that's a topic for another day) and beyond initial development it's never given me any problems. Hmm, might have to write that lot up as a blog post with some example code. Sometime... -Stuart -- http://stut.net/ > -Original Message- > From: bruce [mailto:bedoug...@earthlink.net] > Sent: Friday, May 29, 2009 1:11 PM > To: php-general@lists.php.net > Subject: [PHP] Numerical Recipe - Scheduling Question > > Hi.. > > Got a need to be able to allow a user to specify the frequency to run > certain apps/processes.. I need to be able to have the user specify a > start Time, as well as a periodic frequency (once, hourly, daily, > weekly...) as well as allow the user to specify every XX minutes... > > So i basically need to be able to determine when the future > events/occurances are, based on the user input. > > I've searched the net for alogorithms dealing with scheduling and > haven't come up with any php based solutions.. I've also looked at > numerical recipes and some other sources (freshmeat/sourceforge/etc..) > with no luck.. > > I have found an approach in another language that I could port to php.. > But before I code/recreate this, I figured I'd see if anyone here has > pointers or suggestions... > > Cron doesn't work for me, as it can run a process at a given time.. but > it doesn't tell me when the next 'X' occurance would be... > > Thoughts/Comments.. > > Thanks > > > -- > PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: > http://www.php.net/unsub.php > > > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > > -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
RE: [PHP] Numerical Recipe - Scheduling Question
I'm confused as to why cron doesn't work for you. It doesn't explicitly tell you when the next X occurences will be, but math does. If you schedule something to run every 5 minutes starting at 1:45 PM, it's simple work to be able to report that the next times would be 1:50 PM, 1:55 PM, 2:00 PM etc. Is this running in a web browser, somehow? If not, PHP is not the solution. HTH, Kyle -Original Message- From: bruce [mailto:bedoug...@earthlink.net] Sent: Friday, May 29, 2009 1:11 PM To: php-general@lists.php.net Subject: [PHP] Numerical Recipe - Scheduling Question Hi.. Got a need to be able to allow a user to specify the frequency to run certain apps/processes.. I need to be able to have the user specify a start Time, as well as a periodic frequency (once, hourly, daily, weekly...) as well as allow the user to specify every XX minutes... So i basically need to be able to determine when the future events/occurances are, based on the user input. I've searched the net for alogorithms dealing with scheduling and haven't come up with any php based solutions.. I've also looked at numerical recipes and some other sources (freshmeat/sourceforge/etc..) with no luck.. I have found an approach in another language that I could port to php.. But before I code/recreate this, I figured I'd see if anyone here has pointers or suggestions... Cron doesn't work for me, as it can run a process at a given time.. but it doesn't tell me when the next 'X' occurance would be... Thoughts/Comments.. Thanks -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php