The jobs may take more than 1s to finish, but the backend web service I hit (which is out of my control) gets angry if things come in too fast. My worker is doing non-blocking HTTP requests, so while there may be several jobs in flight, I'm trying to keep the requests/second low.
Originally I was going to just implement this with a Mojo::IOLoop timer, but if my service dies during a batch, I'd have to implement some durable job store... like Minion :) I'll try to work around this with some special logic in the worker or something. Scott On Friday, December 15, 2017 at 10:08:56 AM UTC-7, sri wrote: > > I'm adding many jobs into a queue (Pg backend) with something like this: >> >> >> https://gist.github.com/scottw/18a95ddaab44837e92526cdbc93d4275#file-inject >> >> Each job has a delay that increments by 1s. My worker handles the jobs: >> >> >> https://gist.github.com/scottw/18a95ddaab44837e92526cdbc93d4275#file-worker >> >> but the jobs come in bursts every 5 seconds, as you can see in the >> timestamps in the output: >> >> >> https://gist.github.com/scottw/18a95ddaab44837e92526cdbc93d4275#file-output-log >> >> Can anyone recommend a better technique for ensuring jobs don't run >> before their time—and run as close to their time as possible? Am I using >> 'delay' wrong? >> > > This is a tough case. Delays are definitely not meant for 1s accuracy, > there will always > be delays (haha...i'll see myself out...). I actually find it odd that you > have so many jobs > that take less than a second to finish, maybe a job queue is not the right > tool for the job? > Most of my background jobs are more in the 15s-15m range > > You could also use job dependencies, so later jobs only run when the > earlier ones are > finished. > > -- > sebastian > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
