Hi Stephen, You could definitely get increased throughput using SimpleWorker as you would instantly be going from one server to hundreds to run your jobs. (disclosure: I created SimpleWorker). If you decide to do it, there is one suggestion that will make it work a lot faster/better for your scenario where you have a bunch of small jobs: batch up the work into fewer jobs. For instance, instead of queuing up one email per job, queue up 50 emails per job (pass an array of 50 emails to the worker). So instead of queuing up 20,000 jobs, you'd queue up 400 jobs with 50 emails each. There are a couple of reasons for this:
1) It will take longer to queue up all your jobs since you're sending them to a remote service (unlike Resque) so it's recommended to batch them up. Queuing up 400 jobs takes a fraction of the time compared to queuing up 20,000. 2) There is setup/teardown time involved on SimpleWorker, so if you're jobs only take a couple seconds to run, like sending an email probably does, you'll want to batch them up. If you don't, more time will be spent on the setup/teardown than actually doing the work. But if you do 50 or 100 in a single job, the setup/teardown will be negligible. I hope that helps! Feel free to email me directly if you have any other questions or want some help getting started. Regards, Travis -- You received this message because you are subscribed to the Google Groups "Heroku" group. To view this discussion on the web visit https://groups.google.com/d/msg/heroku/-/XUVNRZUEnbMJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
