On Thu, 2011-04-14 at 13:28 -0400, Jay Dobies wrote: > You lost me after the 30 seconds part. How will the on demand > dispatcher > resolve handling situations where there are other tasks ahead of it in > line? > > So let's take a case of a single thread doing stuff. It just popped a > new task off the queue, beginning the 30 second timer. Left on the > queue > are tasks B, C, and D. > > During that 30 second timeframe, a request to sync a repo comes in. > I'd > expect it to be put on the queue to execute after D, but the way I > read > your comment it'll jump to the front of the line by being run by the > on > demand dispatcher. >
I think you're thinking of the queue as a fifo, when it's actually a priority queue ordered by the tasks's scheduled_time field. > I'm probably missing something though, so maybe some more explanation > will help clear it up. > Ok, so let's take your example. Let's assume that the tasking system is setup to run several task concurrently. This means at the last time the dispatcher woke up, task A was able to run, but the rest of the tasks' scheduled_times were not yet up. When the repo sync comes in (assuming it's not a scheduled sync) it'll get "scheduled" to run immediately and enqueued. This will wake up the dispatcher thread (the on-demand part) which will examine the tasks at the head of the queue, find the repo sync, and kick it off. As well as kick off any other tasks that have become ready. > Also (again, I don't know how you're implementing it), based on the > discussions in this thread it sounds like the queue of tasks will only > exist in the database and not in memory. How is that going to affect > the > uniqueness check? Does that become a series of database retrievals or > can we do that check atomically in the DB? The uniqueness check is start a tad naive, as it's basically pulling all the tasks from the db and iterating over them. I have plans to convert the uniqueness criteria into a db spec and pushing it completely into the database. However, I need to pickle the arguments in order to do so and haven't gotten around to doing that yet. -- Jason L Connor linear on freenode #pulp http://pulpproject.org/ RHCE: 805010912355231 GPG Fingerprint: 2048R/CC4ED7C1
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Pulp-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/pulp-list
