Well, thanks very much Kent, Hugo and Danny. I went with the "never-ending blocking queues" and sentinel data approach. When running tests with a continual stream of packets being received 3ms apart, CPU usage peaked at 15%, was usually around 7-9%, and when deployed the packets will separated by seconds rather than milliseconds.
Thanks for the assistance, I've now overcome my fear of blocking I/O :). Regards, Lia, Clarke On 4/8/06, Liam Clarke <[EMAIL PROTECTED]> wrote: > Thanks very much all. :) I'll have a crack this afternoon and let you know. > > Kent - the increase in the queue size for the socket server is to > allow for any delay in processing packets; it has a default queue size > of 5 and then it starts rejecting packets; more of a safety policy > when reducing CPU usage than a direct attempt to reduce CPU usage. > > Once again, thanks for the input! > > On 4/8/06, Danny Yoo <[EMAIL PROTECTED]> wrote: > > > > > > On Fri, 7 Apr 2006, Kent Johnson wrote: > > > > > Hugo Gonz�lez Monteverde wrote: > > > > You are not using the optional timeout and blocking which 'get' > > > > provides (!) > > > > > > > > Try setting it and see your CPU usage go down. This will implement > > > > blocking, and the queue will be used as soon as data is there. Set > > > > block > > > > to True and a timeout if you need to use it (looks like you only need > > > > blocking) > > > > > > > > while True: > > > > try: > > > > data = self.queue.get(True) > > > > self.DAO.send_data(data) > > > > > > I think he will need the timeout too, otherwise the shutdown flag will > > > only be checked when there is work which is probably not what he wants. > > > > > > This could be fixed: when setting the 'shutdown' flag, also push a > > sentinel piece of data into the queue. That'll wake the thread back up, > > and that'll give it the opportunity to process a shutdown. > > > > The thread here: > > > > http://mail.python.org/pipermail/tutor/2006-January/044557.html > > > > has some more examples of this. > > > > > > Hope this helps! > > > > > > > > _______________________________________________ > > Tutor maillist - Tutor@python.org > > http://mail.python.org/mailman/listinfo/tutor > > > > > > > _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
