First let me say that I think POE is awesome! It's been a mind-blowing couple of weeks trying to get my brain around it.
I am rewriting a Network Monitoring system where the clients periodically connect to my server and send things. Right now I am using a simple accept and forking IO::Socket::INET server. The problem is that we are at the mercy of the clients and sometimes the clients hang or take a long time while sending data. Add to this that many clients may end up connecting at nearly the same time and you have a recipe for out of memory/cannot fork conditions. Some of this is due to other design problems. But I am doing a complete rewrite of the whole thing so I will fix those as well. So on the server I know I want to do something more robust and POE seems to encapsulate that already so that's what I am going to use.
Basically my server would be getting an XML document from the client. I'll only process the data from the client after it disconnects. No need to answer back to it (at least not yet). I would also like to batch up the responses and process them with POE::Wheel::Run (because processing the queue involves a lot of slow running parsing of some XML and entering it into a database). So I am building up a queue and periodically processing it. The next step would be to take the entire queue, hand it of to POE::Wheel::Run, zero the queue, and keep moving. In my mind this is a Poe Man's pre-forking server. Here (below) is a skeleton of what I have so far to demonstrate this (but w/o the POE::Wheel::Run). Here are my questions:
1) Am I on the right track to have the Server and the queue processing in separate sessions where the Server "post"s to the queue's session?
2) Is there any way to high-water mark the queue so that it is processed immediately if reaching that mark? Or should I just be happy with periodically processing the queue as I have done below?
3) Am I using delay_set below in the way it was intended to be used?
4) Can anyone point me to examples of timing-out a client connection gracefully after a certain period of time.
5) Am I crazy?
Thanks for reading, Lance
[snip Lance's code]
Sounds like you are on the right track to me, but I am no expert. This is where I begin to wonder if what I am about to say is better off in a separate discussion, that is, why does everyone go directly to the inline states sessions? I have been wondering this for about 4 months (aka since I found out about the object states), I found the object states setup much much much easier to control and work with than the inline states. Possibly because most of the components are done inline or because the cookbook (in order to keep the examples simple) generally uses them? Back to the point....
I currently have an app in production, that handles files in a similar type of queue to yours. Elements are added to a queue from a directory watcher rather than a socket but the principle is the same. In my scheme I also have a series of queues in order to throttle the processing, aka a single file is processed in 4 steps, each taking a varying amount of time and cpu usage. Yielding 4 queues per direction (inbound and outbound). Each of the queues manipulates an object using the same 'process' method but the object gets blessed from one type of "Processor" to another depending on which queue it is in, this keeps my queue code generic, allows the processors to all act in the same manner (which makes writing each of them simple), while allowing different processing to occur at each stage. Finally, the process method is called within a POE::Wheel::Run as you describe to allow for multi-tasking. Doing all of this with inline states gave me headaches, when I switched to the object states it was so much clearer, the queue itself is just a standard object, things pop in and out of the queue as if it was static, but the implementation that makes it "run" is based on the fact that the queue object wraps a POE::Session, this also allows me to completely de-couple the POE events from the rest of the application, which to me is simpler and keeps my head from falling off. Ok so that is a great story and all but how does it help you?
Well I am not completely allowed to post full code, so I have stripped down my process queue, removed names to protect the innocent, removed the logging and other crap and then put it up online (at least temporarily). About the only other thing to know is that I use a home grown Exception object to throw errors, and normally the modules wouldn't have top level module names. I don't know if the code will run out of the box as I may have made some mistakes when cutting it down, but it may give you ideas. Anyone interested in it, or have suggestions of how it can be improved I would love to hear them. If the POE group is at all interested in what I have done, I will take the discussion to the next level where I work and see if I can't get it and/or parts of it opened enough to post in the cookbook or as a component.
Code can be seen at the following until I get my site finished enough to put it in a better place:
http://danconia.org/perl/POE/ProcessQueue.pm
Thoughts anyone? Use object states, make the world a more POEfect place...
http://danconia.org
