Re: Most time spent on POE/Loop/Select.pm?
Hi ALL, On Thu, Mar 4, 2010 at 4:22 AM, p...@0ne.us p...@0ne.us wrote: profiling. Also, there are numerous Loop adapters ( POE::Loop::IO_Poll, POE::XS::Loop::EPoll, etc ) that could take advantage of your platform or workload to reduce the overhead. As always, benchmark/test your code against the various loops to see if one of them is best for your workload. As taken from the POE::Kernel pod: By default POE uses its select() based loop to drive its event system. This is perhaps the least efficient loop, but it is also the most portable. POE optimizes for correctness above all. Thanks for your reply. I need to clean up my code first so I will upload later (together with the profile report). In the meantime, since I am only with Linux, how can I tell POE to use EPOLL? Thanks.
Re: Most time spent on POE/Loop/Select.pm?
Ryan Chan wrote: Hi ALL, On Thu, Mar 4, 2010 at 4:22 AM, p...@0ne.us p...@0ne.us wrote: profiling. Also, there are numerous Loop adapters ( POE::Loop::IO_Poll, POE::XS::Loop::EPoll, etc ) that could take advantage of your platform or workload to reduce the overhead. As always, benchmark/test your code against the various loops to see if one of them is best for your workload. As taken from the POE::Kernel pod: By default POE uses its select() based loop to drive its event system. This is perhaps the least efficient loop, but it is also the most portable. POE optimizes for correctness above all. Thanks for your reply. I need to clean up my code first so I will upload later (together with the profile report). In the meantime, since I am only with Linux, how can I tell POE to use EPOLL? Thanks. Hello, To use a specific event loop you should read this section of the POE::Kernel POD - http://search.cpan.org/~rcaputo/POE-1.287/lib/POE/Kernel.pm#Using_POE_with_Other_Event_Loops To use EPoll, just use your favorite CPAN client to install it then tell POE to load it via one of the mechanisms listed above. Also, please read the POE::XS::Loop::EPoll POD for more information :) ~Apocalypse
Re: Most time spent on POE/Loop/Select.pm?
Ryan Chan wrote: I am profiling my POE program which fetch HTML pages from remote site. The result show that 80% of time are spent on this file POE/Loop/Select.pm Is it normal and expected behavior? Hello, Without knowing what kind of profiling you are doing ( Devel::DProf? NYTProf? ) and the code you are writing, it's hard to answer your question. However, you are in luck as I'm the author of POE::Devel::Benchmarker and has been benchmarking POE on and off over the years. What you are seeing is very normal - let me explain a little why it's happening :) Your app fetches HTML pages, right? So you are presumably using a HTTP client - and it's making a lot of network requests, right? What's happening here is your app spends a lot of time waiting for network input before it can process them. All this waiting takes up wallclock time, and that's what your profiler is seeing. The 80% number you see does not mean the loop is crunching away at the CPU! ( obviously, a busy-wait loop is a BAD idea, ha! ) If you wanted to reduce the time spent waiting, then you need to increase the number of parallel HTTP requests. That way, your app spends more of it's time processing the HTML instead of waiting for network input. There's plenty of methods to achieve that, and you can ask on this list or browse the CPAN modules to do it. Tip: The POE developers have spent years optimizing POE the best they can. You can safely ignore the core POE modules from your profiling ( POE::Loop::Select, POE::Kernel, etc ) and focus on your own code when profiling. Also, there are numerous Loop adapters ( POE::Loop::IO_Poll, POE::XS::Loop::EPoll, etc ) that could take advantage of your platform or workload to reduce the overhead. As always, benchmark/test your code against the various loops to see if one of them is best for your workload. As taken from the POE::Kernel pod: By default POE uses its select() based loop to drive its event system. This is perhaps the least efficient loop, but it is also the most portable. POE optimizes for correctness above all. Have fun coding POE apps! ~Apocalypse
Re: Most time spent on POE/Loop/Select.pm?
Hello, On Thu, Mar 4, 2010 at 4:22 AM, p...@0ne.us p...@0ne.us wrote: Without knowing what kind of profiling you are doing ( Devel::DProf? NYTProf? ) Are you able to profile POE's code using Devel::DProf without a segfault? -- Best Regards, tszming
Re: Most time spent on POE/Loop/Select.pm?
On 03/03/2010 03:22 PM, p...@0ne.us wrote: Tip: The POE developers have spent years optimizing POE the best they can. You can safely ignore the core POE modules from your profiling ( POE::Loop::Select, POE::Kernel, etc ) and focus on your own code when profiling. I both agree and disagree with this. POE's core modules do, in my experience, perform very well and I always start out assuming that they are not the performance problem. This does not mean you should ignore the usage you see in these modules. High usage in the core modules might indicate an issue in your application. Events are expensive. Maybe some events could be bundled into a single pass through POE rather than several. Maybe you are firing events that could just as easily be function calls. That said, based on the minimal information given in the original post, I think Apocalypse probably correct about what you are seeing in this particular case. -Andrew
