Rajeev Bansal wrote: > I am observing the following error "504 Connect to > www.youtube.com:80failed: Too many open files" when I generate lots of > HTTP traffic using > some web-crawler on Polipo. I understand there are chances system is > running out of descriptors and hence not able to open new TCP scoket > which is resulting into this situation. Can someone help what can be > the best way for not to arrive in this situation.
On which operating system are you running polipo? If it is linux, can you give us the output of `lsof -c polipo' while you are running the crawler? Which version of Polipo are you using? Are you shure Polipo uses up all those file descriptors and not something else? In linux, you can increase the amount of avaible file descriptors as described here: http://linux-quirks.blogspot.ch/2011/02/too-many-open-files.html You can also try to execute `ulimit -n 4096' (or higher) in the shell from which you start polipo. I tried to recreate this situation with an unreasonable test setting and the cache does not use up more file descriptors than it should. Note that there is the option maxDiskEntries. Could it be that you have set this to something too high? It could also be that polipo opens too many connections to servers. I don't know how polipo manages this amount. You could however set maxConnectionAge to something small. Greetings Urs ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct _______________________________________________ Polipo-users mailing list Polipo-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/polipo-users
