Thanks for your input, Niels. The main reason for forking is to avoid asking root to reconfig the system. My target box is Solaris 10(while I was developing with my Ubuntu 6.06). With our Solaris 10, the default limit is 256 fds per process and I (non-root user) can set it up to 2048 (with ulimit -n 2048).
For both Linux and Solaris, is there a way to change the limit (e.g. to 10k, I think it has to be done by root) for a given process instead of system wide. I think that will be more acceptable by the admin. For my implement, only the process which gets the semaphore will call event_add(&event_accept, NULL) so others will block on waiting for the semaphore instead of accepting (as my understanding). Do you see any problem for this? Do you think this will avoid the "thundering herd" problem. Even without semaphore, if I fork 10 processes which each will handle 1k connection (in single thread), then the length of accepting queue is 10 and it shouldn't be a big deal with the "thundering herd" problem, am I right? Actually do think think the semaphore is redundant at all ;-) ? I know nothing regarding the kqueue stuff (also most of the low level io stuff), for the event_reinit() you mentioned, do you think the following is my workaround, or they are not related at all (quoted from my email dated Nov 4th): >>4. sometime crashing after fork -> move event_init after fork (in other >>word, each process needs to event_init itself) >You might be better off with a multi-threaded solution. I think I have another dumb question: for multi-threaded solution, it still suffer from the limitation of the number of fd's per process, then how it could help me. Thanks, Arthur ----- Original Message ----- From: "Niels Provos" <[EMAIL PROTECTED]> To: "arthur" <[EMAIL PROTECTED]> Cc: "Marco Bambini" <[EMAIL PROTECTED]>; <libevent-users@monkey.org> Sent: Thursday, November 29, 2007 11:04 AM Subject: Re: [Libevent-users] Two questions... > On Nov 5, 2007 11:52 AM, arthur <[EMAIL PROTECTED]> wrote: > > 2. one process (I didn't use thread) can only have a limited number of > > fd's -> fork multiple processes to more accept client connection > > Some event backends do not deal with fork, for example, kqueue file > descriptors do not surive across a fork. This used to be a problem > in libevent, however, both trunk and the 1.4 branch now support an > event_reinit() method that can be used to reinitalize the event base > after a fork. > > However, I am really surprised that you are using a fork model because > you are hitting file descriptor limits. Linux and the *BSDs have no > problems dealing with > 50k file descriptors. Furthermore, accept() > used to have a "thundering herd" problem (see > http://www.citi.umich.edu/projects/linux-scalability/reports/accept.html) > which has been fixed on Linux, but I don't know if that's true for > other operating systems. You might be better off with a > multi-threaded solution. > > Niels. _______________________________________________ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
