--- On Mon, 2/15/10, Marc Lehmann <[email protected]> wrote: > From: Marc Lehmann <[email protected]> > Subject: Re: sub milisec socket server write > To: "Neeraj Rai" <[email protected]> > Cc: [email protected] > Date: Monday, February 15, 2010, 12:14 AM > On Sun, Feb 14, 2010 at 05:25:10PM > -0800, Neeraj Rai <[email protected]> > wrote: > > There is a trade off between latency and cpu usage. If > we sleep 1ms after each queue drain, cpu usage is almost nil > on no activity, but latency is ~1ms. > > If sleep is omitted, the cpu usage is 100% crowding > out other procs on same box. > > Q1. is this the best way to do this ? > > Depends - the best baseline is to use an event-based > approach, and > if you need better latency, an extra real-time thread. In > general, > sub-millisecond stuff can't really be done on a normal > multiuser kernel > with high confidence - most kernels still work with time > resolutions of > around 100hz. > > Since you already have a thread for that socket, you should > try switching > to blocking I/O: You already *pay* the thread overhead, so > you might as > well let the kernel do the blocking for you. That sounds like a good idea. we'll try it. However, I am currently limited in resource and not sure when we'll get to it.
> > > Q2. we tried registering for EV_WRITE but had same > issue. > > I have no clue what "same" refers to in your mail - what > issue did you get? Sorry ;-) The issue was same cpu vs latency payoff as write in idle event. > > > Q3. ev_async seems to something that might help - each > write to queue can > > call ev_async to wake the loop. > However, before we knew about ev_async, > > we had attempted similar solution > (write to pipe and register ev_READ on > > the pipe. This sometimes ran into > deadlock when pipe was full. I think > > pipes are 4K by default and we > probably went over it. > > Yeah, the dreaded netscape bug :) > > > Does the ev_async already take care > of pipe full ? > > ev_async doesn't necessarily use a pipe (it uses eventfd if > available), > and if it uses one, it never writes more than a single > octet. is this 1 octet per write ? or is it 1 octet till it is consumed and then next octet ? > > Note that pipes are quite slow compared to memory > manipulation (and > eventfd is even slower if your threads run on different > cores, but that > should be avoided in general!). Sorry, I'm confused by this. So what's the best way to use ev_async ? Or are you hinting that I shouldn't worry about this as it will be slow no matter what. > > So if you already pay for a thread and the thread > switching, use it by > letting the thread read a queue and do blocking writes. One more complication here that I should have mentioned is that single thread handles many connections. Each connection has an input queue. These are polled non-blocking (peek and continue) for data. If data not found, next queue is checked. And this might still work well with what you suggest- poll queue and write blocking. We tried blocking writes stress test a while back ( and I had even little experience then... so it could have been some stupid obvious bug), and got a deadlock with channel full. The write blocked on the sender and the recver had read some data but trying to write more and blocked. Moving to non-block solved it as read and write were alternating and channel was being cleared when writes blocked. thanks for the prompt response. Neeraj > > -- > The > choice of a Deliantra, the > free code+content MORPG > -----==- > _GNU_ > http://www.deliantra.net > ----==-- _ > generation > ---==---(_)__ __ ____ > __ Marc Lehmann > --==---/ / _ \/ // /\ \/ / > [email protected] > -=====/_/_//_/\_,_/ /_/\_\ > _______________________________________________ libev mailing list [email protected] http://lists.schmorp.de/cgi-bin/mailman/listinfo/libev
