Thanks Nick. Are you saying that between evbuffer_add_reference() call and eventual call to write the tcp buffer to the socket, the code needs to through event_loop? I added a small debug code and observed the following.
Some of the events are more process intensive then others. When such intensive events happen, we have periodic writes to TCP via evbuffer_add_reference(). I was monitoring TCP outQ (via netstat/ss) and also output of evbuffer_get_length(). I see that TCP outQ is 0, i.e. there is no data in the pipe, but after every writes, output of evbuffer_get_length() keeps going up, which means that bufferevent is buffering the data even when it could have flushed out the data inline. So it looks like the actual write to TCP socket is happening in the next iteration of event_loop but not in the same. Is there anything I can use to force immediate write (on calling evbuffer_add_reference or bufferevent_write) if socket ready to be written? I see that be_socket_flush() just returns. Can it be used or is there any other alternative? Thanks for your help in advance -Himanshu On Wed, Jan 8, 2014 at 10:27 AM, Nick Mathewson <[email protected]> wrote: > On Mon, Jan 6, 2014 at 2:48 PM, Himanshu S <[email protected]> > wrote: > > Happy new year folks, > > > > I am trying to chase down an issue which maybe be related to > bufferevent. I > > am using bufferevent library to manage my TCP sessions. For TCP writes, > > instead bufferevent_write(), I use evbuffer_add_reference() to avoid > copy of > > the data. TCP sessions exchange HELLO packets in addition to other > relevant > > data. I am seeing a peculiar behavior. > > > > A -------- B > > > > B is sending lots of data to A, while A is just sending Hello packets to > B. > > When A is sending these hello packets, I see a huge latency (in order of > > seconds) on B. Through my internal debugs, I see that A is sending the > hello > > packets when it is suppose to i.e. I see the debug on A that it called > > 'evbuffer_add_reference', but on B I don't see it. B periodically does > check > > if there is any data from A as well. > > > > Q1) Does evbuffer_add_reference() call write() syscall synchronously or > does > > it try to accumulate more data and then calls write()? > > evbuffer_add_reference calls evbuffer_invoke_callbacks, which should > indeed trigger bufferevent_socket_outbuf_cb, which should (if > appropriate conditinos are met) enable the write event with > be_socket_add, which should (when the write event's callback there is > invoked) call bufferevent_writecb(). So if the socket really is > writeable, and you have a > > > This is no different than how evbuffer_add() to an outbuf treats > other, non-reference data. > > To try to debug your latencies, you might want to instrument the > functions mentioned above to see when they're getting called, and if > they're behaving as expected. > > best wishes, > -- > Nick > *********************************************************************** > To unsubscribe, send an e-mail to [email protected] with > unsubscribe libevent-users in the body. >
