On Wed, Nov 7, 2012 at 1:41 AM, Ivan Zhakov <[email protected]> wrote:
> On Tue, Nov 6, 2012 at 10:24 PM, Ivan Zhakov <[email protected]> wrote:
>> On Tue, Nov 6, 2012 at 9:13 PM, Lieven Govaerts <[email protected]> wrote:
>>> Hi,
>>>
>>> On Tue, Nov 6, 2012 at 4:50 PM, Lieven Govaerts <[email protected]> wrote:
>>>> Ben,
>>>>
>>>> On Tue, Nov 6, 2012 at 4:09 PM, Ben Reser <[email protected]> wrote:
>>>>> I worked with Philip today and was able to reproduce the exact problem
>>>>> he's been seeing. I ended up having to get his full httpd.conf to
>>>>> figure it out..
>>>>>
>>>>> Ultimately the problem proved to be that he had this directive:
>>>>> Timeout 3
>>>>>
>>>>> Which would mean if we don't tend a connection for 3 seconds Apache
>>>>> will close it. Serf should be able to deal with the connection being
>>>>> closed.
>>>>>
>>>
>>> okay, so with the Timeout directive added I can reproduce this issue.
>>>
>>> What I see is that the server closes the connection in the middle of
>>> sending a response to the client. It doesn't even finalize the
>>> response first.
>>> So ra_serf is reading the data from the response bucket, but gets an
>>> APR_EOF when it needs more data than specified in the Content-Length
>>> header of the response.
>>>
>>> What is the expected behavior here, let serf close down the connection
>>> and try the request again on a new connection?
>>>
>> I think no. Timeout 3 directive means "abort the connection if client
>> didn't read data within 3 seconds"
>>
>> So most likely reason the client is busy with doing something for a
>> long time without reading data from this network connection.
>>
>> Probably it's related to the issue I found today:
>> 1. We reading REPORT faster than complete PROPFINDs/GETs
>> 2. Data from REPORT response going to spillbuf: 1mb going to memory,
>> others stored to disk.
>> 3. After completing some PROPFINDs/GETs serf update editor resumes
>> parsing data from spillbuf: 1mb in one chunk
>>
>> Two problems comes here:
>> 1. When parsing such big block we do not read data from network,
>> leading timeouts in some cases
>>
>> 2. All requests created while parsing this chunk is created in one
>> connection, that slows ra_serf
>>
> Another problem is how serf reads data from network in case of
> multiple connections: it reads data from one connection until EAGAIN.
> But if data comes from network really fast (from local server for
> example) it continue reading data from this connection, without
> reading data from other connection! Which leads time out them.
>
> See outgoing.c:read_from_connection()
>
Hi Philip,
Could please test attached serf patch. This patch should fix problem
with reading one connection for long time without reading from other
connections.
--
Ivan Zhakov
Index: outgoing.c
===================================================================
--- outgoing.c (revision 1660)
+++ outgoing.c (working copy)
@@ -900,6 +900,7 @@
apr_status_t status;
apr_pool_t *tmppool;
int close_connection = FALSE;
+ int i;
/* Whatever is coming in on the socket corresponds to the first request
* on our chain.
@@ -912,7 +913,7 @@
goto error;
/* Invoke response handlers until we have no more work. */
- while (1) {
+ for(i = 0; ; i++) {
serf_bucket_t *dummy1, *dummy2;
apr_pool_clear(tmppool);
@@ -1009,7 +1010,15 @@
/* If we received APR_SUCCESS, run this loop again. */
if (!status) {
- continue;
+ if (i > 4)
+ {
+ status = APR_EAGAIN;
+ goto error;
+ }
+ else
+ {
+ continue;
+ }
}
close_connection = is_conn_closing(request->resp_bkt);
