Hi, On Fri, Jun 23, 2006 at 01:58:25AM +0200, Thomas Frenzel wrote: > i use neon (version 0.25.5) for a minimalistic webspider. The function > "ne_request_dispatch" is used to read response bodies via callback. > Although "ne_request_create" and "ne_request_destroy" are called for > each single request, multiple requests may be done between calls of > "ne_session_create" and "ne_session_destroy". All that worked fine until > I stumbled about an apparently broken webserver that sends more data > than it specifies in the Content-Length header every now and then. > Well, neon complained that it "Could not parse response status line.". > Some gdb-sessions later I realized why neon got confused. When it > expected to find something like "HTTP/1.1 OK" at the current socket > buffer-position, actually there was old overlap-data from the previous > response-body within the same session, which had not been consumed by > "ne_request_dispatch", because of a Content-Length value that had been > too low. I expect the new response to be appended after that overlapping > data.
What kind of extra data is it the server sends; random bytes, blank lines, or what? The only way workaround the issue with neon that I can think of is to could keep a blacklist of servers which are broken like this, and disable persistent connection support next time you connect: ne_set_persist(sess, 0); or for neon >= 0.26: ne_set_session_flag(sess, NE_SESSFLAG_PERSIST, 0); Alternatively neon could be extended (in a non-default mode) to be tolerant of such errors. Regards, joe _______________________________________________ neon mailing list [email protected] http://mailman.webdav.org/mailman/listinfo/neon
