Re: debugging Squid ICAP interface
On 10/12/2010 11:51 AM, Marcus Kool wrote: I also have seen Squid behaving unexpectedly (2 minutes timeouts where it seems not handle any request from a browser, assertion failure). If you can reproduce this, please file a bug report with details, especially if this happens to a recent Squid version. I have various observations and questions about the Squid ICAP interface and like to discuss these with the persons who wrote or know much about the ICAP client part of Squid. I like to know with whom I can discuss this and which mailing list to use. As Henrik said, you came to the right place. Cheers, Alex.
Re: debugging Squid ICAP interface
tis 2010-10-12 klockan 14:51 -0300 skrev Marcus Kool: I have various observations and questions about the Squid ICAP interface and like to discuss these with the persons who wrote or know much about the ICAP client part of Squid. I like to know with whom I can discuss this and which mailing list to use. This list is the right place for such discussions. Regards Henrik
debugging Squid ICAP interface
Hello, My name is Marcus Kool, author of ufdbGuard - a URL redirector for Squid, and I have started development of an ICAP-based URL filter. As with all new developments, the code of the ICAP server undoubtedly has some bugs that need to be investigated and fixed. I also have seen Squid behaving unexpectedly (2 minutes timeouts where it seems not handle any request from a browser, assertion failure). I have various observations and questions about the Squid ICAP interface and like to discuss these with the persons who wrote or know much about the ICAP client part of Squid. I like to know with whom I can discuss this and which mailing list to use. Thanks, Marcus
Re: Squid-ICAP 2.6
tis 2007-02-20 klockan 15:05 +0100 skrev Christophe Boyanique: I would like to give a try to the 2.6 branch of the ICAP patch and I didn't manage to apply the patch available at: http://devel.squid-cache.org/projects.html#icap I tried on squid-2.6.STABLE9-20070220, squid-2.6.STABLE9 and squid-2.6.STABLE8 without success. Is there anywhere: - an history if the icap patch ? - a changelog I preliminary history/changelog view of the devel.squid-cache.org projects can be found here: http://www.squid-cache.org/~hno/changesets/squid/ - a documentation explaining on which squid version an icap patch apply ? From the above: Squid-2 around 2007/01/31. Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Squid-ICAP 2.6
Hello, I would like to give a try to the 2.6 branch of the ICAP patch and I didn't manage to apply the patch available at: http://devel.squid-cache.org/projects.html#icap I tried on squid-2.6.STABLE9-20070220, squid-2.6.STABLE9 and squid-2.6.STABLE8 without success. Is there anywhere: - an history if the icap patch ? - a changelog - a documentation explaining on which squid version an icap patch apply ? Regards, Christophe.
Re: Squid-ICAP 2.6
Bonjour Jeremy, Here is the answer I got, and I managed to patch squid correctly thanks to it: http://www.squid-cache.org/mail-archive/squid-users/200702/0143.html Would you have a corrected patch ? Because I am applying manually and it is a bit long and not obvious to not make a mistake. Christophe.
Re: Squid-ICAP 2.6
Christophe Boyanique a écrit : Hello, I would like to give a try to the 2.6 branch of the ICAP patch and I didn't manage to apply the patch available at: http://devel.squid-cache.org/projects.html#icap I tried on squid-2.6.STABLE9-20070220, squid-2.6.STABLE9 and squid-2.6.STABLE8 without success. Is there anywhere: - an history if the icap patch ? - a changelog - a documentation explaining on which squid version an icap patch apply ? Regards, Christophe. Here is the answer I got, and I managed to patch squid correctly thanks to it: http://www.squid-cache.org/mail-archive/squid-users/200702/0143.html Hope it helps -- Jérémy Lardon Laboratoire DIOM, équipe SATIn - Doctorant 04 77 48 50 34
Re: Squid-ICAP 2.6
Hi Christophe, I think the best is to get the icap branch from cvs: cvs -d :pserver:[EMAIL PROTECTED]:/cvsroot/squid co -r icap-2_6 squid After that run the bootstrap.sh script which included in cvs sources. Do not try to correct rejected patches. The last merge with main squid26 done after STABLE9 released and after some more changes added by squid developers to the main squid26 branch. The current patch does not apply to any stable release but to the latest(I think) squid26 cvs sources. Sorry about that. Regards, Christos Christophe Boyanique wrote: Hello, I would like to give a try to the 2.6 branch of the ICAP patch and I didn't manage to apply the patch available at: http://devel.squid-cache.org/projects.html#icap I tried on squid-2.6.STABLE9-20070220, squid-2.6.STABLE9 and squid-2.6.STABLE8 without success. Is there anywhere: - an history if the icap patch ? - a changelog - a documentation explaining on which squid version an icap patch apply ? Regards, Christophe.
Re: Squid-ICAP problem (bug?)
Christophe Boyanique wrote: Yes we tried it and this leads to segfault too. :-( By reading the log I just noticed that with your modification the icapReqModPassHttpBody is not called: This is for the specific request (POST): 2006/11/24 14:15:22| handing request bodies in ICAP REQMOD 2006/11/24 14:15:22| icapReqModReadHttpBody: FD 38 called 2006/11/24 14:15:22| icapReqModReadHttpBody: read returns 33 . 2006/11/24 14:15:22| icap_reqmod.c:882 chunk_size=-2 2006/11/24 14:15:22| icap_reqmod.c:892 http_entity.callback=(nil) 2006/11/24 14:15:22| icap_reqmod.c:894 http_entity.buf.size=27 2006/11/24 14:15:22| icap_reqmod.c:896 http_entity.callback_bufsize=0 2006/11/24 14:15:22| icapSendRespMod: Create a new connection to icap server service_4/icap://xx.xx.xx.xx:1344/wwrespmod/?wwprofile=HTTP_ The strange here is that the http_entity.callback=(nil) In my test cases, this never happens... II don't know why . So why is called icapReqModPassHttpBody if the request does not contain a body ? And why dos it find 27 bytes of data which is exactly the body of the previous request ? This looks very strange, isn't it ? Yes it is .
Re: Squid-ICAP problem (bug?)
Hi all,
Tsantilas Christos wrote :
If you are still looking for the solution, try the following patch in
icap_reqmod.c file. With this patch the icapReqModPassHttpBody function
called if the icap-reqmod.http_entity.buf.size is zero, and at this
phase can handle correctly, the case that the
icap-flags.reqmod_http_entity_eof==1.
Unfortunately it still segfaults.
But, I had time to work on my client's test platform and I spent a whole
day hacking icap_reqmod.c.
The only modification (except adding excessive debug logging) is that one:
in icapReqModReadHttpBody function replacing the test
if (icap-reqmod.http_entity.bytes_read = icap-request-content_length)
by
if (icap-chunk_size 0)
which correct the initial problem ie not reading the 0\r\n marking thee
end of the chunk.
The new problem is that squid correctly reads the end of the chunk but
dies immediatly.
The dying occurs in the icapReqModPassHttpBody function and I spent many
time to find why. In fact I focussed on the call of this function from
the icapReqModReadHttpBody function and I didn't manage to find the
logic of the test and the function parameters (because I don't know well
the squid and icap patch code).
Anyway this I found that the function was called with icap = 0 parameter
(I spent too more time to search a buf=0 or callback=0); so I added a
simple test in the beginning of the function:
static void
icapReqModPassHttpBody(IcapStateData * icap, char *buf, size_t size,
CBCB * callback, void *cbdata)
{
debug(81, 3) (%s:%d: icapReqModPassHttpBody: called\n, __FILE__,
__LINE__);
if (!icap) {
debug(81, 1) (%s:%d: icapReqModPassHttpBody: FD %d called with
icap = (nil) !\n, __FILE__, __LINE__);
return;
}
I added this to prevent segfaulting but without knowing if the return
would lead to a correct behaviour. And it seems that is is ok. We made
some tests and we did not have any more segfaults.
I then tried with gdb (I should have used it before but I am not
familiar with it) to get a backtrace as you asked in a previous mail and
I saw that the icapReqModPassHttpBody call with icap=0 is not called
from the icapReqModReadHttpBody:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1218496992 (LWP 9508)]
0x08087411 in icapReqModPassHttpBody (icap=0x0, buf=0x0,
size=4294967295, callback=0, cbdata=0x8c95e00) at icap_reqmod.c:933
933 icap_reqmod.c: Aucun fichier ou repertoire de ce type.
in icap_reqmod.c
(gdb) bt
#0 0x08087411 in icapReqModPassHttpBody (icap=0x0, buf=0x0,
size=4294967295, callback=0, cbdata=0x8c95e00) at icap_reqmod.c:933
#1 0x08083905 in requestAbortBody (request=0x8c94578) at HttpRequest.c:215
#2 0x0807a183 in httpStateFree (fd=9, data=0x8cc72a0) at http.c:69
#3 0x08065ccd in commCallCloseHandlers (fd=44) at comm.c:573
#4 0x08065dec in comm_close (fd=44) at comm.c:655
#5 0x0807b32c in httpReadReply (fd=44, data=0x8cc72a0) at http.c:852
#6 0x08067749 in comm_poll (msec=13) at comm_select.c:445
#7 0x0808fc77 in main (argc=4, argv=0xbfff9494) at main.c:754
Here is a sumup of the logs when icap value is tested and squid does not
segfault anymore:
2006/12/01 18:18:19| icapReqModParseHttpRequest: successfully parsed the
HTTP request
2006/12/01 18:18:19| handing request bodies in ICAP REQMOD
2006/12/01 18:18:19| icap_reqmod.c:859: icapReqModReadHttpBody: FD 53 called
2006/12/01 18:18:19| icap_reqmod.c:861: icapReqModReadHttpBody: read
returns 33
2006/12/01 18:18:19| icap_reqmod.c:872: icapReqModReadHttpBody: Before
icapParseChunkedBody bytes_read = 0
2006/12/01 18:18:19| icap_common.c:695: chunk_size=0
2006/12/01 18:18:19| icap_common.c:702: bufOffset=0, len=33
2006/12/01 18:18:19| icap_common.c:705: bufOffset=0, len=33
2006/12/01 18:18:19| icap_common.c:604: icapParseChunkSize:
buf=0x8ccd1b8, len=33
2006/12/01 18:18:19| icap_common.c:607: icapParseChunkSize: start=0
2006/12/01 18:18:19| icapLineLength: returning 4
2006/12/01 18:18:19| icap_common.c:646: icapParseChunkSize: start=0, end=2
2006/12/01 18:18:19| icap_common.c:672: icapParseChunkSize: return
nextStart=4
2006/12/01 18:18:19| icap_common.c:716: got chunksize 27, new offset 4
2006/12/01 18:18:19| icap_common.c:723: X
2006/12/01 18:18:19| icap_common.c:705: bufOffset=31, len=33
2006/12/01 18:18:19| icap_common.c:604: icapParseChunkSize:
buf=0x8ccd1d7, len=2
2006/12/01 18:18:19| icap_common.c:607: icapParseChunkSize: start=0
2006/12/01 18:18:19| icapLineLength: returning 2
2006/12/01 18:18:19| icap_common.c:607: icapParseChunkSize: start=2
2006/12/01 18:18:19| icap_reqmod.c:876: icapReqModReadHttpBody: After
icapParseChunkedBody bytes_read = 27
2006/12/01 18:18:19| icap_reqmod.c:884 icapReqModReadHttpBody: chunk_size=0
Here is the first modification: even if we read the 27 bytes of the body
we continue reading because chunk_size is not -2:
2006/12/01 18:18:19| icap_reqmod.c:892 icapReqModReadHttpBody: call
commSetSelect -
Re: Squid-ICAP problem (bug?)
Tsantilas Christos a e'crit :
Hello Christos,
Maybe I was not clear on my last mail. Sorry.
You were but I forgot to mention that I tried what you adviced.
But in the same function replace the test:
if (icap-reqmod.http_entity.callback
icap-reqmod.http_entity.buf.size) {
icapReqModPassHttpBody(icap,
icap-reqmod.http_entity.callback_buf,
icap-reqmod.http_entity.callback_bufsize,
with:
if (icap-reqmod.http_entity.callback) {
icapReqModPassHttpBody(icap,
icap-reqmod.http_entity.callback_buf,
icap-reqmod.http_entity.callback_bufsize,
Did you try it?
Yes we tried it and this leads to segfault too.
and we exit the icapReqModReadHttpBody function without calling
icapReqModPassHttpBody because http_entity.buf.size is 0.
With the previous change this function must called here.
Did you make this change? Are you still seeing segfaults?
I did not test extensively but it seems that calling
icapReqModPassHttpBody without testing if http_entity.buf.size is 0 will
work in this very specific case (when end chunk is read in an extra
pass) but will lead to segfault in cases which used to work perfectly
before changing this test.
In my tests I am not seeing any problem any more. But maybe I am losing
something
I don't have any magic recipe to reproduce the problem but it occurs
when parsing the request's body in REQMOD and that occurs generally when
doint a POST request (as there is no body in a GET request).
With your modification, we ends with segfaults during GET request !
By reading the log I just noticed that with your modification the
icapReqModPassHttpBody is not called:
This is for the specific request (POST):
2006/11/24 14:15:22| handing request bodies in ICAP REQMOD
2006/11/24 14:15:22| icapReqModReadHttpBody: FD 38 called
2006/11/24 14:15:22| icapReqModReadHttpBody: read returns 33
2006/11/24 14:15:22| icap_common.c:695: chunk_size=0
2006/11/24 14:15:22| icap_common.c:702: bufOffset=0, len=33
2006/11/24 14:15:22| icap_common.c:705: bufOffset=0, len=33
2006/11/24 14:15:22| icapParseChunkSize: buf=0x997fbe0, len=33
2006/11/24 14:15:22| icapParseChunkSize: start=0
2006/11/24 14:15:22| icapLineLength: returning 4
2006/11/24 14:15:22| icapParseChunkSize: start=0, end=2
2006/11/24 14:15:22| icapParseChunkSize: return nextStart=4
2006/11/24 14:15:22| got chunksize 27, new offset 4
2006/11/24 14:15:22| icap_common.c:723: X
2006/11/24 14:15:22| icap_common.c:705: bufOffset=31, len=33
2006/11/24 14:15:22| icapParseChunkSize: buf=0x997fbff, len=2
2006/11/24 14:15:22| icapParseChunkSize: start=0
2006/11/24 14:15:22| icapLineLength: returning 2
2006/11/24 14:15:22| icapParseChunkSize: start=2
2006/11/24 14:15:22| icap_reqmod.c:882 chunk_size=0
2006/11/24 14:15:22| icap_reqmod.c:892 http_entity.callback=(nil)
2006/11/24 14:15:22| icap_reqmod.c:894 http_entity.buf.size=27
2006/11/24 14:15:22| icap_reqmod.c:896 http_entity.callback_bufsize=0
2006/11/24 14:15:22| icapReqModReadHttpBody: FD 38 called
2006/11/24 14:15:22| icapReqModReadHttpBody: read returns 5
2006/11/24 14:15:22| icap_common.c:695: chunk_size=0
2006/11/24 14:15:22| icap_common.c:702: bufOffset=0, len=7
2006/11/24 14:15:22| icap_common.c:705: bufOffset=0, len=7
2006/11/24 14:15:22| icapParseChunkSize: buf=0x997fbe0, len=7
2006/11/24 14:15:22| icapParseChunkSize: start=0
2006/11/24 14:15:22| icapLineLength: returning 2
2006/11/24 14:15:22| icapParseChunkSize: start=2
2006/11/24 14:15:22| icapLineLength: returning 3
2006/11/24 14:15:22| icapParseChunkSize: start=2, end=3
2006/11/24 14:15:22| icapParseChunkSize: return nextStart=5
2006/11/24 14:15:22| got chunksize -2, new offset 5
2006/11/24 14:15:22| zero end chunk reached
2006/11/24 14:15:22| icap_reqmod.c:882 chunk_size=-2
2006/11/24 14:15:22| icap_reqmod.c:892 http_entity.callback=(nil)
2006/11/24 14:15:22| icap_reqmod.c:894 http_entity.buf.size=27
2006/11/24 14:15:22| icap_reqmod.c:896 http_entity.callback_bufsize=0
2006/11/24 14:15:22| icapSendRespMod: Create a new connection to icap
server service_4/icap://xx.xx.xx.xx:1344/wwrespmod/?wwprofile=HTTP_
But what is very strange is:
- respmod seems not to be called;
- the next request which is a GET is handled strangely:
2006/11/24 14:15:34| icapParseEncapsulated: res-hdr=-1, req-hdr=0,
null-body=791, res-body=-1, req-body=-1, opt-body=-1
2006/11/24 14:15:34| icapReqModReadIcapPart: directResponse=0 from ICAP
server service_3/icap://xx.xx.xx.xx:1344/wwreqmod/?wwprofile=HTTP_
2006/11/24 14:15:34| icapReqModReadHttpHdrs:
2006/11/24 14:15:34| expect=791
2006/11/24 14:15:34| so_far=0
2006/11/24 14:15:34| needed=791
2006/11/24 14:15:34| icapReqModReadHttpHdrs: read 791 bytes
2006/11/24 14:15:34| icapReqModReadHttpHdrs: read the entire request headers
2006/11/24 14:15:34| icapReqModParseHttpRequest: URI is
'http://x.xxx/rtn_FR.gif'
2006/11/24 14:15:34| icapReqModParseHttpRequest: Client HTTP version 1.1.
2006/11/24 14:15:34|
Re: Squid-ICAP problem (bug?)
Hi,
From your logs what I am seeing is that the
icap-reqmod.http_entity.buf.size==0 so the icapRqModPassHttpBody must
not called, but it is called. It is strange.
Did you touch something else in code? Can you run squid in gdb and send
a backtrace after the crash?
Regards,
Christos
Christophe Boyanique wrote:
..
After the conditionnal call to commSetSelect there is a conditionnal
call to icapReqModPassHttpBody and this where is the problem:
if (icap-reqmod.http_entity.callback
icap-reqmod.http_entity.buf.size) {
icapReqModPassHttpBody(icap,
icap-reqmod.http_entity.callback_buf,
icap-reqmod.http_entity.callback_bufsize,
icap-reqmod.http_entity.callback,
icap-reqmod.http_entity.callback_data);
icap-reqmod.http_entity.callback = NULL;
cbdataUnlock(icap-reqmod.http_entity.callback_data);
}
When calling this function, the test is on
icap-reqmod.http_entity.buf.size but the parameter sent to the function
is cap-reqmod.http_entity.callback_bufsize. Is it normal ?
I added some logs and here is the output:\
2006/11/14 13:43:25| zero end chunk reached
2006/11/14 13:43:25| icap_reqmod.c:882 chunk_size=-2
2006/11/14 13:43:25| icap_reqmod.c:892 http_entity.callback=0x807c994
2006/11/14 13:43:25| icap_reqmod.c:894 http_entity.buf.size=0
2006/11/14 13:43:25| icap_reqmod.c:896 http_entity.callback_bufsize=8192
2006/11/14 13:43:25| icapReqModPassHttpBody: called
FATAL: Received Segment Violation...dying.
Perhaps there is a problem with parameters passed to
icapReqModPassHttpBody ?
Re: Squid-ICAP problem (bug?)
Hello,
Thanks for you support.
I tried the patch from Tsantilas:
if (icap-chunk_size 0)
icap-flags.reqmod_http_entity_eof = 1;
instead of:
if (icap-reqmod.http_entity.bytes_read = icap-request-content_length)
icap-flags.reqmod_http_entity_eof = 1;
And that corrects the initial problem: remaining data waiting on the
wire is read (\r\n0\r\n).
But there is a new problem.
After the conditionnal call to commSetSelect there is a conditionnal
call to icapReqModPassHttpBody and this where is the problem:
if (icap-reqmod.http_entity.callback
icap-reqmod.http_entity.buf.size) {
icapReqModPassHttpBody(icap,
icap-reqmod.http_entity.callback_buf,
icap-reqmod.http_entity.callback_bufsize,
icap-reqmod.http_entity.callback,
icap-reqmod.http_entity.callback_data);
icap-reqmod.http_entity.callback = NULL;
cbdataUnlock(icap-reqmod.http_entity.callback_data);
}
When calling this function, the test is on
icap-reqmod.http_entity.buf.size but the parameter sent to the function
is cap-reqmod.http_entity.callback_bufsize. Is it normal ?
I added some logs and here is the output:
2006/11/14 13:43:25| icapReqModReadHttpBody: FD 38 called
2006/11/14 13:43:25| icapReqModReadHttpBody: read returns 33
2006/11/14 13:43:25| icap_common.c:695: chunk_size=0
2006/11/14 13:43:25| icap_common.c:702: bufOffset=0, len=33
2006/11/14 13:43:25| icap_common.c:705: bufOffset=0, len=33
2006/11/14 13:43:25| icapParseChunkSize: buf=0x9e362d0, len=33
2006/11/14 13:43:25| icapParseChunkSize: start=0
2006/11/14 13:43:25| icapLineLength: returning 4
2006/11/14 13:43:25| icapParseChunkSize: start=0, end=2
2006/11/14 13:43:25| icapParseChunkSize: return nextStart=4
2006/11/14 13:43:25| got chunksize 27, new offset 4
2006/11/14 13:43:25| icap_common.c:723: X
2006/11/14 13:43:25| icap_common.c:705: bufOffset=31, len=33
2006/11/14 13:43:25| icapParseChunkSize: buf=0x9e362ef, len=2
2006/11/14 13:43:25| icapParseChunkSize: start=0
2006/11/14 13:43:25| icapLineLength: returning 2
2006/11/14 13:43:25| icapParseChunkSize: start=2
2006/11/14 13:43:25| icap_reqmod.c:882 chunk_size=0
2006/11/14 13:43:25| icap_reqmod.c:892 http_entity.callback=(nil)
2006/11/14 13:43:25| icap_reqmod.c:894 http_entity.buf.size=27
2006/11/14 13:43:25| icap_reqmod.c:896 http_entity.callback_bufsize=0
2006/11/14 13:43:25| icapService: type=ICAP_SERVICE_REQMOD_POSTCACHE
2006/11/14 13:43:25| icapService: checking service service_3/id=0
2006/11/14 13:43:25| icapService: checking service service_4/id=0
2006/11/14 13:43:25| icapService: no service found
2006/11/14 13:43:25| icapReqModPassHttpBody: called
2006/11/14 13:43:25| icapReqModPassHttpBody: entity buf size = 27
2006/11/14 13:43:25| icapReqModPassHttpBody: giving 27 bytes to other side
2006/11/14 13:43:25| icapReqModPassHttpBody: entity buf size now = 0
2006/11/14 13:43:25| icapReqModPassHttpBody: called
2006/11/14 13:43:25| icapReqModPassHttpBody: entity buf size = 0
2006/11/14 13:43:25| icapReqModReadHttpBody: FD 38 called
2006/11/14 13:43:25| icapReqModReadHttpBody: read returns 5
2006/11/14 13:43:25| icap_common.c:695: chunk_size=0
2006/11/14 13:43:25| icap_common.c:702: bufOffset=0, len=7
2006/11/14 13:43:25| icap_common.c:705: bufOffset=0, len=7
2006/11/14 13:43:25| icapParseChunkSize: buf=0x9e362d0, len=7
2006/11/14 13:43:25| icapParseChunkSize: start=0
2006/11/14 13:43:25| icapLineLength: returning 2
2006/11/14 13:43:25| icapParseChunkSize: start=2
2006/11/14 13:43:25| icapLineLength: returning 3
2006/11/14 13:43:25| icapParseChunkSize: start=2, end=3
2006/11/14 13:43:25| icapParseChunkSize: return nextStart=5
2006/11/14 13:43:25| got chunksize -2, new offset 5
2006/11/14 13:43:25| zero end chunk reached
2006/11/14 13:43:25| icap_reqmod.c:882 chunk_size=-2
2006/11/14 13:43:25| icap_reqmod.c:892 http_entity.callback=0x807c994
2006/11/14 13:43:25| icap_reqmod.c:894 http_entity.buf.size=0
2006/11/14 13:43:25| icap_reqmod.c:896 http_entity.callback_bufsize=8192
2006/11/14 13:43:25| icapReqModPassHttpBody: called
FATAL: Received Segment Violation...dying.
Perhaps there is a problem with parameters passed to
icapReqModPassHttpBody ?
Christophe.
Re: Squid-ICAP problem (bug?)
ons 2006-11-08 klockan 16:26 +0100 skrev Christophe Boyanique: We have read enough data (27 bytes) to have all the body (as content-length is 27), so we set the eof flag. if (!icap-flags.reqmod_http_entity_eof) commSetSelect(fd, COMM_SELECT_READ, icapReqModReadHttpBody, icap, 0); eof in ICAP is that 0 chunk... so something is not right here in the ICAP patch. Note: As Adrian said most efforts is focused on the Squid-3 release, which also includes ICAP support. Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Re: Squid-ICAP problem (bug?)
Henrik Nordstrom a écrit : We have read enough data (27 bytes) to have all the body (as content-length is 27), so we set the eof flag. if (!icap-flags.reqmod_http_entity_eof) commSetSelect(fd, COMM_SELECT_READ, icapReqModReadHttpBody, icap, 0); eof in ICAP is that 0 chunk... so something is not right here in the ICAP patch. The problem is that this 0\r\n arrives after the first read of the 27 first bytes. Note: As Adrian said most efforts is focused on the Squid-3 release, which also includes ICAP support. I understand that but the problem is that we have this bug in production on heavy loaded site (we took several weeks to find it out) and we have to correct it anyway (and I don't think think that targetting Squid3 is possible (I thought Squid3 was considered beta?)). Regards, Christophe.
Re: Squid-ICAP problem (bug?)
Hi, I send this mail from un account which is not subscribed to the mailing list so I am re-sending it. Sory about that. Please to the mailing list moderator to ingore the first mail. - Hi Christophe, First of all squid3 has a better support for icap protocol now. Maybe it is better to use squid3 as icap client. However, yes the problem looks that exists in squid25-icap and squid26-icap. In fle icap_reqmod.c, in function icapReqModReadHttpBody near line 878-880 try this small patch : -if (icap-reqmod.http_entity.bytes_read = icap-request-content_length) - icap-flags.reqmod_http_entity_eof = 1; + // if (icap-reqmod.http_entity.bytes_read = icap-request-content_length ) + if (icap-chunk_size 0 ) + icap-flags.reqmod_http_entity_eof = 1; It must marks the http_entity_eof when it has read the 0\r\n\r\n chunk. Works for me but I did not test it enough Regards, Christos Christophe Boyanique wrote: Hello, I think I have found a bug in the Squid-ICAP patch and I would like to have your opinion about it. I use a tcpdump strace and verbose log to track a problem which occurs sometimes during a respmod request but is triggered during the reqmod answer analysis I think. We use squid-2.5.STABLE14 and Squid-Icap patch from 2006/06/26. .
Re: Squid-ICAP problem (bug?)
On Thu, Nov 09, 2006, Christophe Boyanique wrote: Note: As Adrian said most efforts is focused on the Squid-3 release, which also includes ICAP support. I understand that but the problem is that we have this bug in production on heavy loaded site (we took several weeks to find it out) and we have to correct it anyway (and I don't think think that targetting Squid3 is possible (I thought Squid3 was considered beta?)). I think it'd be great if the bug were fixed in squid-2.6-icap - but this kinda stuff also needs to go into squid-3. Squid-3 won't be beta for long if I have anything to say about it in a couple of weeks. Adrian
Re: Squid-ICAP problem (bug?)
tor 2006-11-09 klockan 12:37 +0200 skrev Tsantilas Christos: + if (icap-chunk_size 0 ) + icap-flags.reqmod_http_entity_eof = 1; Shouldn't that be = 0 or maybe even == 0? Regards Henrik signature.asc Description: Detta är en digitalt signerad meddelandedel
Re: Squid-ICAP problem (bug?)
Hi, I think it should be icap-chunk_size == -2. In icapParseChunkSize function in common_icap.c file the icap-chunk_size set to -2 when the 0\r\n\r\n sequence parsed Regards, Christos Henrik Nordstrom wrote: tor 2006-11-09 klockan 12:37 +0200 skrev Tsantilas Christos: + if (icap-chunk_size 0 ) + icap-flags.reqmod_http_entity_eof = 1; Shouldn't that be = 0 or maybe even == 0? Regards Henrik
Squid-ICAP problem (bug?)
Hello, I think I have found a bug in the Squid-ICAP patch and I would like to have your opinion about it. I use a tcpdump strace and verbose log to track a problem which occurs sometimes during a respmod request but is triggered during the reqmod answer analysis I think. We use squid-2.5.STABLE14 and Squid-Icap patch from 2006/06/26. Here is the request sent by Squid to the ICAP server (webwasher): --- begin cut --- REQMOD icap://10.17.30.41:1344/wwreqmod/?wwprofile=HTTP_Sortante ICAP/1.0 Encapsulated: req-hdr=0, req-body=1021 X-Client-IP: 129.182.109.109 X-Authenticated-User: V2lOVDovL2VyaWM= POST http://bubv-u.fraz.bull.fr/ulysse/SelTypF2.jsp HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */* Referer: http://bubv-u.fraz.bull.fr/ulysse/SelTypF1.jsp Accept-Language: fr Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322) Host: bubv-u.fraz.bull.fr Content-Length: 27 Proxy-Connection: Keep-Alive Pragma: no-cache Cookie: REFUTI=27930; LANGUE=FR; NOM=BUL78604FR; STE=BUL; USER=UBUL_BUL78604FR; JSESSIONID=FE83A4E238E9375B7A9C55F6EE76C7EA; WS_LastUid=frank-e; WS_UsrRef=FRANK-E; WS_UsrLng=FR; WS_UsrSid=9743; WS_UsrAut=ClBiRTt7Nz59IjBscjXnrypOQWMAAwd7cnRcW3lZNFd8T3t3JGtgbSxMPXFyc3RUNFhzfjR%2bcjxQVjRcZWEmTGBLLVI6KCM8SHYoTjhSXm97WVtBOnggb3B1KEdhbyZ7ZS1cLywvIDZoLE5lNSxtK09YXEY1QlE1fHBobztxfEsyPSxcTA== Proxy-Authorization: Basic ZXJpYzplcmlj 1b GOTO=TRI=ORDER=COD=LIB= 0 end cut The request seems ok, the string GOTO=TRI=ORDER=COD=LIB= is 27 characters long as specified in the Content-Lenght header. The request is sent successfully to webwasher: 2006/11/02 15:10:23| icapReqModSendBodyChunk: FD 34 wrote 1208 errflag 0. 2006/11/02 15:10:23| icapReqModBodyHandler: writing chunk size 27 2006/11/02 15:10:23| icapReqModSendBodyChunk: FD 34 wrote 33 errflag 0. 2006/11/02 15:10:23| icapReqModBodyHandler: writing chunk size 0 2006/11/02 15:10:23| icapSendReqModDone: FD 34: size 5: errflag 0. Then comes the response from webwasher. The problem is that the response seems to be split in several packets. Here is the first part: --- begin cut --- ICAP/1.0 200 OK Cache-Control: max-age=3600 Encapsulated: req-hdr=0, req-body=1021 ISTAG: 001-000-000162 X-Attribute-Cacheability: user=V2lOVDovL2VyaWM= X-ICAP-Profile: HTTP_Sortante POST http://bubv-u.fraz.bull.fr/ulysse/SelTypF2.jsp HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */* Accept-Encoding: gzip, deflate Accept-Language: fr Content-Length: 27 Content-Type: application/x-www-form-urlencoded Cookie: REFUTI=27930; LANGUE=FR; NOM=BUL78604FR; STE=BUL; USER=UBUL_BUL78604FR; JSESSIONID=24FE81492AD05440B0434197D087AC18; WS_LastUid=frank-e; WS_UsrRef=FRANK-E; WS_UsrLng=FR; WS_UsrSid=3168; WS_UsrAut=ClBiRTt7Nz59IjBsckIn7iooyDAAAwd7cnRcW3lZNFd8T3t3JGtgbSxMPXFyc3RUNFhzfjR%2bcjxQVjRcZWEmTGBLLVI6KCM8SHYoTjhSXm97WVtBOnggb3B1KEdhbyZ7ZS1cLywvIDZoLE5lNSxtK09YXEY1QlE1fHBobztxfEsyPSxcTA== Host: bubv-u.fraz.bull.fr Pragma: no-cache Proxy-Authorization: Basic ZXJpYzplcmlj Proxy-Connection: Keep-Alive Referer: http://bubv-u.fraz.bull.fr/ulysse/SelTypF1.jsp User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322) 1B GOTO=TRI=ORDER=COD=LIB= end cut And the second part: --- begin cut --- 0 end cut The problem is that the end of the chunk (0\r\n\r\n) is not sent (and read) in the same stream that the beginning of the chunk. If we dig in the squid-icap source code we see: 2006/11/02 15:10:23| icapReqModReadHttpBody: FD 34 called 2006/11/02 15:10:23| icapReqModReadHttpBody: read returns 33 - this is the first part read 2006/11/02 15:10:23| icap_common.c:695: chunk_size=0 2006/11/02 15:10:23| icap_common.c:702: bufOffset=0, len=33 2006/11/02 15:10:23| icap_common.c:705: bufOffset=0, len=33 2006/11/02 15:10:23| icapParseChunkSize: buf=0x8f832e8, len=33 2006/11/02 15:10:23| icapParseChunkSize: start=0 2006/11/02 15:10:23| icapLineLength: returning 4 - this is the part 1b\r\n 2006/11/02 15:10:23| icapParseChunkSize: start=0, end=2 - this is 1b 2006/11/02 15:10:23| icapParseChunkSize: return nextStart=4 2006/11/02 15:10:23| got chunksize 27, new offset 4 - 1b - 27 bytes, to read at offset 4 (after 1b\r\n) 2006/11/02 15:10:23| icap_common.c:723: X 2006/11/02 15:10:23| icap_common.c:705: bufOffset=31, len=33 2006/11/02 15:10:23| icapParseChunkSize: buf=0x8f83307, len=2 2006/11/02 15:10:23| icapParseChunkSize: start=0 2006/11/02 15:10:23| icapLineLength: returning 2 2006/11/02 15:10:23| icapParseChunkSize: start=2 This is where is the first problem: after reading the first chunk size (1b) and the 27 bytes of data, the buffer only
2 fixes for the squid icap implementation
Hi, Could someone add the 2 attached patches to the squid icap client? Here's a description of what they fix... 15minutefix.patch = I noticed that if an icap server replied with a 204 response, the download would continue without being forwarded to icap but it would then abort after 15 minutes. This was because the read timeout on the icap connection wasn't being cancelled. This fix cancels the timeout which avoids the aborted downloads. squidcrashfix.patch === A fix that I submitted some time ago (squidgrowthfix.patch) caused squid to crash under heavy load. This was because the deferred read check I added relied on the store entry object being present but didn't lock it. Under heavy load the store entry was sometimes freed before the defer read check which caused squid to crash. This patch maintains a lock on the store entry object and also has some other code to make sure the store entry object is unlocked and freed when it is no longer required. Thanks, Graeme This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com 15minutefix.patch Description: 15minutefix.patch squidcrashfix.patch Description: squidcrashfix.patch
Re: 2 fixes for the squid icap implementation
Hi Graeme,
I did not verified the problems but looks that the problems exist.
About the first patch I am proposing a somehow different approach.
When squid-icap takes a 204 response immediately releases the connection
and put it to connections poll so it can be used for other icap requests.
The icap connection's file descriptor is no more related to the
IcapStateData structure and only we have to free this structure when we
have all data from the http server.
The patch is attached if you want to test it.
I must verify it and correct it before apply it to the cvs.
If it causes problems I will proceed with your patch.
About the second patch if there is not any problem I want to test it
more and if it is possible to move most of its code to the icap_*.c
files. This will save as time during merging with main branch..
Regards,
Christos
Graeme Bisset wrote:
Hi,
Could someone add the 2 attached patches to the squid icap client?
Here's a description of what they fix...
15minutefix.patch
=
I noticed that if an icap server replied with a 204 response, the
download would continue without being forwarded to icap but it would
then abort after 15 minutes. This was because the read timeout on the
icap connection wasn't being cancelled. This fix cancels the timeout
which avoids the aborted downloads.
squidcrashfix.patch
===
A fix that I submitted some time ago (squidgrowthfix.patch) caused squid
to crash under heavy load. This was because the deferred read check I
added relied on the store entry object being present but didn't lock it.
Under heavy load the store entry was sometimes freed before the defer
read check which caused squid to crash. This patch maintains a lock on
the store entry object and also has some other code to make sure the
store entry object is unlocked and freed when it is no longer required.
Thanks,
Graeme
diff -u -r1.1.2.65 icap_respmod.c
--- icap_respmod.c 25 May 2006 16:04:55 - 1.1.2.65
+++ icap_respmod.c 15 Jun 2006 18:01:52 -
@@ -500,6 +500,8 @@
#if SUPPORT_ICAP_204 || ICAP_PREVIEW
if (204 == status) {
debug(81, 3) (got 204 status from ICAP server\n);
+ icapRespModKeepAliveOrClose(icap,0);
+
debug(81, 3) (setting icap-flags.no_content\n);
icap-flags.no_content = 1;
/*
@@ -511,7 +513,8 @@
icap-respmod.resp_copy.buf, icap-respmod.resp_copy.size);
icap-respmod.resp_copy.size = 0;
if (icapReadReply2(icap) 0)
- comm_close(fd);
+ icapStateFree(-1, icap);
+// comm_close(fd);
/*
* XXX ideally want to clean icap-respmod.resp_copy here
* XXX ideally want to close ICAP server connection here
@@ -801,26 +804,29 @@
* transaction.
*/
static void
-icapRespModKeepAliveOrClose(IcapStateData * icap)
+icapRespModKeepAliveOrClose(IcapStateData * icap,int free_icap_state)
{
int fd = icap-icap_fd;
if (fd 0)
return;
-if (!icap-flags.keep_alive) {
- debug(81, 3) (%s:%d keep_alive not set, closing\n, __FILE__,
- __LINE__);
- comm_close(fd);
- return;
-}
debug(81, 3) (%s:%d FD %d looks good, keeping alive\n, __FILE__, __LINE__,
fd);
commSetDefer(fd, NULL, NULL);
commSetTimeout(fd, -1, NULL, NULL);
commSetSelect(fd, COMM_SELECT_READ, NULL, NULL, 0);
comm_remove_close_handler(fd, icapStateFree, icap);
-pconnPush(fd, icap-current_service-hostname, icap-current_service-port);
icap-icap_fd = -1;
-icapStateFree(-1, icap);
+if(free_icap_state)
+ icapStateFree(-1, icap);
+if (!icap-flags.keep_alive) {
+ debug(81, 3) (%s:%d keep_alive not set, closing\n, __FILE__,
+ __LINE__);
+ comm_close(fd);
+ return;
+}
+else{
+ pconnPush(fd, icap-current_service-hostname, icap-current_service-port);
+}
}
@@ -1042,7 +1048,10 @@
comm_close(fd);
} else if (icapPconnTransferDone(fd, icap)) {
storeComplete(entry);
- icapRespModKeepAliveOrClose(icap);
+ if(icap-flags.no_content)
+ icapStateFree(-1, icap);
+ else
+ icapRespModKeepAliveOrClose(icap,1);
} else if (!icap-flags.no_content) {
/* Wait for EOF condition */
commSetSelect(fd, COMM_SELECT_READ, icapReadReply, icap, 0);
context data for squid ICAP patched
Hi all squiders, I found the right way to implement the content filter extention to squid. I will continue the implementation and will send it to our QA Team, I hope the whole squid community will use this extention to squids ICAP capabilities. Best regards, Moshe Beeri. Software Engineer, Server Team. [EMAIL PROTECTED] Petach-Tiqva Bazel 16, Israel. Tel: +972 (3) 928-0400 ext. 429 Fax: +972 (3) 921-7594 Hi Christos, Thank you for your help, but you suggestion is not secure nor best perform, Please read my other remarks below. Now that I read the question again I see it is not clear enough, I will ask again. I would like squid ICAP client to do the logic for couple of reasons, 1. Security - Origin sever might change the replied http header and add the X-MY-SCANNER: Allow it self, and bypass the content filter, In that case I would not be able to prevent kids from viewing un honest pages :-( 2. Performance - Redundant call since I already know that request is allowed. There for I would like to keep in squids session data the classification and upon to the classification prior to response-mod call. For now I have figured out that the best place to set the data between the req-mod and resp-mod if in the fde structure, but since squid saves that object in fd_table (hash?) keyed by ICAP FD there is no continuity with the HTTP FD. I realized that I need to look for the mechanism that changes the next handler (hdl) that switch FD to read from, is the KEY to set up the fde related to the HTTP response, with the classification information. In squid ICAP client implementation there is no connection between the FD sets, ICAP's and HTTP's. Again 10X for reading and good will, I hope there is a short cut out there, If someone has an implementation suggestion or realizes I am missing something please write me. Hi Beeri, Maybe you do not need to modify the squid-ICAP code to support your model. I think that the correct implementation of your problem using squid-ICAP is: 1) An http request come into the squid. Squid sends the reqmod request to the ICAP server and server classifies the request: a) In the case of the BLOCK ICAP server creates a http response saying to the web client that the request blocked b) In the case of UNKNOWN ICAP server does nothing c) In the case of ALLOW ICAP server adds a proprietary http header to the http request for example X-MY-SCANNER: Allow 2) When squid has the http response then sends a respmod request to the ICAP server. The respmod request contains the http response headers AND the http request headers. a) When ICAP server founds the X-MY-SCANNER: Allow header in http request headers it responds with an allow204 response to squid b)The X-MY-SCANNER: Allow is not in the http request headers so the ICAP server takes the http body from squid and check it or modify it or what else. An other solutions is to use only the respmod request because here you have both the http request headers and the http response. The Question: I would like to pass the information that, no call to response mode (call the ICAP Server for the response) is needed. ... I am not sure that I fully understand your question, but I think that this functionality can not included in a general ICAP client of squid. But maybe I am loosing something here. Regards, Christos Background information: I am implementing an extension to squid ICAP client based upon ICAP Patch and squid 2.5 STABLE 10. The squid ICAP client does not support Content Filtering the way we at PureSight.com using it. The ICAP protocol is defined to support also Content Filtering and defines a return value at the request mod stage. I receive the value that can be one of the following: ALLOW, BLOCK, UNKNOWN ..
context data for squid ICAP patched
Hi all squids, Background information: I am implementing an extension to squid ICAP client based upon ICAP Patch and squid 2.5 STABLE 10. The squid ICAP client does not support Content Filtering the way we at PureSight.com using it. The ICAP protocol is defined to support also Content Filtering and defines a return value at the request mod stage. I receive the value that can be one of the following: ALLOW, BLOCK, UNKNOWN So far as described by the ICAP protocol (RFC 3507) and implemented by our ICAP server. As for squid client I added the ability to identify classification value, I would like squid to behave according to that value: 1. In case of UNKNOWN the response should go back to the ICAP server to be reclassified. 2. In case of BLOCK the ICAP server is changing the URL to retrieve a blocking information page. 3. In case of ALLOW squid should reply directly to the client with or without using the cache. The Question: I would like to pass the information that, no call to response mode (call the ICAP Server for the response) is needed. I would like your help with the way squid saves session data, I look at the code and I could not find that particular object, is there one? Any suggestions || directions would be blessed. The Polite part: Thanks for reading Participating on that grate project of squid. Your help Moshe Beeri. Software Engineer, Servers Team. [EMAIL PROTECTED] Petch-Tiqva Bazel 16, Israel. Tel: +972 (3) 928-0400 ext. 429 Fax: +972 (3) 921-7594
Re: context data for squid ICAP patched
Hi Beeri, Maybe you do not need to modify the squid-icap code to support your model. I think that the correct implementation of your problem using squid-icap is: 1) An http request come into the squid. Squid sends the reqmod request to the icap server and server clasifies the request: a) In the case of the BLOCK icap server creates a http responce saying to the web client that the request blocked b) In the case of UNKNOWN icap server does nothing c) In the case of ALLOW icap server adds a proprietary http header to the http request for example X-MY-SCANNER: Allow 2) When squid has the http responce then sends a respmod request to the icap server. The respmod request contains the http responce headers AND the http request headers. a) When icap server founds the X-MY-SCANNER: Allow header in http request headers it responds with an allow204 responce to squid b)The X-MY-SCANNER: Allow is not in the http request headers so the icap server takes the http body from squid and check it or modify it or what else. An other solutions is to use only the respmod request becouse here you have both the http request headers and the http responce. The Question: I would like to pass the information that, no call to response mode (call the ICAP Server for the response) is needed. ... I am not sure that I fully understand your question, but I think that this functionality can not included in a general ICAP client of squid. But maybe I am loosing something here. Regards, Christos Background information: I am implementing an extension to squid ICAP client based upon ICAP Patch and squid 2.5 STABLE 10. The squid ICAP client does not support Content Filtering the way we at PureSight.com using it. The ICAP protocol is defined to support also Content Filtering and defines a return value at the request mod stage. I receive the value that can be one of the following: ALLOW, BLOCK, UNKNOWN ..
RE: context data for squid ICAP patched
Hi Christos, Thank you for your help, but you suggestion is not secure nor best perform, Please read my other remarks below. Now that I read the question again I see it is not clear enough, I will ask again. I would like squid ICAP client to do the logic for couple of reasons, 1. Security - Origin sever might change the replied http header and add the X-MY-SCANNER: Allow it self, and bypass the content filter, In that case I would not be able to prevent kids from viewing un honest pages :-( 2. Performance - Redundant call since I already know that request is allowed. There for I would like to keep in squids session data the classification and upon to the classification prior to response-mod call. For now I have figured out that the best place to set the data between the req-mod and resp-mod if in the fde structure, but since squid saves that object in fd_table (hash?) keyed by ICAP FD there is no continuity with the HTTP FD. I realized that I need to look for the mechanism that changes the next handler (hdl) that switch FD to read from, is the KEY to set up the fde related to the HTTP response, with the classification information. In squid ICAP client implementation there is no connection between the FD sets, ICAP's and HTTP's. Again 10X for reading and good will, I hope there is a short cut out there, If someone has an implementation suggestion or realizes I am missing something please write me. Hi Beeri, Maybe you do not need to modify the squid-ICAP code to support your model. I think that the correct implementation of your problem using squid-ICAP is: 1) An http request come into the squid. Squid sends the reqmod request to the ICAP server and server classifies the request: a) In the case of the BLOCK ICAP server creates a http response saying to the web client that the request blocked b) In the case of UNKNOWN ICAP server does nothing c) In the case of ALLOW ICAP server adds a proprietary http header to the http request for example X-MY-SCANNER: Allow 2) When squid has the http response then sends a respmod request to the ICAP server. The respmod request contains the http response headers AND the http request headers. a) When ICAP server founds the X-MY-SCANNER: Allow header in http request headers it responds with an allow204 response to squid b)The X-MY-SCANNER: Allow is not in the http request headers so the ICAP server takes the http body from squid and check it or modify it or what else. An other solutions is to use only the respmod request because here you have both the http request headers and the http response. The Question: I would like to pass the information that, no call to response mode (call the ICAP Server for the response) is needed. ... I am not sure that I fully understand your question, but I think that this functionality can not included in a general ICAP client of squid. But maybe I am loosing something here. Regards, Christos Background information: I am implementing an extension to squid ICAP client based upon ICAP Patch and squid 2.5 STABLE 10. The squid ICAP client does not support Content Filtering the way we at PureSight.com using it. The ICAP protocol is defined to support also Content Filtering and defines a return value at the request mod stage. I receive the value that can be one of the following: ALLOW, BLOCK, UNKNOWN ..
RE: context data for squid ICAP patched
Hi Beeri, 1. Security - Origin sever might change the replied http header and add the X-MY-SCANNER: Allow it self, and bypass the content filter, In that case I would not be able to prevent kids from viewing un honest pages :-( Yes this problem exists. I am just refering it as way to pass info from reqmod to respmod icap services. 2. Performance - Redundant call since I already know that request is allowed. Yes the performance is a problem here. Assuming that you have 95% accepted requests and only 5% blocked maybe is not so problem to make your checks only in respmod request. But you know beter than me your requirements. There for I would like to keep in squids session data the classification and upon to the classification prior to response-mod call. For now I have figured out that the best place to set the data between the req-mod and resp-mod if in the ... I am not sure but I think that the best place to put your classification data is the request_t structure. Regards, Christos
Re: Squid-ICAP
On Thu, 2005-09-15 at 17:10 +0200, Ghislain Garçon wrote: does the ICAP team plan to implement load balancing for v3? If yes, will it only be round-robin or a different protocol?// The plan is to support basic ICAP first, without load balancing complications. The ICAP framework being developed should simplify adding load balancing features later. Do you expect to contribute ICAP load-balancing algorithms? Thank you, Alex. Duane Wessels a écrit : On Tue, 13 Sep 2005, Ghislain Garcon wrote: Hello, I'm interrested in the ICAP patch for SQUID-Cache and I would like to know if the developpement for squid 3.x will start. What kind of help will be the more needed? Hi Ghislain, I am working on ICAP for Squid-3. The code is in the sourceforge CVS repository with tag 'squid3-icap'. I don't think it is very useable yet, but in a while perhaps you can run some tests with Squid talking to an ICAP server. Duane W.
Volunteering for squid-icap development
Hello, I'm a software engineer working in England. I have a use for squid-icap and I would like to contribute to its development. In particular, I want to extend it so that it can do all 4 vector points: reqmod pre/postcache and respmod pre/postcache. I wrote to Henrik Nordstrm (the only developer for this who has a mailto: link) and he replied that I should request developer access direct from you. Would it be possible for me to get access to the squid-icap development tree? TIA, Jeff Silver
Re: Squid/icap 2.5 CVS
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Henrik Nordstrom wrote: Just tested compiling the icap-2.5 branch and the above problem did not show up. A number of other problems did however... - strnstr prototype in util.h. needed for compiling strnstr.c - a number module-local functions declared without static - strcasestr is a GNU extension. Prototype only available if _GNU_SOURCE is defined (glibc systems). All required changes committed. Hello, Just updated, works fine, thanks Henrik :) - -- Olivier Girondel [EMAIL PROTECTED] Project Leader Dolphian -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCibpcpqVXaJzJYNIRAuLNAJwN6/hnE7kqMQCxOSRTNTYmuhCT7gCeLvxG H7OSwRsnarW0LN4qZDK8png= =DmMz -END PGP SIGNATURE-
Re: Squid/icap 2.5 CVS
On Tue, 17 May 2005, Tsantilas Christos wrote: I hope that this means that you are starting developement in icap-client :-) Not in this icap client. But I have another (see below). I think icap is a must for squid. Agreed. Thinks like modifing headers (like those you are talking about with James ) are easier using icap. It is better for someone to put efford on icap developement (one patch) than on a number of different patches I already have a what I consider good ICAP client for Squid (the icap_streaming project). Just missing a customer willing to pay for it to have it published. icap_streaming was initially developed for a bigger potential customer project, but this project unfortunately got cancelled leaving icap_streaming somewhat hanging mid-air but still completed. If there is interest in this please contact [EMAIL PROTECTED] Regards Henrik
Re: Squid/icap 2.5 CVS
Henrik Nordstrom wrote: Not in this icap client. But I have another (see below). I wrote in a previous mail that I am considering the current ICAP client for squid problematic. And I now that it will never be really good (if it will not rewritten from scratch.). I already have a what I consider good ICAP client for Squid (the icap_streaming project). Just missing a customer willing to pay for it to have it published. icap_streaming was initially developed for a bigger potential customer project, but this project unfortunately got cancelled leaving icap_streaming somewhat hanging mid-air but still completed. If there is interest in this please contact [EMAIL PROTECTED] OK, it is good for all squid users to know that already exist a really good ICAP client for squid. :-) Regards, Christos
Re: Squid/icap 2.5 CVS
Henrik Nordstrom wrote: .. All required changes committed. I hope that this means that you are starting developement in icap-client :-) I think icap is a must for squid. Thinks like modifing headers (like those you are talking about with James ) are easier using icap. It is better for someone to put efford on icap developement (one patch) than on a number of different patches At the lastest tests which I done with squid-icap looks that it becomes more stable, but still I do not know about its stability in a production system. Regards, Christos
Re: squid-icap crash...
I will provide a better bug report. I believe I found the problem with generation of coredumps. On Apr 6, 2005 6:07 PM, Tsantilas Christos [EMAIL PROTECTED] wrote: This mail posted to c-icap mailing list. I believe that it is related with ICAP's request modification operation. It does not contain enough info but sometimes it is useful to just know the problems. - Christos Mateus Gröess wrote: Hi, Christos Today I was looking in cache.log of Squid ICAP and found another message that was followed by a Squid restart. ... Unfortunally there wasn't messages between the error and the normal Squid startup. 2005/04/04 14:07:00| storeLateRelease: released 0 objects 2005/04/04 17:12:31| assertion failed: client_side.c:3268: cbdataValid(conn) 2005/04/04 17:12:42| Starting Squid Cache version 2.5.STABLE9-CVS for i386-slackware-linux-gnu...
Re: squid-icap crash...
Mateus send me the backtrace On Apr 5, 2005 10:05 AM, Mateus Gröess wrote: 2005/04/04 14:07:00| storeLateRelease: released 0 objects 2005/04/04 17:12:31| assertion failed: client_side.c:3268: cbdataValid(conn) 2005/04/04 17:12:42| Starting Squid Cache version 2.5.STABLE9-CVS for i386-slackware-linux-gnu... Here is the stack trace that I think is the same problem of the assertion above: 2005/04/07 11:22:36| storeLateRelease: released 0 objects 2005/04/07 11:23:38| assertion failed: client_side.c:3268: cbdataValid(conn) Program received signal SIGABRT, Aborted. 0x402299f1 in __kill () from /lib/libc.so.6 (gdb) backtrace #0 0x402299f1 in __kill () from /lib/libc.so.6 #1 0x402296d4 in raise (sig=6) at ../sysdeps/posix/raise.c:27 #2 0x4022ae31 in abort () at ../sysdeps/generic/abort.c:88 #3 0x80736c7 in xassert (msg=0x80e15ed cbdataValid(conn), file=0x80defdc client_side.c, line=3268) at debug.c:270 #4 0x806cfbe in clientReadBody (request=0x8502f38, buf=0x84c56d8 , size=8192, callback=0x809cad4 icapReqModBodyHandler, cbdata=0x85390d0) at client_side.c:3268 #5 0x809cacc in icapReqModSendBodyChunk (fd=26, bufnotused=0x0, size=627, errflag=0, data=0x85390d0) at icap_reqmod.c:758 #6 0x806ee9a in CommWriteStateCallbackAndFree (fd=26, code=0) at comm.c:99 #7 0x8071380 in commHandleWrite (fd=26, data=0x84f5830) at comm.c:929 #8 0x8072656 in comm_poll (msec=268) at comm_select.c:459 #9 0x80a7aae in main (argc=2, argv=0xbb34) at main.c:748 #10 0x4021a2eb in __libc_start_main (main=0x80a7664 main, argc=2, ubp_av=0xbb34, init=0x804a6a4 _init, fini=0x80d7a1c _fini, rtld_fini=0x4000c130 _dl_fini, stack_end=0xbb2c) at ../sysdeps/generic/libc-start.c:129 (gdb) quit The program is running. Exit anyway? (y or n) y
squid-icap crash...
This mail posted to c-icap mailing list. I believe that it is related with ICAP's request modification operation. It does not contain enough info but sometimes it is useful to just know the problems. - Christos Mateus Gröess wrote: Hi, Christos Today I was looking in cache.log of Squid ICAP and found another message that was followed by a Squid restart. ... Unfortunally there wasn't messages between the error and the normal Squid startup. 2005/04/04 14:07:00| storeLateRelease: released 0 objects 2005/04/04 17:12:31| assertion failed: client_side.c:3268: cbdataValid(conn) 2005/04/04 17:12:42| Starting Squid Cache version 2.5.STABLE9-CVS for i386-slackware-linux-gnu...
Re: squid ICAP developer
On Thu, 3 Mar 2005, Tsantilas Christos wrote: I am Tsantilas Christos (chtsanti at users dot sourceforge dot net), I am working with ICAP protocol and I am interested to help the development of ICAP client for squid. You have now been given access to the devel.squid-cache.org services. While you wait for the permissions to get updated (can take up to 12 hours) please read the documents on how to use the services and rules http://devel.squid-cache.org/CVS.html (tools, routines, howto tips etc) http://devel.squid-cache.org/rules.html (usage guidelines) And please subscribe to the squid-dev mailinglist by sending a message to [EMAIL PROTECTED] Regards Henrik
Squid ICAP client possibly bugs and corrections......
Hi squid developers,
I want to report 1-2 small bugs and corrections at squid
icap client interface that are relative with keeped alive
requests to an icap server.
I am using a (corrected) squid-icap named squid-2.5.STABLE5-icap-6-pre1.
but as I seen this bugs exists and in squid-icap-2.5-200405131634
icap enabled squid.
1)when icapReadReply3 called after a response request with null-body
(e.g when the content of a web page not changed ...)
then enters the block if (EBIT_TEST(i..., ENTRY_ABORTED)){.
Squid icap-client has request keeped alive connection and
now the connection not closed. This behaviour can cause
a lot of open connection to an icap server until connections expired.
An addition of comm_close(fd) in this block can solve the problem.
Moreover if think there is no need for the connection to be closed
in this phase and can keeped alive. So the entire block :
if (EBIT_TEST(i..., ENTRY_ABORTED)){
...
}
in function icapReadReply3 can removed.
2) In file icap_reqmod.c in function icapSendReqMod
when compute the position of entities for
Encapsulated: header squid does not include crlf (2 bytes) in
possition computation.
This fields are the possition in the header not the size of the header.
I propose to
a) add a crlf at the end of icap-request-header
before Encapsulated header computation
eg memBufAppend(mb_hdr, crlf, 2); at line 613
and
b) remove the memBufAppend(mb, crlf, 2); line 637 that adds the
crlf at the end of
entire buffer.
Regards,
Christos
Re: Fwd: Squid ICAP client problems
Henrik Nordstrom wrote: I had fogot about this issue. I'll note this issue so it is not forgotten again. Needs to be fixed to not use MSG_PEEK so it can wait for more data to arrive via commSetSelect(), which involves some magic to get the interactions with the main Squid code correct. Yes, use of commSetSelect is ideal. Regards Henrik
Christian Kunst - Squid iCAP Client
Hi, My name is Christian Kunst, and I would like to join the Squid team. I am interested in helping develop the Squid iCAP client. Please let me know what are the next steps that I should take. Thank you and best regards, Christian Kunst.
