[squid-users] Exception with squid 3.5.25 in ICAP options path

[Arunabha Saha]

Seeing a exception followed by termination fairly regularly with
Squid-->Icap with this stacktrace on 3.5.25.

The problem typically happens when icap has not been started yet and squid
is processing some requests and is configured for ICAP adaptation service.

It seems like the ICAP Options request goes unanswered (as it would since
ICAP is not UP) and we try to shut down the ICAP service but there's some
dangling state with respect to the options  requests that were sent.

Appreciate it if someone can comment on whether this is known and fixed
before i dig a little deeper.



#0  0x7ff944120428 in __GI_raise (sig=sig@entry=6) at
../sysdeps/unix/sysv/linux/raise.c:54
#1  0x7ff94412202a in __GI_abort () at abort.c:89
#2  0x7ff944a670d5 in __gnu_cxx::__verbose_terminate_handler() () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x7ff944a64cc6 in ?? () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x7ff944a63bc9 in ?? () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x7ff944a645b8 in __gxx_personality_v0 () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x7ff9444c4f03 in ?? () from /lib/x86_64-linux-gnu/libgcc_s.so.1
#7  0x7ff9444c5410 in _Unwind_RaiseException () from
/lib/x86_64-linux-gnu/libgcc_s.so.1
#8  0x7ff944a64f47 in __cxa_throw () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x006f3486 in Throw (message=message@entry=0x8ae66e
"!theOptionsFetcher", fileName=fileName@entry=0x8ae647 "ServiceRep.cc",
lineNo=lineNo@entry=50,
id=149798962) at TextException.cc:93
#10 0x008071f9 in Adaptation::Icap::ServiceRep::~ServiceRep
(this=this@entry=0x1298b58, __in_chrg=,
__vtt_parm=)
at ServiceRep.cc:50
#11 0x008073c9 in Adaptation::Icap::ServiceRep::~ServiceRep
(this=0x1298b58, __in_chrg=, __vtt_parm=) at
ServiceRep.cc:52
#12 0x0081d6c3 in RefCount::dereference
(newP=0x0, this=0x133ef88) at ../../../src/base/RefCount.h:79
#13 RefCount::~RefCount (this=0x133ef88,
__in_chrg=) at ../../../src/base/RefCount.h:35
#14 Adaptation::Icap::Launcher::~Launcher (this=0x133ef68,
__vtt_parm=0xb5e208 ,
__in_chrg=)
at Launcher.cc:32
#15 0x00830137 in
Adaptation::Icap::OptXactLauncher::~OptXactLauncher (this=0x133ef68,
__in_chrg=, __vtt_parm=)
at ../../../src/adaptation/icap/OptXact.h:55
#16 Adaptation::Icap::OptXactLauncher::~OptXactLauncher (this=0x133ef68,
__in_chrg=, __vtt_parm=)
at ../../../src/adaptation/icap/OptXact.h:55
#17 0x006f0666 in AsyncJob::callEnd (this=0x133efa8) at
AsyncJob.cc:144
#18 0x007f4732 in JobDialer::dial
(this=0x153f2f0, call=...) at ../../src/base/AsyncJobCalls.h:181
#19 0x006ee331 in AsyncCall::make (this=0x153f2c0) at
AsyncCall.cc:40
#20 0x006f26e5 in AsyncCallQueue::fireNext (this=this@entry=0xdc2c60)
at AsyncCallQueue.cc:56
#21 0x006f2b09 in AsyncCallQueue::fire (this=0xdc2c60) at
AsyncCallQueue.cc:42
#22 0x0057def9 in EventLoop::dispatchCalls (this=0x7fffca605010) at
EventLoop.cc:143
#23 EventLoop::runOnce (this=this@entry=0x7fffca605010) at EventLoop.cc:120
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid 3.5.25 does not recognise ICAP 408 status code

[Arunabha Saha]

>As with any timeout, it is impossible to say in general which side of
>the connection is at fault. This case has at least three sides: It could
>be the HTTP agent, Squid, and/or the ICAP service. Did one of them stall
>the transaction? Or was the ICAP service just too impatient? See option
>#4 below.
I've tried to track this down.   There are some  persistent sockets used by
SaaS apps for APIs (otservice api from google sites) and sometimes the HTTP
response takes a long time to trickle in.  I have seen upto 25 seconds for
the response body to trickle in after the response header.   I don't know
yet if this is due to network delays but given that it happens only for
this particular uri I'm theorizing that this is how it works.I can
whitelist the one i am aware of that is causing this issue but again the
concern is what about others that might throw this exception.The
timeout at 10 seconds is somewhat aggressive so moving that up should help
but some code changes in either squid or icap as suggested look necessary.

>Please note that ICAP is a protocol, not a product/software name. It
>probably does not matter what ICAP service you are using though.

Correct.  I was referring to the c-icap implementation.

On Wed, Oct 31, 2018 at 5:00 AM 
wrote:

> Send squid-users mailing list submissions to
> squid-users@lists.squid-cache.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.squid-cache.org/listinfo/squid-users
> or, via email, send a message with subject or body 'help' to
> squid-users-requ...@lists.squid-cache.org
>
> You can reach the person managing the list at
> squid-users-ow...@lists.squid-cache.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of squid-users digest..."
>
>
> Today's Topics:
>
>1.
> On 10/30/18 6:45 PM, Arunabha Saha wrote:
>
> > Squid 3.5.25 does not seem to recognise the 408 request timeout error
> > code from ICAP.
>
> Squid effectively recognizes ICAP 408 response as an ICAP transaction
> error response and blames the ICAP service for that error. That
> (minimal) support can be improved, of course. See options #1 and #3 below.
>
>
>

>
> Needless to say, treating all ICAP service timeouts as if nothing bad
> happened would break some existing Squid deployments (while possibly
> fixing yours). A proper general solution (option #3 below) would most
> likely require making Squid behavior configurable.
>
>
> > The more troublesome issue for me is the exception it generates and then
> > declares ICAP down after a certain number of such exceptions.
> >
> > I don't want to disable the failure limit entirely given that we can
> > often have genuine failures that squid needs to detect.
> >
> > What i'd like to see is squid should not throw an exception in this
> > case.
>
> The "exception" is a minor low-level/technical detail. What you really
> want to see is Squid blaming itself (rather than the ICAP service) for
> the problem. Squid indeed lacks that kind of functionality, but it can
> be added if really needed. See options #1 and #3 below.
>
>
> > The timeout is somewhat aggressive but works with an earlier
> > version of ICAP (0.1.x).  The one i'm testing is 0.5.3.
>
> Please note that ICAP is a protocol, not a product/software name. It
> probably does not matter what ICAP service you are using though.
>
>
> > Any suggestions?
>
> I can suggest a few options, in no particular order:
>
> 1. Modify your Squid to treat 408 differently.
> 2. Modify your ICAP service to stop sending ICAP 408 responses to Squid.
> 3. Add proper ICAP timeout support feature to Squid.
> 4. Investigate why your ICAP service times out. If you are lucky, you
>may be able to fix or work around the problem by adjusting Squid
>and/or your ICAP service configuration.
>
> For option #1, Adaptation::Icap::ModXact::parseIcapHead() may be a good
> starting point.
>
> For options #1 and #3, see also
>
> https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F
>
> In most cases, option #4 is the best first step but YMMV.
>
>
> HTH,
>
> Alex.
>
>
> --
>
> Message: 2
> Date: Tue, 30 Oct 2018 23:59:18 -0500 (CDT)
> From: Sid 
> To: squid-users@lists.squid-cache.org
> Subject: Re: [squid-users] Squid 4.3: SSL Bump fails to send client
> certificate
> Message-ID: <1540961958277-0.p...@n4.nabble.com>
> Content-Type: text/plain; charset=us-ascii
>
> Thank you Alex for the reply.
>
> Alex: 1. Servers never send SNI. Clients usually sen

[squid-users] Squid 3.5.25 does not recognise ICAP 408 status code

[Arunabha Saha]

Squid 3.5.25 does not seem to recognise the 408 request timeout error code
from ICAP.
The more troublesome issue for me is the exception it generates and then
declares ICAP down after a certain number of such exceptions.

I don't want to disable the failure limit entirely given that we can often
have genuine failures that squid needs to detect.

What i'd like to see is squid should not throw an exception in this case.
 The timeout is somewhat aggressive but works with an earlier version of
ICAP (0.1.x).  The one i'm testing is 0.5.3.

Any suggestions?  I tried looking at squid 3.5.26 through 3.5.28 but i
don't see any support for 408 timeout there (maybe i missed it?).

Debug Logs:

2018/10/29 20:20:25.656| 93,5| ModXact.cc(654) parseMore:
ICAP/1.0 408 Request timeout^M
Server: C-ICAP/0.5.3^M
Connection: close^M
ISTag: CI0001-X^M
^M

2018/10/29 20:20:25.656| 93,5| ModXact.cc(749) parseHeaders: parse ICAP
headers
2018/10/29 20:20:25.656| 93,5| ModXact.cc(1079) parseHead: have 98 head
bytes to parse; state: 0
2018/10/29 20:20:25.656| 93,5| ModXact.cc(1094) parseHead: parse success,
consume 98 bytes, return true
2018/10/29 20:20:25.656| 93,5| ModXact.cc(785) parseIcapHead: found
connection close
2018/10/29 20:20:25.656| 93,5| ModXact.cc(815) parseIcapHead: ICAP status
408
2018/10/29 20:20:25.656| 93,3| ../../../src/base/AsyncJobCalls.h(177) dial:
Adaptation::Icap::Xaction::noteCommRead threw exception: Unsupported ICAP
status code
2018/10/29 20:20:25.656| 11,5| HttpRequest.cc(472) detailError: current
error details: 35/396407234
2018/10/29 20:20:25.656| 93,4| Xaction.cc(514) setOutcome: ICAP_ERR_OTHER
2018/10/29 20:20:25.656| 93,4| ServiceRep.cc(80) noteFailure:  failure 4
out of 10 allowed in 0sec [up,fail4]
...
2018/10/29 20:21:06.981| 93,5| ModXact.cc(815) parseIcapHead: ICAP status
408
2018/10/29 20:21:06.981| 93,3| ../../../src/base/AsyncJobCalls.h(177) dial:
Adaptation::Icap::Xaction::noteCommRead threw exception: Unsupported ICAP
status code
2018/10/29 20:21:06.981| 11,5| HttpRequest.cc(472) detailError: current
error details: 35/396407234
2018/10/29 20:21:06.981| 93,4| Xaction.cc(514) setOutcome: ICAP_ERR_OTHER
2018/10/29 20:21:06.981| 93,4| ServiceRep.cc(80) noteFailure:  failure 11
out of 10 allowed in 0sec [up,fail11]
2018/10/29 20:21:06.981| suspending ICAP service for too many failures



-- 
regards,
Arun
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] squid-users Digest, Vol 59, Issue 10

[Arunabha Saha]

>The client will attempt to open a TLS/TCP connection to the origin
>server. Your router (or some such) will redirect client TLS/TCP bytes to
>your Squid's https_port. If configured correctly, Squid will accept that
>TCP connection and wrap/forward it into/inside an HTTP CONNECT tunnel
>through the corporate proxy.

   I'm trying to accomplish something similiar but i don't see squid
wrap the connection to parent proxy in a HTTP CONNECT tunnel.
   User ->Squid(Transparent Proxy)->Parent Proxy-->Internet.
   I need to see a CONNECT tunnel between Squid(Transparent Proxy)
and Parent Proxy but I don't.   Based on another thread, Is this
something that works only starting squid 4.X.   My version is squid
3.5.25.


On Wed, Jul 10, 2019 at 5:02 AM
 wrote:
>
> Send squid-users mailing list submissions to
> squid-users@lists.squid-cache.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.squid-cache.org/listinfo/squid-users
> or, via email, send a message with subject or body 'help' to
> squid-users-requ...@lists.squid-cache.org
>
> You can reach the person managing the list at
> squid-users-ow...@lists.squid-cache.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of squid-users digest..."
>
>
> Today's Topics:
>
>1. Non-standard proxy setup (Tardif, Christian)
>2. Re: Non-standard proxy setup (Alex Rousskov)
>
>
> --
>
> Message: 1
> Date: Tue, 9 Jul 2019 13:10:21 +
> From: "Tardif, Christian" 
> To: "squid-users@lists.squid-cache.org"
> 
> Subject: [squid-users] Non-standard proxy setup
> Message-ID:
> 
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> I'm trying to figure out how to make the following setup work:
>
> I have a node on which there's an application which isn't proxy aware so 
> basically, the only remaining option would be to use a transparent proxy. But 
> my corporate proxy isn't a transparent proxy. So I have to build this in two 
> layers. My solution would be to:
>
>
> 1) Have a squid proxy on the node's router host configured as a 
> transparent proxy for both HTTP and HTTPS
>
> 2) Have this squid proxy configured to talk to the parent host, which 
> would be my corporate proxy
>
> 3) Have this squid proxy able to decide if a particular flow should go to 
> the corporate proxy or connect "directly" with the destination host
>
> I've been successful at tasks #2 and #3 (well, in fact, I did it with 
> tinyproxy but stopped because of task #1
>
> I've partly succedded at task #1. In fact, it worked for HTTP. I haven't 
> figured out how to do it for HTTPS. My questions are:
>
>
> 1) I do not understand how the client would be able to perform a CONNECT 
> to reach squid in HTTPS. So I'm assuming that there's some other magic.
>
> 2) The second thing I don't understand is the certificates management. 
> Let's say my node tries to reach https://www.google.com but does not know 
> anything about the proxy. I assume that the client will get the certificate 
> from squid in some way, but would probably expect to receive a certificate 
> from Google. How would that work?
>
> Can someone help me?   I'm running out of options...
>
> Thanks,
>
> Christian Tardif
> -- next part --
> An HTML attachment was scrubbed...
> URL: 
> 
>
> --
>
> Message: 2
> Date: Tue, 9 Jul 2019 09:54:25 -0400
> From: Alex Rousskov 
> To: squid-users@lists.squid-cache.org
> Subject: Re: [squid-users] Non-standard proxy setup
> Message-ID:
> 
> Content-Type: text/plain; charset=windows-1252
>
> On 7/9/19 9:10 AM, Tardif, Christian wrote:
>
> > I have a node on which there’s an application which isn’t proxy aware so
> > basically, the only remaining option would be to use a transparent
> > proxy. But my corporate proxy isn’t a transparent proxy. So I have to
> > build this in two layers. My solution would be to:
> >
> >
> >
> > 1) Have a squid proxy on the node’s router host configured as a
> > transparent proxy for both HTTP and HTTPS
> >
> > 2) Have this squid proxy configured to talk to the parent host,
> > which would be my corporate proxy
> >
> > 3) Have this squid proxy able to decide if a particular flow should
> > go to the corporate proxy or connect “directly” with the destination host
> >
> >
> >
> > I’ve been successful at tasks #2 and #3 (well, in fact, I did it with
> > tinyproxy but stopped because of task #1
> >
> >
> >
> > I’ve partly succedded at task #1. In fact, it worked for HTTP. I haven’t
> > figured out how to do it for HTTPS. My questions are:
> >
> >
> >
> > 1) I do not understand how the client would be able to perform a
> > CONNECT to reach squid in HTTPS. So I’m assuming that there’s 

Re: [squid-users] squid-users Digest, Vol 59, Issue 12

[Arunabha Saha]

Thanks.   i did get it working with the latest 5.0.0 (unreleased) code
in github.The configuration has to be  "ssl-bump client-first .."
for this to work.
Does that sound right?

On Fri, Jul 12, 2019 at 5:02 AM
 wrote:
>
> Send squid-users mailing list submissions to
> squid-users@lists.squid-cache.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.squid-cache.org/listinfo/squid-users
> or, via email, send a message with subject or body 'help' to
> squid-users-requ...@lists.squid-cache.org
>
> You can reach the person managing the list at
> squid-users-ow...@lists.squid-cache.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of squid-users digest..."
>
>
> Today's Topics:
>
>1. Re: Non-standard proxy setup (Alex Rousskov)
>
>
> --
>
> Message: 1
> Date: Thu, 11 Jul 2019 09:31:02 -0400
> From: Alex Rousskov 
> To: squid-users@lists.squid-cache.org
> Subject: Re: [squid-users] Non-standard proxy setup
> Message-ID:
> <42a9f4e2-8ca2-eb2b-88e9-751d4af75...@measurement-factory.com>
> Content-Type: text/plain; charset=utf-8
>
> On 7/10/19 7:44 PM, Arunabha Saha wrote:
> >> The client will attempt to open a TLS/TCP connection to the origin
> >> server. Your router (or some such) will redirect client TLS/TCP bytes to
> >> your Squid's https_port. If configured correctly, Squid will accept that
> >> TCP connection and wrap/forward it into/inside an HTTP CONNECT tunnel
> >> through the corporate proxy.
>
> > i don't see squid
> > wrap the connection to parent proxy in a HTTP CONNECT tunnel.
> >User ->Squid(Transparent Proxy)->Parent Proxy-->Internet.
> >I need to see a CONNECT tunnel between Squid(Transparent Proxy)
> > and Parent Proxy but I don't.   Based on another thread, Is this
> > something that works only starting squid 4.X.
>
> I do not remember for sure, but you may need a development version of
> Squid (future v5) or an unofficial patch to forward intercepted tunnels
> to a cache peer. If SslBump-related peering support is indeed required
> to support such forwarding, then please see this seemingly unrelated bug
> report for more details and options:
>
>   https://bugs.squid-cache.org/show_bug.cgi?id=4968
>
> Alex.
>
>
> --
>
> Subject: Digest Footer
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
>
> --
>
> End of squid-users Digest, Vol 59, Issue 12
> ***



-- 
regards,
Arun
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users