subject:"\[squid\-users\] squid tproxy"

Re: [squid-users] Squid TPROXY issues with Google sites

2017-05-31 Thread Vieri



From: Alex Rousskov 
>
> You need to figure out why. Two common reasons are SSL-level errors and
> http_access denials. Both should be reflected in access.log and
> debugging cache.log.


I finally found out it was an http_access deny on an ACL match with url_regex.

Thanks Alex.

Vieri
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid TPROXY issues with Google sites

2017-05-28 Thread Alex Rousskov

On 05/28/2017 05:40 AM, Vieri wrote:

> Please keep in mind that I'm basically an end-user, a sys-admin. I
> wish I had the time to study Squid's source code.

Nobody (certainly not me) has suggested anything that requires studying
Squid source code. If you think that I have, you have misinterpreted
what I have said.

> The cache log reports errors but they are not necessarily related to
> this client as there are many others actively browsing.

I recommend triaging this using a Squid instance isolated from all other
traffic. You are making both your job and the job of those who are
trying to help you more difficult by trying to save a few minutes/hours
that are usually required to set up an isolated test.

> Anyway, as a workaround I'm willing to splice/tunnel traffic to
> accounts.google.com *ONLY*, and bump everything else (although I'd
> prefer to understand why bumping isn't "working" for this site).

> I've tried this:

> acl GoogleAccounts ssl::server_name accounts.google.com
> acl step1 at_step SslBump1
> ssl_bump peek step1
> ssl_bump splice GoogleAccounts
> ssl_bump bump all

> However, traffic to accounts.google.com is not spliced, it's bumped
> like the rest.

You need to figure out why. Two common reasons are SSL-level errors and
http_access denials. Both should be reflected in access.log and
debugging cache.log.

> Can FQDNs be used in ACLs as in the example above even when peeking at step 1?

Yes. They may not work, but they can be used. They should work if the
request contains TLS SNI. Modern browser requests usually do, but you
can confirm by studying browser-Squid traffic with a tool like Wireshark.

> If I peek at step 2 then I won't be able to "bump all"

Correct.

> Likewise, If I need to stare at step 2 then I'll never be able to splice

Correct.

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid TPROXY issues with Google sites

2017-05-28 Thread Vieri

Hi Alex et al.,

Thank you very much for your analysis and help. I really appreciate it.

Please keep in mind that I'm basically an end-user, a sys-admin. I wish I had 
the time to study Squid's source code. All I can do for now is read the docs 
that so many people have kindly published.

In 99% of my use cases, I only need this:

ssl_bump stare all
ssl_bump bump all

However, some sites simply don't behave well when accessed with Squid TPROXY. 
This is an example I'm reporting regarding access to 
https://accounts.google.com.

The use case is simple. A client browser successfully connects to 
https://accounts.google.com and I can see this in the access log (there might 
be some garbage but I'm posting it all for completeness):

# tail -f /var/log/squid/access.log | grep 10.215.145.8
1495969366.990 90 10.215.145.8 TCP_MISS/302 870 GET 
https://accounts.google.com/ - ORIGINAL_DST/216.58.201.141 text/html
1495969367.089 91 10.215.145.8 TCP_MISS/302 1206 GET 
https://accounts.google.com/ManageAccount - ORIGINAL_DST/216.58.201.141 
text/html
1495969367.165165 10.215.145.8 TAG_NONE/200 0 CONNECT 216.58.201.141:443 - 
ORIGINAL_DST/216.58.201.141 -
1495969367.546452 10.215.145.8 TCP_MISS/200 254275 GET 
https://accounts.google.com/ServiceLogin? - ORIGINAL_DST/216.58.201.141 
text/html
1495969367.684 99 10.215.145.8 TCP_MISS/200 837 GET 
https://accounts.google.com/_/common/diagnostics/? - 
ORIGINAL_DST/216.58.201.141 application/json
1495969367.799218 10.215.145.8 TAG_NONE/200 0 CONNECT 216.58.201.141:443 - 
ORIGINAL_DST/216.58.201.141 -
1495969368.341356 10.215.145.8 TCP_MISS/200 9598 GET 
https://ssl.gstatic.com/accounts/static/_/js/k=gaia.gaiafe_glif.es.QCvs5i6XPsY.O/m=ZJkSm,ssIgD,GJkP8c,HUb4Ab,sy3j,DnoIKd,sy1a,sy1g,YKZpNb,sy19,VI9RTb,sy18,sy24,GEsPC/am=gggAAACgARcEwFGwAlAM/rt=j/rs=ABkqax2H2XpBhaGl4fmxx-IOq5MdI_K9yw
 - ORIGINAL_DST/172.217.9.227 text/javascript
1495969373.609249 10.215.145.8 TCP_MISS/200 9598 GET 
https://ssl.gstatic.com/accounts/static/_/js/k=gaia.gaiafe_glif.es.QCvs5i6XPsY.O/m=ZJkSm,ssIgD,GJkP8c,HUb4Ab,sy3j,DnoIKd,sy1a,sy1g,YKZpNb,sy19,VI9RTb,sy18,sy24,GEsPC/am=gggAAACgARcEwFGwAlAM/rt=j/rs=ABkqax2H2XpBhaGl4fmxx-IOq5MdI_K9yw
 - ORIGINAL_DST/172.217.9.227 text/javascript
1495969393.879248 10.215.145.8 TCP_MISS/200 9598 GET 
https://ssl.gstatic.com/accounts/static/_/js/k=gaia.gaiafe_glif.es.QCvs5i6XPsY.O/m=ZJkSm,ssIgD,GJkP8c,HUb4Ab,sy3j,DnoIKd,sy1a,sy1g,YKZpNb,sy19,VI9RTb,sy18,sy24,GEsPC/am=gggAAACgARcEwFGwAlAM/rt=j/rs=ABkqax2H2XpBhaGl4fmxx-IOq5MdI_K9yw
 - ORIGINAL_DST/172.217.9.227 text/javascript
1495969393.940166 10.215.145.8 TCP_MISS/200 452 GET 
http://detectportal.firefox.com/success.txt - ORIGINAL_DST/23.219.93.219 
text/plain
1495969394.116225 10.215.145.8 TCP_MISS/200 1261 GET 
https://ssl.gstatic.com/accounts/static/_/js/k=gaia.gaiafe_glif.es.QCvs5i6XPsY.O/m=ZJkSm/am=gggAAACgARcEwFGwAlAM/rt=j/rs=ABkqax2H2XpBhaGl4fmxx-IOq5MdI_K9yw
 - ORIGINAL_DST/172.217.9.227 text/javascript
1495969394.204873 10.215.145.8 TAG_NONE/200 0 CONNECT 54.148.190.222:443 - 
ORIGINAL_DST/54.148.190.222 -
1495969394.724488 10.215.145.8 TCP_MISS/200 195 POST 
https://incoming.telemetry.mozilla.org/submit/telemetry/3474d8df-c0c5-454b-916f-20ad7f8cb3f3/main/Firefox/52.0.2/release/20170323105023?
 - ORIGINAL_DST/54.148.190.222 text/plain
1495969399.355223 10.215.145.8 TCP_MISS/200 1261 GET 
https://ssl.gstatic.com/accounts/static/_/js/k=gaia.gaiafe_glif.es.QCvs5i6XPsY.O/m=ZJkSm/am=gggAAACgARcEwFGwAlAM/rt=j/rs=ABkqax2H2XpBhaGl4fmxx-IOq5MdI_K9yw
 - ORIGINAL_DST/172.217.9.227 text/javascript

The client browser successfully renders Google's log-in page where you enter a 
username. However, it is NOT possible to "click next" and enter a password.
No matter what the user does on that page, nothing is logged in 
/var/log/squid/access.log.

The cache log reports errors but they are not necessarily related to this 
client as there are many others actively browsing.

# grep -i error /var/log/squid/cache.log 
2017/05/28 12:55:48 kid1| Error negotiating SSL on FD 93: error:14090086:SSL 
routines:ssl3_get_server_certificate:certificate verify failed (1/-1/2)
2017/05/28 12:55:48 kid1| Error negotiating SSL connection on FD 90: 
error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate (1/0)
2017/05/28 12:55:49 kid1| Error negotiating SSL on FD 143: error:1409F07F:SSL 
routines:ssl3_write_pending:bad write retry (1/-1/0)
2017/05/28 12:55:50 kid1| Error negotiating SSL on FD 172: error:1409F07F:SSL 
routines:ssl3_write_pending:bad write retry (1/-1/0)
2017/05/28 12:55:55 kid1| Error negotiating SSL on FD 57: error:14090086:SSL 
routines:ssl3_get_server_certificate:certificate verify failed (1/-1/0)
2017/05/28 12:55:55 kid1| Error negotiating SSL connection on FD 27: 
error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher (1/-1)
2017/05/28 12:55:58 kid1| Error negotiating SSL on FD 57: error:1409F07F:SSL 
routines:ssl3_write_pending:bad write retry

Re: [squid-users] Squid TPROXY issues with Google sites

2017-05-26 Thread Alex Rousskov

On 05/26/2017 05:22 PM, Vieri wrote:

> If I have this:
> 
> ssl_bump peek all
> ssl_bump splice AllowTroublesome
> ssl_bump bump all

... then you have a configuration that does not make sense because one
cannot bump after peeking at step2. Your configuration is equivalent to

  * if the current step is 1 or 2, then peek
  * if AllowTroublesome during step 3, then splice
  * otherwise, do the impossible

which, bugs notwithstanding, is equivalent to

  ssl_bump peek all
  ssl_bump splice all

If the above does not block anything then your http_access rules allow
all CONNECTs (and you never get beyond CONNECTs because you do not bump).

> If I replace the above snippet with this:
> 
> ssl_bump stare all
> ssl_bump bump all

This configuration makes sense (but it may not do what you want).

If you want to be able to make a "splice or bump" decision, then you
have to make it during step2:

  ssl_bump peek step1
  ssl_bump splice AllowTroublesome
  ssl_bump bump all

> If I had an http_access rule that allowed the transaction to take
> place then I would expect it to happen regardless of the ssl_bump
> directive.

Your expectations are wrong. SslBump directives expose http_access rules
to more (or fewer) transactions. For example, the "splice all"
configuration does not expose http_access rules to any GET requests.

> Alex, you mention the SSLPeekAndSplice web page. I'll try to sum it
> up in just a few lines

The SslBump feature is too complex to sum it up in just a few lines
unless those lines are something like "do not use it without fully
understanding it". Once you learn the basics of SSL handshake, which
Squid steps look at what parts of the handshake, and what the essential
difference between peeking and staring is, then SslBump becomes less of
black magic. Without that knowledge, it is a dark mystery.

> - peek implies splice which means you can't do content analysis (as
> in scan for threats via c-icap modules)

Wrong. Peeking at step1 does not preclude future bumping. Peeking at
step2 precludes future bumping. If you peek at step2, then you have to
splice or terminate at step3.

> - stare implies bump which means you can do content analysis

Wrong for similar/symmetrical reasons.

> - you don't need to stare, you can just bump

Wrong in many cases -- usually you _do_ need to stare (or peek) at least
at step1, but YMMV.

> - you need to stare before bump if you want the clients to accept a
> certificate with domain names instead of IP addresses

Misleading. You need to stare or peek to get more information, including
the server domain name. That information comes from either the client or
the server, depending on the step. That information is used to generate
a fake certificate. The more info Squid has, the better it can
fake/mimic the true certificate, but learning more information restricts
the set of ssl_bump actions.

> - you can bump first by ACLs and then splice the rest

If you mean that ssl_bump rules may start with "bump" rules and end with
"splice" rules, then this is true, but the reverse is also true, and the
rules may contain a mixture of many actions.

> - you can bump after peek but only if you do that at SslBump1

Too vague to be generally useful. You can bump after peeking at step1.
You cannot bump after peeking at step2.

> the "Bump All Sites Except Banks" example where the next
> phrase contradicts the title by saying that the requests to non-banks
> won't be bumped.

Correct! The example, the title, and the warning were written by
different people. One of them is right. If you know how SslBump works,
you know which part is correct. At that time, please feel free to
propose changes to fix the wiki page. Hundreds of folks receive SslBump
help on this mailing list, but only one of them took the pains to
improve the page afterwards (thank you again, Marcus Kool!). Parts of
that page still need a lot of work.

HTH,

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid TPROXY issues with Google sites

2017-05-26 Thread Amos Jeffries

On 27/05/17 03:44, Vieri wrote:

Hi,

I'd like to block access to Google Mail but allow it to Google Drive. I also
need to intercept Google Drive traffic (https) and scan its content via c-icap
modules for threats (with clamav and other tools which would block potentially
harmful files).

I've failed so far.

I added mail.google.com to a custom file named "denied.domains" and loaded as
denied_domains ACL in Squid. I know that in TLS traffic there are only IP addresses, so I created
the "server_name" ACL as seen below.

Erm, not quite. The raw-IP having to be dealt with comes from the use of
TPROXY (or NAT), not TLS. It is used when Squid is deciding whether to
permit the traffic on the TCP connection to be processed.

Once the TLS is received (by "peek") the TLS SNI (if any) becomes available.

[...]
acl denied_domains dstdomain "/usr/local/share/proxy-settings/denied.domains"
http_access deny denied_domains !allowed_groups !allowed_ips
http_access deny CONNECT denied_domains !allowed_groups !allowed_ips
[...]
reply_header_access Alternate-Protocol deny all
acl AllowTroublesome ssl::server_name .google.com .gmail.com
acl DenyTroublesome ssl::server_name mail.google.com
http_access deny DenyTroublesome
ssl_bump peek all
ssl_bump splice AllowTroublesome
ssl_bump bump all

First of all, I was expecting that if a client tried to open
https://mail.google.com, the connection would be blocked by Squid
(DenyTroublesome ACL). It isn't. Why?

Any of the http_access lines you omitted from the config snippet might
be letting it through. Order is important, and knowing the whole
http_access sequence (and more) is just as important to correctly answer
a question such as this. So take the below with a grain of salt, I am
assuming nothing else in your config has subtle effects on the
processing outcome.

There are several things that can lead to it;

* Google servers do have working rDNS. So raw-IP becomes a server
hostname for dstdomain ACL to match.
- the rDNS is within *.1e1.net so will not match your list shown, but
it is enough to possibly evade your IP rules.

* If no provides access control lines match Squid inverted the action on
the last one and does that.

- yours are all deny, so the implicit action there is "allow all"

* "ssl_bump peek all" fetches the TLS SNI server name for
ssl::server_name ACL to match.
- so by the time Squid gets to processing the AllowTroublesome it
already knows the client is trying to reach a *.google.com domain.

Second, I am unable to scan content since Squid is splicing all Google traffic. However,
if I "bump AllowTroublesome", I can enter my username in
https://accounts.google.com, but trying to access to the next step (user password) fails
with an unreported error.

Any suggestions?

The rest of your related squid.conf is needed for that, including
details of the files includes into the ACLs. In particular it is not
clear what this "unreported error" is or why it happens.

87 matches

Mail list logo