Re: [squid-users] filtering http(s) sites, transparently

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 


27.10.2016 4:37, Amos Jeffries пишет:
> On 27/10/2016 7:55 a.m., Yuri Voinov wrote:
>>
>> 27.10.2016 0:54, Jok Thuau пишет:
>>
>>> Setting up the client and the proxy to use a common infrastructure for
>>> DNS (dnsmasq on the network) helped a lot.
>>
>> Yes, this is common and best practice already. I think, time to write
>> article on Wiki ;)
>>
>
> Thanks for the reminder. This and the SSL-Bump issue are now mentioned
> in 
Wow, good to know. Much time ago last checked this page
>
>
> Amos
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

- -- 
Cats - delicious. You just do not know how to cook them.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJYETPzAAoJENNXIZxhPexGUBcIAJ4C+/V+V0mMqQpyFYgR2u/J
BMpUnKuwGXCX3Cq+j+rH4J9/IQgk1vKtxkV5JrgJaE+4R80KLQUDlYmHgzAAlo1Y
UwuR2bljRd07JgLn+Oz5oirBCFF8aP2vTXnMaZG3EmAyXQeclyb69xVfIKXdungE
3XxFKkOjARdIcGa/md2gZswCVKlF4mZCi38p4i0Smps1zyVMEhpPBN6J/3rHT2ur
rX6sBDPNbGwn34a7XEm7AP0/91qj8CW3ddz4AH23iQsP67pGPJg/IWKnhZIgMbwz
u9XTq4xmZddEIKUT1awghXMgamyIqo3oKD+OzidwCJvIV1HI4YSfWYIaN3DfiM8=
=4j39
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[Amos Jeffries]

On 27/10/2016 7:55 a.m., Yuri Voinov wrote:
> 
> 27.10.2016 0:54, Jok Thuau пишет:
> 
>> Setting up the client and the proxy to use a common infrastructure for
>> DNS (dnsmasq on the network) helped a lot.
> 
> Yes, this is common and best practice already. I think, time to write
> article on Wiki ;)
> 

Thanks for the reminder. This and the SSL-Bump issue are now mentioned
in 

Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 


27.10.2016 0:54, Jok Thuau пишет:
>
> On Wed, Oct 26, 2016 at 11:45 AM, Yuri Voinov > wrote:
>
>
>
> Jok,
>
> it can be DNS leak. Does you tested it? 8.8.8.8 can be poisoned
(probably) or intercepted by ISP.
>
>
> DNS is working fine and is not being poisoned/intercepted/messed with.
The records that come back from the google servers appear to not be
consistent (likely due to some anycast system, and not talking each time
to the same "host"). So when i request the same records back to back,
each results in one record, and that record changes really fast
(non-coherent set of data, so the results are correct, but random).
Setting up the client and the proxy to use a common infrastructure for
DNS (dnsmasq on the network) helped a lot.
Yes, this is common and best practice already. I think, time to write
article on Wiki ;)
>
> Thanks,
> Jok

- -- 
Cats - delicious. You just do not know how to cook them.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 
iQEcBAEBCAAGBQJYEPw1AAoJENNXIZxhPexGIW0H/Rk82EjHy/UfQm44SvsHgBeq
Pw5b1yavLtFNXSpRsLyw8wekepJvLLk1XEtGWbLC33Z3O7REBYXL2nzXD9iNzFbp
RhdF4aaIgCfp+WqHtVxRgnqoHNAmDs2U7uhJqYmXIubvbFyddRwwh/vS2Ns89/t7
BK9GuqkkeG4PrGG3ogAX8YpRaE57LaTcDXrOlco5JU/wGkxbMJzUxOvmFyl+0SLI
4xbUBgEaFFAAmJ46PWm3c8e+zo5O6k2E86asfDJUCMtnKvRPnhce8MxTH8MwkLxl
GR5UNyVAIpYWlJAqjDRRwYlEcTGfxofyZD3SKKDUP4SduwicdZArBGGirtKUdJA=
=uXwb
-END PGP SIGNATURE-



0x613DEC46.asc
Description: application/pgp-keys
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[Jok Thuau]

On Wed, Oct 26, 2016 at 11:45 AM, Yuri Voinov  wrote:

>
>
> Jok,
>
> it can be DNS leak. Does you tested it? 8.8.8.8 can be poisoned (probably)
> or intercepted by ISP.
>
>
DNS is working fine and is not being poisoned/intercepted/messed with. The
records that come back from the google servers appear to not be consistent
(likely due to some anycast system, and not talking each time to the same
"host"). So when i request the same records back to back, each results in
one record, and that record changes really fast (non-coherent set of data,
so the results are correct, but random). Setting up the client and the
proxy to use a common infrastructure for DNS (dnsmasq on the network)
helped a lot.

Thanks,
Jok
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[Yuri Voinov]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
 
Jok,

it can be DNS leak. Does you tested it? 8.8.8.8 can be poisoned
(probably) or intercepted by ISP.


27.10.2016 0:01, Jok Thuau пишет:
> After being side-tracked with a few different project, I ended up with the 
> config below. It appears to
do the right things, though the ACL organization could use some cleanup...
> (Browsing to authorized sites works, browsing to something else, i get
a denied page from squid)
>
> However, even though msdn.microsoft.com  is
on my whitelist, it appears to "timeout". Looking at the logs, I see
entries like this:
>
> 2016/09/20 15:10:19.640 kid1| SECURITY ALERT: Host header forgery
detected on local=65.54.226.150:443 
remote=10.11.12.13:51984  FD 18 flags=33
(local IP does not match any domain IP)
> 2016/09/20 15:10:19.640 kid1| SECURITY ALERT: By user agent:
> 2016/09/20 15:10:19.640 kid1| SECURITY ALERT: on URL:
msdn.microsoft.com:443 
> 2016/09/20 15:10:19.640 kid1| 4,2| errorpage.cc(1262) BuildContent: No
existing error page language negotiated for ERR_CONFLICT_HOST. Using
default error file.
> 2016/09/20 15:10:19.641 kid1| 20,2| store.cc(954) checkCachable:
StoreEntry::checkCachable: NO: not cachable
> 2016/09/20 15:10:19.641 kid1| 20,2| store.cc(954) checkCachable:
StoreEntry::checkCachable: NO: not cachable
> 2016/09/20 15:10:19.641 kid1| 88,2| client_side_reply.cc(2001)
processReplyAccessResult: The reply for CONNECT msdn.microsoft.com:443
 is ALLOWED, because it matched SniBypass
> 2016/09/20 15:10:19.641 kid1| 33,2| client_side.cc(925)
deferRecipientForLater: clientSocketRecipient: Deferring request
msdn.microsoft.com:443 
>
> What is interesting is that we changed from using 8.8.8.8 as our DNS
server to a locally installed bind instance, and we no longer see any
issues.
> that is NOT what I expected... I can't explain why the client browser
was hanging. Any suggestions?
>
> It's all happy now, with the following config, with a firewall doing
policy based routing, a local iptables rule to redirect from port 443 to
8443, and from 80 to 3129, as well as the certificate deployed as
trusted on each endpoint:
>
> squid.conf:
> # setup standard ports
> acl SSL_ports port 443
> acl Safe_ports port 80
> acl Safe_ports port 21
> acl Safe_ports port 443
> acl Safe_ports port 70
> acl Safe_ports port 210
> acl Safe_ports port 1025-65535
> acl Safe_ports port 280
> acl Safe_ports port 488
> acl Safe_ports port 591
> acl Safe_ports port 777
> acl CONNECT method CONNECT
> # for security
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
>
> # not actually used and blocked by local firewall on host, but squid
complains if it's not there...
> http_port 3128
>
> # http intercept, and ACL that matches that inbound port
> http_port 3129 intercept
> acl http_proxy myportname 3129
>
> # same for https
> https_port 8443 intercept ssl-bump \
> generate-host-certificates=on \
> dynamic_cert_mem_cache_size=64MB \
> cert=/etc/squid/ssl/proxy.pem \
> key=/etc/squid/ssl/proxy.key \
> cafile=/etc/squid/ssl/proxy.pe m
> acl https_proxy myportname 8443
>
> always_direct allow all
>
> # define the steps needed for bumping...
> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> acl step3 at_step SslBump3
>
> # and the list of domains that are allowed
> acl SniBypass ssl::server_name_regex -i "/etc/squid/snibypass.acl"
> acl http_bypass dstdom_regex -n -i "/etc/squid/snibypass.acl"
>
> # ensure we have target SSL port when checking data
> acl https_ok all-of SniBypass SSL_ports
> # and the destination domain when not SSL...
> acl http_ok all-of http_bypass Safe_ports
>
> # splice when we know the target matches
> ssl_bump splice SniBypass
> # peek and bump otherwise
> ssl_bump peek step1
> ssl_bump stare step2
> ssl_bump bump all
>
> # some options for the certificate generation..
> sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_ECDH_USE
> sslproxy_cert_sign_hash sha256
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 64MB
> sslcrtd_children 8 startup=1 idle=1
>
> # for http, let's block if we're not on the whitelist
> http_access deny !http_okhttp_proxy
> # for https, let's wait until step3 of the bumping, so we can replace
the SSL content post-bumping)
> http_access deny !https_ok step3 https_proxy
>
> # never cache anything
> cache deny all
>
> #for windows updates
> quick_abort_min -1
> range_offset_limit 0 all
>
> shutdown_lifetime 2 seconds
> connect_timeout 20 seconds
> #debug_options ALL,2
>
>
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

- -- 
Cats - delicious. You just do not know how to cook them.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
 

Re: [squid-users] filtering http(s) sites, transparently

[Jok Thuau]

After being side-tracked with a few different project, I ended up with the
config below. It appears to do the right things, though the ACL
organization could use some cleanup...
(Browsing to authorized sites works, browsing to something else, i get a
denied page from squid)

However, even though msdn.microsoft.com is on my whitelist, it appears to
"timeout". Looking at the logs, I see entries like this:

2016/09/20 15:10:19.640 kid1| SECURITY ALERT: Host header forgery detected
on local=65.54.226.150:443 remote=10.11.12.13:51984
 FD 18 flags=33 (local IP does not match any
domain IP)
2016/09/20 15:10:19.640 kid1| SECURITY ALERT: By user agent:
2016/09/20 15:10:19.640 kid1| SECURITY ALERT: on URL: msdn.microsoft.com:443
2016/09/20 15:10:19.640 kid1| 4,2| errorpage.cc(1262) BuildContent: No
existing error page language negotiated for ERR_CONFLICT_HOST. Using
default error file.
2016/09/20 15:10:19.641 kid1| 20,2| store.cc(954) checkCachable:
StoreEntry::checkCachable: NO: not cachable
2016/09/20 15:10:19.641 kid1| 20,2| store.cc(954) checkCachable:
StoreEntry::checkCachable: NO: not cachable
2016/09/20 15:10:19.641 kid1| 88,2| client_side_reply.cc(2001)
processReplyAccessResult: The reply for CONNECT msdn.microsoft.com:443 is
ALLOWED, because it matched SniBypass
2016/09/20 15:10:19.641 kid1| 33,2| client_side.cc(925)
deferRecipientForLater: clientSocketRecipient: Deferring request
msdn.microsoft.com:443

What is interesting is that we changed from using 8.8.8.8 as our DNS server
to a locally installed bind instance, and we no longer see any issues.
that is NOT what I expected... I can't explain why the client browser was
hanging. Any suggestions?

It's all happy now, with the following config, with a firewall doing policy
based routing, a local iptables rule to redirect from port 443 to 8443, and
from 80 to 3129, as well as the certificate deployed as trusted on each
endpoint:

squid.conf:
# setup standard ports
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
# for security
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports

# not actually used and blocked by local firewall on host, but squid
complains if it's not there...
http_port 3128

# http intercept, and ACL that matches that inbound port
http_port 3129 intercept
acl http_proxy myportname 3129

# same for https
https_port 8443 intercept ssl-bump \
generate-host-certificates=on \
dynamic_cert_mem_cache_size=64MB \
cert=/etc/squid/ssl/proxy.pem \
key=/etc/squid/ssl/proxy.key \
cafile=/etc/squid/ssl/proxy.pem
acl https_proxy myportname 8443

always_direct allow all

# define the steps needed for bumping...
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

# and the list of domains that are allowed
acl SniBypass ssl::server_name_regex -i "/etc/squid/snibypass.acl"
acl http_bypass dstdom_regex -n -i "/etc/squid/snibypass.acl"

# ensure we have target SSL port when checking data
acl https_ok all-of SniBypass SSL_ports
# and the destination domain when not SSL...
acl http_ok all-of http_bypass Safe_ports

# splice when we know the target matches
ssl_bump splice SniBypass
# peek and bump otherwise
ssl_bump peek step1
ssl_bump stare step2
ssl_bump bump all

# some options for the certificate generation..
sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_ECDH_USE
sslproxy_cert_sign_hash sha256
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 64MB
sslcrtd_children 8 startup=1 idle=1

# for http, let's block if we're not on the whitelist
http_access deny !http_okhttp_proxy
# for https, let's wait until step3 of the bumping, so we can replace the
SSL content post-bumping)
http_access deny !https_ok step3 https_proxy

# never cache anything
cache deny all

#for windows updates
quick_abort_min -1
range_offset_limit 0 all

shutdown_lifetime 2 seconds
connect_timeout 20 seconds
#debug_options ALL,2
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[Amos Jeffries]

On 6/04/2016 6:50 a.m., Jok Thuau wrote:
> On Mon, Apr 4, 2016 at 6:23 PM, Amos Jeffries  wrote:
> 
>
> If i remove *all* the http_access lines, then the behavior appears
 correct
> (from a "splicing/bumping" standpoint).
>

 Strange. Squid without any http_access lines should be denying traffic
 100%.


>>> I do not see this behavior. Traffic appears to be allowed, and bumped
>>> (though with the wrong certificate, depending on the config, as explained
>>> before).
>>>
>>>
>>
> 
> 
> 
> 
>>>
>>> my apologies for trying to show only the relevant parts. Find below the
>>> current config.
>>> It appears to be bumping everything rather than splicing any of the
>> config
>>> (which may be due to the limitations documented on the wiki)
>>>
>>> acl Safe_ports port 80 # http
>>> acl Safe_ports port 443 # https
>>> acl SSL_ports port 443
>>> acl CONNECT method CONNECT
>>> http_port 3129 intercept
>>> https_port 8443 intercept ssl-bump generate-host-certificates=on
>>> dynamic_cert_mem_cache_size=64MB \
>>> cert=/etc/squid/ssl/proxy.pem \
>>> key=/etc/squid/ssl/proxy.key \
>>> cafile=/etc/squid/ssl/proxy.pem
>>> always_direct allow all
>>
>> always_direct has not been necessary with SSL-Bump sice 3.1 series. You
>> should remove it.
>>
>>> acl step1 at_step SslBump1
>>> acl step2 at_step SslBump2
>>> acl step3 at_step SslBump3
>>> acl SniBypass ssl::server_name_regex \.slashdot\.org
>>> acl SniBypass ssl::server_name_regex \.fsdn\.com
>>
> 
> I have moved those "SniBypass" acl into a separate files and replaced this
> with an include, as that list will end up growing.
> 

It would be better to add only the list of values to a file. It will
save you entering the ACL name and type repeatedly in the included file.
Like this ...

/some/file/bypass_domains:
 .slashdot.org
 .fsdn.com


/etc/squid/squid.conf:
 ...
 acl http_bypass dstdomain "/some/file/bypass_domains"



> 
>>> acl http_bypass dstdomain .slashdot.org
>>> acl http_bypass dstdomain .fsdn.com
> 
> 
> and similarly here, replaced by an include...
> 
> 
>>
>>> acl https_bypass all-of CONNECT SniBypass
>>
>> This https_bypass ACL definition is a bit weird. It requires a single
>> message to match both TLS and HTTP properties simultaneously.
> 
> 
>> As you might imagine it is difficult for a TLS messages to match HTTP
>> properties, and vice versa. So it wont ever match.
>>
>>
> I don't understand. SniBypass is based on ssl::server_name_regex which
> shouldn't apply to http at all...

Yes.

> Would that not be coming from the (client|server)Hello?
> 

yes.

Also the CONNECT ACL is based on the HTTP method.

For the https_bypass ACL to return true/match the message being tested
must match both ("all-of") CONNECT and SniBypass.

HTTP messages cannot match SniBypass.
TLS messages cannot match CONNECT.
Therefore nothing will ever match all of https_bypass requirements.

> 
>> Note: SNI is *not* equivalent to Host or URL domain name. They can
>> contain very different values. The only thing they have in common is
>> that they both are supposed to point at the IP of the server being
>> contacted.
>>
>>
>>> acl http_ok all-of http_bypass Safe_ports
>>> ssl_bump peek step1
>>> ssl_bump splice SniBypass step2
>>
>> This splice will work if (and only if) the client sends TLS SNI values
>> to Squid. It will ignore the server cert details.
>>
>> For clients which do not send SNI or for all connections where the SNI
>> does not match your ACL the bump rule below will do client-first bumping
>> (without the server cert).
>>
>>> ssl_bump bump all
>>
>> I suggets you try these ssl_bump rules instead:
>> [snip]
> 
>  OK
> 
>> [snip]
>> Okay. That sort of matches your policy. Except that you are missing the
>> security defaults. Those lines are carefully tuned for the specific
>> behaviour to protect against security attacks:
>>
>>  http_access deny !Safe_ports
>>  http_access deny CONNECT !SSL_ports
>>
>> .. and should be above your custom rules.
>>
> 
> I added those at the top as requested...
> 
> 
>>  cache allow all
>>  cache deny all
>>
>> ... pick one.
>>
>>
> done - the deny one is the one left in there now.
> 
>>
>>> shutdown_lifetime 3 seconds
>>
> 
> for clarification, I also moved the two sets of ACLs into separate files,
> as those will eventually be maintained externally (SniBypass and
> http_bypass).
> 
> The config file is now:
> 
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_port 3128
> http_port 3129 intercept
> https_port 8443 intercept 

Re: [squid-users] filtering http(s) sites, transparently

[Jok Thuau]

On Mon, Apr 4, 2016 at 6:23 PM, Amos Jeffries  wrote:

> >>>
> >>> If i remove *all* the http_access lines, then the behavior appears
> >> correct
> >>> (from a "splicing/bumping" standpoint).
> >>>
> >>
> >> Strange. Squid without any http_access lines should be denying traffic
> >> 100%.
> >>
> >>
> > I do not see this behavior. Traffic appears to be allowed, and bumped
> > (though with the wrong certificate, depending on the config, as explained
> > before).
> >
> >
>




> >
> > my apologies for trying to show only the relevant parts. Find below the
> > current config.
> > It appears to be bumping everything rather than splicing any of the
> config
> > (which may be due to the limitations documented on the wiki)
> >
> > acl Safe_ports port 80 # http
> > acl Safe_ports port 443 # https
> > acl SSL_ports port 443
> > acl CONNECT method CONNECT
> > http_port 3129 intercept
> > https_port 8443 intercept ssl-bump generate-host-certificates=on
> > dynamic_cert_mem_cache_size=64MB \
> > cert=/etc/squid/ssl/proxy.pem \
> > key=/etc/squid/ssl/proxy.key \
> > cafile=/etc/squid/ssl/proxy.pem
> > always_direct allow all
>
> always_direct has not been necessary with SSL-Bump sice 3.1 series. You
> should remove it.
>
> > acl step1 at_step SslBump1
> > acl step2 at_step SslBump2
> > acl step3 at_step SslBump3
> > acl SniBypass ssl::server_name_regex \.slashdot\.org
> > acl SniBypass ssl::server_name_regex \.fsdn\.com
>

I have moved those "SniBypass" acl into a separate files and replaced this
with an include, as that list will end up growing.


> > acl http_bypass dstdomain .slashdot.org
> > acl http_bypass dstdomain .fsdn.com


and similarly here, replaced by an include...


>
> > acl https_bypass all-of CONNECT SniBypass
>
> This https_bypass ACL definition is a bit weird. It requires a single
> message to match both TLS and HTTP properties simultaneously.


> As you might imagine it is difficult for a TLS messages to match HTTP
> properties, and vice versa. So it wont ever match.
>
>
I don't understand. SniBypass is based on ssl::server_name_regex which
shouldn't apply to http at all...
Would that not be coming from the (client|server)Hello?


> Note: SNI is *not* equivalent to Host or URL domain name. They can
> contain very different values. The only thing they have in common is
> that they both are supposed to point at the IP of the server being
> contacted.
>
>
> > acl http_ok all-of http_bypass Safe_ports
> > ssl_bump peek step1
> > ssl_bump splice SniBypass step2
>
> This splice will work if (and only if) the client sends TLS SNI values
> to Squid. It will ignore the server cert details.
>
> For clients which do not send SNI or for all connections where the SNI
> does not match your ACL the bump rule below will do client-first bumping
> (without the server cert).
>
> > ssl_bump bump all
>
> I suggets you try these ssl_bump rules instead:
> [snip]

 OK

> [snip]
> Okay. That sort of matches your policy. Except that you are missing the
> security defaults. Those lines are carefully tuned for the specific
> behaviour to protect against security attacks:
>
>  http_access deny !Safe_ports
>  http_access deny CONNECT !SSL_ports
>
> .. and should be above your custom rules.
>

I added those at the top as requested...


>  cache allow all
>  cache deny all
>
> ... pick one.
>
>
done - the deny one is the one left in there now.

>
> > shutdown_lifetime 3 seconds
>

for clarification, I also moved the two sets of ACLs into separate files,
as those will eventually be maintained externally (SniBypass and
http_bypass).

The config file is now:

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_port 3128
http_port 3129 intercept
https_port 8443 intercept ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=64MB \
cert=/etc/squid/ssl/proxy.pem \
key=/etc/squid/ssl/proxy.key \
cafile=/etc/squid/ssl/proxy.pem
workers 6
always_direct allow all
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
include "/etc/squid/snibypass.acl"
include "/etc/squid/dstbypass.acl"
acl https_ok all-of CONNECT SniBypass
acl http_ok all-of http_bypass Safe_ports
ssl_bump splice SniBypass
ssl_bump peek step1
ssl_bump stare step2
ssl_bump bump all
sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_ECDH_USE
sslproxy_cert_sign_hash sha256
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1
http_access allow http_ok
http_access allow https_ok
http_access deny all
cache deny all
shutdown_lifetime 3 seconds

Re: [squid-users] filtering http(s) sites, transparently

[Amos Jeffries]

On 5/04/2016 4:49 a.m., Jok Thuau wrote:
> On Sun, Apr 3, 2016 at 9:59 PM, Amos Jeffries  wrote:
> 
>> On 4/04/2016 4:18 p.m., Jok Thuau wrote:
>>> I'm attempting to build a transparent proxy (policy based routing on
>>> firewall to squid proxy) with the following behavior:
>>>
>>> 1) proxies http traffic for a given set of domains, provide an message
>>> otherwise such "domain not allowed" or similar
>>> 2) proxies https traffic for a given set of domains (ideally, splicing
>>> those, so as not to break HSTS, if enabled), otherwise provide an error
>>> message (bumping and providing "domain not allowed")
>>>
>>> I'm attempting this with a 3.5.15 compiled with icap (not yet used) and
>>> ssl-bumping.
>>>
>>> Part 1 seems easy enough (and is well documented)...
>>>
>>> acl whitelist dstdomain .domain1.tld
>>> acl whitelist dstdomain .domain2.tld
>>>
>>> acl http_ok all-of whitelist !SSL_ports
>>>
>>> http_access allow http_ok
>>> http_access deny all
>>
>> This is denying the HTTPS traffic CONNECT requests (synthesized by
>> Squid), since they only have IP address no domain name.
>>
> 
> yes, this is where I started with just http.
> 
> 
>>
>>
>>>
>>> Moving onto Part 2 (the peek and splice setup) appears to be the topic
>> of a
>>> few discussions out there...
>>>
>>> acl sni_whitelist ssl::server_name .domain1.tld
>>> acl sni_whitelist ssl::server_name .domain2.tld
>>>
>>> ssl_bump peek step1
>>
>> You have omitted the definition of step1 ACL.
>>
>>
> the definition of "step1" is the same as the one on the wiki (See full
> config below)
> 
> 
>>> ssl_bump splice sni_whitelist
>>> ssl_bump bump all
>>>
>>> It appears however that when combining the two, the generated
>>> certificate(s), instead of mimic'ing the original server's certificate
>>> comes out with the CN= where  is the ip used by the "connect"
>> part
>>> of the connection. In addition, it appears that only the first entry ever
>>> matches (at this point, i've tried so many combinations, i'm no longer
>>> certain of anything).
>>
>> You have omitted the http(s)_port configuration details, and the step1
>> ACL. So its not possible to say if you have the cert generation settings
>> wrong, or if the peeking step is matching wrong, or something else.
>>
>>
> That's included in the config below.
> 
> 
>>>
>>> If i remove *all* the http_access lines, then the behavior appears
>> correct
>>> (from a "splicing/bumping" standpoint).
>>>
>>
>> Strange. Squid without any http_access lines should be denying traffic
>> 100%.
>>
>>
> I do not see this behavior. Traffic appears to be allowed, and bumped
> (though with the wrong certificate, depending on the config, as explained
> before).
> 
> 
>>> Can anyone confirm that this is indeed possible to achieve?
>>>
>>> I believe, based on experimentation that any http_access i have, because
>> of
>>> the "deny all" cause the bumping to "short circuit" and effectively send
>> an
>>> early "access denied" based on the only information it has (the ip
>> address
>>> from the "connect", rather than the SNI that would come later).
>>>
>>> Would a setup where "deny http+!whitelist" so have the allow be the
>> default
>>> allow for the bumping to work and get to step2 and match the sni* acls
>>> somehow? (with a "deny step2 !sni_whitelist").
>>>
>>> Is 3.5.15 capable of doing this? If this requires some feature/effort,
>> what
>>> would be the procedure to sponsor that work?
>>
>> It is not possible to answer any of those questsions properly without
>> full config details. You have omitted a lot.
>>
> 
> my apologies for trying to show only the relevant parts. Find below the
> current config.
> It appears to be bumping everything rather than splicing any of the config
> (which may be due to the limitations documented on the wiki)
> 
> acl Safe_ports port 80 # http
> acl Safe_ports port 443 # https
> acl SSL_ports port 443
> acl CONNECT method CONNECT
> http_port 3129 intercept
> https_port 8443 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=64MB \
> cert=/etc/squid/ssl/proxy.pem \
> key=/etc/squid/ssl/proxy.key \
> cafile=/etc/squid/ssl/proxy.pem
> always_direct allow all

always_direct has not been necessary with SSL-Bump sice 3.1 series. You
should remove it.

> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> acl step3 at_step SslBump3
> acl SniBypass ssl::server_name_regex \.slashdot\.org
> acl SniBypass ssl::server_name_regex \.fsdn\.com
> acl http_bypass dstdomain .slashdot.org
> acl http_bypass dstdomain .fsdn.com
> acl https_bypass all-of CONNECT SniBypass

This https_bypass ACL definition is a bit weird. It requires a single
message to match both TLS and HTTP properties simultaneously.

As you might imagine it is difficult for a TLS messages to match HTTP
properties, and vice versa. So it wont ever match.

Note: SNI is *not* equivalent to Host or URL domain name. They can
contain very different values. The only thing they have in common 

Re: [squid-users] filtering http(s) sites, transparently

[Jok Thuau]

Thanks James! This is really close to what I need. Comparing this to my
existing config, it looks like i'm pretty close, except that i don't want
to "terminate" the sslbump, i need to send an error notification to the
end-user.

​
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[Jok Thuau]

On Sun, Apr 3, 2016 at 9:59 PM, Amos Jeffries  wrote:

> On 4/04/2016 4:18 p.m., Jok Thuau wrote:
> > I'm attempting to build a transparent proxy (policy based routing on
> > firewall to squid proxy) with the following behavior:
> >
> > 1) proxies http traffic for a given set of domains, provide an message
> > otherwise such "domain not allowed" or similar
> > 2) proxies https traffic for a given set of domains (ideally, splicing
> > those, so as not to break HSTS, if enabled), otherwise provide an error
> > message (bumping and providing "domain not allowed")
> >
> > I'm attempting this with a 3.5.15 compiled with icap (not yet used) and
> > ssl-bumping.
> >
> > Part 1 seems easy enough (and is well documented)...
> >
> > acl whitelist dstdomain .domain1.tld
> > acl whitelist dstdomain .domain2.tld
> >
> > acl http_ok all-of whitelist !SSL_ports
> >
> > http_access allow http_ok
> > http_access deny all
>
> This is denying the HTTPS traffic CONNECT requests (synthesized by
> Squid), since they only have IP address no domain name.
>

yes, this is where I started with just http.


>
>
> >
> > Moving onto Part 2 (the peek and splice setup) appears to be the topic
> of a
> > few discussions out there...
> >
> > acl sni_whitelist ssl::server_name .domain1.tld
> > acl sni_whitelist ssl::server_name .domain2.tld
> >
> > ssl_bump peek step1
>
> You have omitted the definition of step1 ACL.
>
>
the definition of "step1" is the same as the one on the wiki (See full
config below)


> > ssl_bump splice sni_whitelist
> > ssl_bump bump all
> >
> > It appears however that when combining the two, the generated
> > certificate(s), instead of mimic'ing the original server's certificate
> > comes out with the CN= where  is the ip used by the "connect"
> part
> > of the connection. In addition, it appears that only the first entry ever
> > matches (at this point, i've tried so many combinations, i'm no longer
> > certain of anything).
>
> You have omitted the http(s)_port configuration details, and the step1
> ACL. So its not possible to say if you have the cert generation settings
> wrong, or if the peeking step is matching wrong, or something else.
>
>
That's included in the config below.


> >
> > If i remove *all* the http_access lines, then the behavior appears
> correct
> > (from a "splicing/bumping" standpoint).
> >
>
> Strange. Squid without any http_access lines should be denying traffic
> 100%.
>
>
I do not see this behavior. Traffic appears to be allowed, and bumped
(though with the wrong certificate, depending on the config, as explained
before).


> > Can anyone confirm that this is indeed possible to achieve?
> >
> > I believe, based on experimentation that any http_access i have, because
> of
> > the "deny all" cause the bumping to "short circuit" and effectively send
> an
> > early "access denied" based on the only information it has (the ip
> address
> > from the "connect", rather than the SNI that would come later).
> >
> > Would a setup where "deny http+!whitelist" so have the allow be the
> default
> > allow for the bumping to work and get to step2 and match the sni* acls
> > somehow? (with a "deny step2 !sni_whitelist").
> >
> > Is 3.5.15 capable of doing this? If this requires some feature/effort,
> what
> > would be the procedure to sponsor that work?
>
> It is not possible to answer any of those questsions properly without
> full config details. You have omitted a lot.
>

my apologies for trying to show only the relevant parts. Find below the
current config.
It appears to be bumping everything rather than splicing any of the config
(which may be due to the limitations documented on the wiki)

acl Safe_ports port 80 # http
acl Safe_ports port 443 # https
acl SSL_ports port 443
acl CONNECT method CONNECT
http_port 3129 intercept
https_port 8443 intercept ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=64MB \
cert=/etc/squid/ssl/proxy.pem \
key=/etc/squid/ssl/proxy.key \
cafile=/etc/squid/ssl/proxy.pem
always_direct allow all
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
acl SniBypass ssl::server_name_regex \.slashdot\.org
acl SniBypass ssl::server_name_regex \.fsdn\.com
acl http_bypass dstdomain .slashdot.org
acl http_bypass dstdomain .fsdn.com
acl https_bypass all-of CONNECT SniBypass
acl http_ok all-of http_bypass Safe_ports
ssl_bump peek step1
ssl_bump splice SniBypass step2
ssl_bump bump all
sslproxy_cert_sign_hash sha256
sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_ECDH_USE
always_direct allow all
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1
http_access allow http_ok
http_access allow CONNECT
no_cache allow all
cache deny all
shutdown_lifetime 3 seconds
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[James Lay]

On Sun, 2016-04-03 at 21:18 -0700, Jok Thuau wrote:
> I'm attempting to build a transparent proxy (policy based routing on
> firewall to squid proxy) with the following behavior:
> 
> 
> 
> 1) proxies http traffic for a given set of domains, provide an message
> otherwise such "domain not allowed" or similar
> 2) proxies https traffic for a given set of domains (ideally, splicing
> those, so as not to break HSTS, if enabled), otherwise provide an
> error message (bumping and providing "domain not allowed")
> 
> 
> 
> I'm attempting this with a 3.5.15 compiled with icap (not yet used)
> and ssl-bumping.
> 
> 
> Part 1 seems easy enough (and is well documented)...
> 
> 
> acl whitelist dstdomain .domain1.tld
> 
> acl whitelist dstdomain .domain2.tld
> 
> 
> acl http_ok all-of whitelist !SSL_ports
> 
> 
> http_access allow http_ok
> http_access deny all
> 
> 
> Moving onto Part 2 (the peek and splice setup) appears to be the topic
> of a few discussions out there...
> 
> 
> acl sni_whitelist ssl::server_name .domain1.tld
> acl sni_whitelist ssl::server_name .domain2.tld
> 
> 
> ssl_bump peek step1
> ssl_bump splice sni_whitelist
> ssl_bump bump all
> 
> 
> It appears however that when combining the two, the generated
> certificate(s), instead of mimic'ing the original server's certificate
> comes out with the CN= where  is the ip used by the "connect"
> part of the connection. In addition, it appears that only the first
> entry ever matches (at this point, i've tried so many combinations,
> i'm no longer certain of anything). 
> 
> 
> If i remove *all* the http_access lines, then the behavior appears
> correct (from a "splicing/bumping" standpoint).
> 
> 
> Can anyone confirm that this is indeed possible to achieve?
> 
> 
> I believe, based on experimentation that any http_access i have,
> because of the "deny all" cause the bumping to "short circuit" and
> effectively send an early "access denied" based on the only
> information it has (the ip address from the "connect", rather than the
> SNI that would come later). 
> 
> 
> Would a setup where "deny http+!whitelist" so have the allow be the
> default allow for the bumping to work and get to step2 and match the
> sni* acls somehow? (with a "deny step2 !sni_whitelist").
> 
> 
> Is 3.5.15 capable of doing this? If this requires some feature/effort,
> what would be the procedure to sponsor that work?
> 
> 
> Thanks,
> Jok
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


This may assist:

http://article.gmane.org/gmane.comp.web.squid.general/114389

James
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] filtering http(s) sites, transparently

[Amos Jeffries]

On 4/04/2016 4:18 p.m., Jok Thuau wrote:
> I'm attempting to build a transparent proxy (policy based routing on
> firewall to squid proxy) with the following behavior:
> 
> 1) proxies http traffic for a given set of domains, provide an message
> otherwise such "domain not allowed" or similar
> 2) proxies https traffic for a given set of domains (ideally, splicing
> those, so as not to break HSTS, if enabled), otherwise provide an error
> message (bumping and providing "domain not allowed")
> 
> I'm attempting this with a 3.5.15 compiled with icap (not yet used) and
> ssl-bumping.
> 
> Part 1 seems easy enough (and is well documented)...
> 
> acl whitelist dstdomain .domain1.tld
> acl whitelist dstdomain .domain2.tld
> 
> acl http_ok all-of whitelist !SSL_ports
> 
> http_access allow http_ok
> http_access deny all

This is denying the HTTPS traffic CONNECT requests (synthesized by
Squid), since they only have IP address no domain name.


> 
> Moving onto Part 2 (the peek and splice setup) appears to be the topic of a
> few discussions out there...
> 
> acl sni_whitelist ssl::server_name .domain1.tld
> acl sni_whitelist ssl::server_name .domain2.tld
> 
> ssl_bump peek step1

You have omitted the definition of step1 ACL.

> ssl_bump splice sni_whitelist
> ssl_bump bump all
> 
> It appears however that when combining the two, the generated
> certificate(s), instead of mimic'ing the original server's certificate
> comes out with the CN= where  is the ip used by the "connect" part
> of the connection. In addition, it appears that only the first entry ever
> matches (at this point, i've tried so many combinations, i'm no longer
> certain of anything).

You have omitted the http(s)_port configuration details, and the step1
ACL. So its not possible to say if you have the cert generation settings
wrong, or if the peeking step is matching wrong, or something else.

> 
> If i remove *all* the http_access lines, then the behavior appears correct
> (from a "splicing/bumping" standpoint).
> 

Strange. Squid without any http_access lines should be denying traffic 100%.

> Can anyone confirm that this is indeed possible to achieve?
> 
> I believe, based on experimentation that any http_access i have, because of
> the "deny all" cause the bumping to "short circuit" and effectively send an
> early "access denied" based on the only information it has (the ip address
> from the "connect", rather than the SNI that would come later).
> 
> Would a setup where "deny http+!whitelist" so have the allow be the default
> allow for the bumping to work and get to step2 and match the sni* acls
> somehow? (with a "deny step2 !sni_whitelist").
> 
> Is 3.5.15 capable of doing this? If this requires some feature/effort, what
> would be the procedure to sponsor that work?

It is not possible to answer any of those questsions properly without
full config details. You have omitted a lot.

Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users