Re: [squid-users] Squid 3.5.23-1 is available for Ubuntu 16.04 LTS (online repo ubuntu16.diladele.com)

[Amos Jeffries]

On 2017-01-03 05:25, vinay wrote:

Rafael,
Am facing below 2 issues.

1.I am working on squid 3.3.8 on ubuntu 14.04 , I am trying to upgrade 
the
squid version to higher than the default one I.e higher than 3.4 
version but
am getting an error. "Unable to locate package libecap3" on giving 
apt-get

install libecap3 command. Any suggestions to upgrade the version ?


The eCAP package needs to be rebuilt from the libecap3 source package 
from Xenial.


Or just upgrade to Xenial.



2. In the existing version of squid 3.3.8 am getting the below lines in
access logs
"TCP_MISS/200" and it's not caching .
Any suggestions on this to overcome the caching issue?



Can you provide some actual details about what you think the problem is?

There are things which are simply non-cacheable, and things which should 
never be cached, and things for which a cached response is not possible.


Amos

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid Websocket Issue

[Amos Jeffries]

On 2017-01-02 19:46, Hardik Dangar wrote:

@amos or anyone else from dev team

Can you confirm this is intentional behavior or bug ?



Alex is core dev team, and one of the designers of SSL-Bump. So if he 
says bug, it probably is.




On Mon, Jan 2, 2017 at 9:18 AM, Alex Rousskov wrote:


On 12/27/2016 04:50 AM, Hardik Dangar wrote:


If i remove !serverIsws somehow websockets will not work.


Then there is a bug somewhere AFAICT. It is your call whether to
find
out what that bug is [while continuing to use a potentially
dangerous
workaround].

Alex.



FWIW I agree with him based on the info given. Removal of !serverIsws 
from the rules you had should have had zero effect on the visible 
behaviour of Squid.


Amos
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid Websocket Issue

[Eliezer Croitoru]

I am using the next ontop of squid 3.5.23 in non intercept mode:
##STARTOF SNIPPET
acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex -i "/etc/squid/url.nobump"

ssl_bump splice NoSSLIntercept

ssl_bump peek DiscoverSNIHost
ssl_bump bump all
##END OF SNIPPPET

And the content of url.nobump is:##START OF QUOTE
# WU (Squid 3.5.x and above with SSL Bump)
# Only this sites must be spliced.
update\.microsoft\.com$
update\.microsoft\.com\.akadns\.net$
v10\.vortex\-win\.data\.microsoft.com$
settings\-win\.data\.microsoft\.com$
# The next are trusted SKYPE addresses
a\.config\.skype\.com$
pipe\.skype\.com$
w[0-9]+\.web\.whatsapp\.com$
##END OF  QUOTE

And whatsapp web sockets works for me.

Please be more specific whats not working and on what platform..
If it works for me and not for you there is a difference between our clients or 
systems.
Try using latest 3.5.23.
If you are using an RPM based system you can use one of my RPMS.

Let me know if my rules helps you and if there is a specific site that doesn’t 
work for you,
Eliezer


http://ngtech.co.il/lmgtfy/
Linux System Administrator
Mobile: +972-5-28704261
Email: elie...@ngtech.co.il


From: hardikdan...@gmail.com [mailto:hardikdan...@gmail.com] On Behalf Of 
Hardik Dangar
Sent: Monday, January 2, 2017 6:50 PM
To: Eliezer Croitoru 
Cc: Squid Users 
Subject: Re: [squid-users] Squid Websocket Issue

Hey Eliezer,

The issue was with whatsapp web socket was not working, here is detailed 
information about issue


Here is some information about my squid version,

Squid Cache: Version 3.5.22-20161115-r14113
Service Name: squid
configure options:  '--prefix=/usr' '--localstatedir=/var/squid' 
'--libexecdir=/lib/squid' '--srcdir=.' '--datadir=/share/squid' 
'--sysconfdir=/etc/squid' '--with-default-user=proxy' 
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' 
'--with-openssl' '--enable-ssl-crtd' '--enable-inline' '--disable-arch-native' 
'--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' 
'--enable-removal-policies=lru,heap' '--enable-delay-pools' 
'--enable-follow-x-forwarded-for' '--enable-url-rewrite-helpers=fake' 
'--enable-ecap'

My squid config file is located at, http://pastebin.com/raw/LvDxEF4x

Now the issue is whenever someone requests a page which contains web socket 
requests response is always bad request. 
Here is an example,

Request URL:wss://http://w4.web.whatsapp.com/ws
Request Method:GET
Status Code:400 Bad Request

Response Headers
#
Connection:keep-alive
Date:Sat, 17 Dec 2016 09:05:36 GMT
Transfer-Encoding:chunked
X-Cache:MISS from Proxy

Request Headers
#
Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:Upgrade
Host:http://w4.web.whatsapp.com
Origin:https://web.whatsapp.com
Pragma:no-cache
Sec-WebSocket-Extensions:permessage-deflate; client_max_window_bits
Sec-WebSocket-Key:kzrB2ZcMHDAqvjDNXnjL/w==
Sec-WebSocket-Version:13
Upgrade:websocket
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like 
Gecko) Chrome/55.0.2883.75 Safari/537.36


My question is how we can work with web socket requests in squid or if not by 
pass them squid. My squid instance is in interception mode and requests are 
intercepted at instance via iptables and forwarded to squid using below rules,

SQUIDIP=192.168.1.1

# your proxy listening port
SQUIDHTTPPORT=3128
SQUIDHTTPSPORT=3129


iptables -t nat -A PREROUTING -s $SQUIDIP -p tcp --dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 
$SQUIDHTTPPORT

iptables -t nat -A PREROUTING -s $SQUIDIP -p tcp --dport 443 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 
$SQUIDHTTPSPORT

iptables -t nat -A POSTROUTING -j MASQUERADE
iptables -t mangle -A PREROUTING -p tcp --dport $SQUIDHTTPPORT -j DROP
iptables -t mangle -A PREROUTING -p tcp --dport $SQUIDHTTPSPORT -j DROP


If anyone can help me with this it would be really awesome. Thanks for your 
support.

--

Solution to above problem was,

acl serverIsws ssl::server_name_regex ^w[0-9]+\.web\.whatsapp\.com$

acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump splice serverIsws
ssl_bump bump !serverIsws all

[ above is a feature of whatsapp which allows you to connect to 
http://web.whatsapp.com/ from browser]


now what happens at request level is following,

Request URL:wss://http://w8.web.whatsapp.com/ws
Request Method:GET
Status Code:101 Switching Protocols

--

Response Headers

Connection:Upgrade
Sec-WebSocket-Accept:Z6CC+QVdvB0cCHPbJAQMaHKL2uQ=
Upgrade:websocket

--
Request Headers

Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:Upgrade
Host:http://w8.web.whatsapp.com/

[squid-users] Squid 3.5.22 Bug when using Mimetype Detection? rep_mime_type

[Flashdown]

Hello together,

with Squid 3.5.22 I have switched from using a url-regex to Mime Type 
Detection, which seemed to work nicely until now... :/


OS: Debian Stretch 4.8.0-1-amd64 #1 SMP Debian 4.8.7-1 (2016-11-13) 
x86_64 GNU/Linux


I faced the following Situation:

When I globally deny specific mimetypes using a blockfile, then it 
performs as it should, so only mime types I defined in the block file 
are getting blocked, so far so good.


When I do an exception for a group I belong to like unblocking 
application/octet-stream, then I can download files, so the exception 
works in the first place.

acl mime_IT rep_mime_type application/octet-stream
http_reply_access allow IT mime_IT

Normally internal targets are excluded from the Proxy using Proxy 
Exception lists. But I do not get these settings automatically, so my 
browser did not contain this exception so I was able to discover the 
following behavior:


The Issue is occuring when browsing to an internal OTRS Web Server via 
FQDN (It's a web ticket system) through the proxy I get "Access Denied" 
from the Proxy on all requests. But when browsing to an online OTRS Demo 
site with the same OTRS version like this one: 
http://itsm-demo.otrs.com/otrs/index.pl then it works. When I now try 
again to access the internal OTRS Server through the proxy it works. 
That's strange, when I now force reload (CTRL+F5 in Firefox) the 
internal OTRS Ticketsystems webpage, I get the "access denied" again.


When I remove the exception from the global block list for the group I 
belong to,- here it's IT- then this issue does not occur and the website 
is accessible like it should.

So I just need to comment out these lines:
#acl mime_IT rep_mime_type application/octet-stream
#http_reply_access allow IT mime_IT

When I add text/html & application/xml to the global block exception, 
then this error does not occur anymore.
acl mime_IT rep_mime_type application/octet-stream text/html 
application/xml

http_reply_access allow IT mime_IT

So currently I can workaround the issue in 3 different ways:
1. Do not create a global mimetype block exception for groups I belong 
to
2. Browse to the start page of an Online Demo OTRS Site and then reload 
the internal Website
3. Add text/html & application/xml to my exception even if these 
Mimetypes are not part of the global block list, so they are not 
supposed to be blocked. (I just looked at the internal website and it 
just uses text/html and application/xml on the start page (Login Page) 
so I added them to the exception list for my group and it worked)


Conclusion: When having a global mime type block and unblocking a 
specific mime type for a specific group, then this group will most 
propably face issues with mime types that are not supposed to be 
blocked. So in case of errors, I need to unblock not blocked mimetypes 
,too.



My Squid config for mime type blocking:
---
## Define Default MIMETYPE ERROR Message and global block access list
acl block_mimetypes rep_mime_type "/etc/squid/mimetype_blacklist.acl"
deny_info ERR_BLOCKED_FILES block_mimetypes

# Configure Execptions
acl mime_IT rep_mime_type application/octet-stream
http_reply_access allow IT mime_IT

acl mime_SpecialGroup rep_mime_type application/octet-stream
http_reply_access allow SpecialGroup mime_SpecialGroup


#Applying Global MimeType Block
http_reply_access deny block_mimetypes
---
Squid main config:
---
http_port 8080 ssl-bump generate-host-certificates=on 
dynamic_cert_mem_cache_size=16MB cert=/*

ssl_bump splice localhost
ssl_bump splice SSL_Exclude
ssl_bump bump all
sslproxy_cert_error allow SSL_TrustedSites
sslproxy_cert_error deny all
---



Contents of mimetype_blacklist.acl:
---
##
#This is the global blocklist
# Executables: bin exe com dll class
application/x-msdownload
application/octet-stream
application/exe
application/x-exe
application/dos-exe
vms/exe
application/x-winexe
application/msdos-windows
application/x-msdos-program

# .msi
application/x-msi

# .vbs
text/vbscript
text/vbs

# Archives
# .gz
application/gzip
# .z
application/x-compress
# .gtar
application/x-gtar
# .zip
#application/zip
# .tar
application/x-tar
# .rar
application/x-rar-compressed
# .7z
application/x-7z-compressed

# .torrent
application/x-bittorrent
#
---

Squid Cache: Version 3.5.22
Service Name: squid
Debian linux
configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr' 
'--includedir=${prefix}/include' '--mandir=${prefix}/share/man' 
'--infodir=${prefix}/share/info' '--sysconfdir=/etc' 
'--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' 
'--disable-maintainer-mode' '--disable-dependency-tracking' 
'--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 

[squid-users] squid tproxy connection time out

[mr ghorbani]

hello masters
I have a problem on the squid in tproxy mode and it is that  squid
return Error "110 connection
the timeout." for all Requests on port 3129, which is related to tproxy
Of course, by eliminating the code
ip route add local 0.0.0.0/0 dev lo table 100
Problem solved, but in this case, Squid does not record any activity log.

i had attached the network topology.

In this network, 185.12.32.1 is gw for both of the client and squid. it is set
to That route the client communication to the squid and then, squid
connect it to the internet by using this gw and also redirect port 80
to 3129 and tproxy to cache the data.
The gw is connected to the Internet and also it is the bgp for these
IP addresses.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid Websocket Issue

[Hardik Dangar]

Hey Eliezer,

The issue was with whatsapp web socket was not working, here is detailed
information about issue


Here is some information about my squid version,

Squid Cache: Version 3.5.22-20161115-r14113
Service Name: squid
configure options:  '--prefix=/usr' '--localstatedir=/var/squid'
'--libexecdir=/lib/squid' '--srcdir=.' '--datadir=/share/squid'
'--sysconfdir=/etc/squid' '--with-default-user=proxy'
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
'--with-openssl' '--enable-ssl-crtd' '--enable-inline'
'--disable-arch-native' '--enable-async-io=8'
'--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap'
'--enable-delay-pools' '--enable-follow-x-forwarded-for'
'--enable-url-rewrite-helpers=fake' '--enable-ecap'

My squid config file is located at, http://pastebin.com/raw/LvDxEF4x

Now the issue is whenever someone requests a page which contains web socket
requests response is always bad request.
Here is an example,

Request URL:wss://w4.web.whatsapp.com/ws
Request Method:GET
Status Code:400 Bad Request

Response Headers
#
Connection:keep-alive
Date:Sat, 17 Dec 2016 09:05:36 GMT
Transfer-Encoding:chunked
X-Cache:MISS from Proxy

Request Headers
#
Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:Upgrade
Host:w4.web.whatsapp.com
Origin:https://web.whatsapp.com
Pragma:no-cache
Sec-WebSocket-Extensions:permessage-deflate; client_max_window_bits
Sec-WebSocket-Key:kzrB2ZcMHDAqvjDNXnjL/w==
Sec-WebSocket-Version:13
Upgrade:websocket
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/55.0.2883.75 Safari/537.36


My question is how we can work with web socket requests in squid or if not
by pass them squid. My squid instance is in interception mode and requests
are intercepted at instance via iptables and forwarded to squid using below
rules,

SQUIDIP=192.168.1.1

# your proxy listening port
SQUIDHTTPPORT=3128
SQUIDHTTPSPORT=3129


iptables -t nat -A PREROUTING -s $SQUIDIP -p tcp --dport 80 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port
$SQUIDHTTPPORT

iptables -t nat -A PREROUTING -s $SQUIDIP -p tcp --dport 443 -j ACCEPT
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port
$SQUIDHTTPSPORT

iptables -t nat -A POSTROUTING -j MASQUERADE
iptables -t mangle -A PREROUTING -p tcp --dport $SQUIDHTTPPORT -j DROP
iptables -t mangle -A PREROUTING -p tcp --dport $SQUIDHTTPSPORT -j DROP


If anyone can help me with this it would be really awesome. Thanks for your
support.

--

*Solution to above problem was,*

acl serverIsws ssl::server_name_regex ^w[0-9]+\.web\.whatsapp\.com$

acl step1 at_step SslBump1
ssl_bump peek step1
ssl_bump splice serverIsws
ssl_bump bump !serverIsws all

[ above is a feature of whatsapp which allows you to connect to
web.whatsapp.com from browser]


now what happens at request level is following,

Request URL:wss://w8.web.whatsapp.com/ws
Request Method:GET
Status Code:101 Switching Protocols

--

Response Headers

Connection:Upgrade
Sec-WebSocket-Accept:Z6CC+QVdvB0cCHPbJAQMaHKL2uQ=
Upgrade:websocket

--
Request Headers

Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:Upgrade
Host:w8.web.whatsapp.com
Origin:https://web.whatsapp.com
Pragma:no-cache
Sec-WebSocket-Extensions:permessage-deflate; client_max_window_bits
Sec-WebSocket-Key:mbCFLN/Q1KMt58t6DoQI9Q==
Sec-WebSocket-Version:13
Upgrade:websocket
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/55.0.2883.75 Safari/537.36


So basically websockets are connected as normal https request( i think this
is a very nature of Web sockets and define somewhere in web socket
standards).


Now the problem statement is,

ssl_bump bump !serverIsws all

If i remove !serverIsws then it stops working. as per alex it shoudn't
happen and its a bug most probably.


On Mon, Jan 2, 2017 at 7:17 PM, Eliezer Croitoru 
wrote:

> Can we start from 0.
> Currently when squid knows about the Connection being a one with websocket
> support it is already too late to do anything about this specific
> connection.
> The only option for now is to identify these using some ICAP service that
> will for example redirect the request after a small delay that will add the
> destination domain ip address to a bypass list.
> It’s not trivial but I have seen such implementation on ssl bump.
>
> Can you please redirect me to the specific email with the bug details?
>
> Eliezer
>
> 
> http://ngtech.co.il/lmgtfy/
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: elie...@ngtech.co.il
>
>
> From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On
> Behalf Of Hardik Dangar
> Sent: Monday, January 2, 2017 

Re: [squid-users] Squid 3.5.23-1 is available for Ubuntu 16.04 LTS (online repo ubuntu16.diladele.com)

[vinay]

Rafael,
Am facing below 2 issues.

1.I am working on squid 3.3.8 on ubuntu 14.04 , I am trying to upgrade the
squid version to higher than the default one I.e higher than 3.4 version but
am getting an error. "Unable to locate package libecap3" on giving apt-get
install libecap3 command. Any suggestions to upgrade the version ?
2. In the existing version of squid 3.3.8 am getting the below lines in
access logs
"TCP_MISS/200" and it's not caching .
Any suggestions on this to overcome the caching issue? 



--
View this message in context: 
http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-3-5-23-1-is-available-for-Ubuntu-16-04-LTS-online-repo-ubuntu16-diladele-com-tp4680948p4681025.html
Sent from the Squid - Users mailing list archive at Nabble.com.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Squid Websocket Issue

[Eliezer Croitoru]

Can we start from 0.
Currently when squid knows about the Connection being a one with websocket 
support it is already too late to do anything about this specific connection.
The only option for now is to identify these using some ICAP service that will 
for example redirect the request after a small delay that will add the 
destination domain ip address to a bypass list.
It’s not trivial but I have seen such implementation on ssl bump.

Can you please redirect me to the specific email with the bug details?

Eliezer


http://ngtech.co.il/lmgtfy/
Linux System Administrator
Mobile: +972-5-28704261
Email: elie...@ngtech.co.il


From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On Behalf 
Of Hardik Dangar
Sent: Monday, January 2, 2017 8:47 AM
To: Alex Rousskov 
Cc: Squid Users 
Subject: Re: [squid-users] Squid Websocket Issue

@amos or anyone else from dev team

Can you confirm this is intentional behavior or bug ?

On Mon, Jan 2, 2017 at 9:18 AM, Alex Rousskov 
 wrote:
On 12/27/2016 04:50 AM, Hardik Dangar wrote:

> If i remove !serverIsws somehow websockets will not work.

Then there is a bug somewhere AFAICT. It is your call whether to find
out what that bug is [while continuing to use a potentially dangerous
workaround].

Alex.


> On Tue, Dec 20, 2016 at 10:27 PM, Alex Rousskov wrote:
>
> On 12/20/2016 02:42 AM, Hardik Dangar wrote:
> > Following changes in config works and whatsapp starts working,
> >
> > acl serverIsws ssl::server_name_regex ^w[0-9]+\.web\.whatsapp\.com$
> >
> > acl step1 at_step SslBump1
> > ssl_bump peek step1
> > ssl_bump splice serverIsws
> > ssl_bump bump !serverIsws all
>
> You do not need the "!serverIsws" part because if serverIsws matches,
> then the splice rule wins, and Squid does not reach the bump rule. This
> configuration is sufficient:
>
>   ssl_bump peek step1
>   ssl_bump splice serverIsws
>   ssl_bump bump all
>
> In theory, adding "!serverIsws" does not hurt. However, negating complex
> ACLs is tricky/dangerous and should be avoided when possible.
>
> Alex.
>
>


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users