Re: [squid-users] Delay pools in squid4 not working with https
On 08/07/2018 09:20 AM, Julian Perconti wrote: > Yesterday i have compiled squid 4.2. > > When site is spliced delay_pools still does not working. > > Any news? The latest information and suggestions I have is at http://lists.squid-cache.org/pipermail/squid-users/2018-July/018636.html Alex. >> -Mensaje original- >> De: squid-users En nombre de >> Eliezer Croitoru >> Enviado el: miércoles, 18 de julio de 2018 13:47 >> Para: squid-users@lists.squid-cache.org >> Asunto: Re: [squid-users] Delay pools in squid4 not working with https >> >> Just to mention QUIC related wiki links: >> - https://wiki.squid- >> cache.org/KnowledgeBase/Block%20QUIC%20protocol?highlight=%28QUIC% >> 29 >> - https://wiki.squid- >> cache.org/ConfigExamples/Intercept/CiscoIOSv15Wccp2?highlight=%28QUIC >> %29#QUIC.2FSPDY_protocol_blocking >> >> Eliezer >> >> >> Eliezer Croitoru >> Linux System Administrator >> Mobile: +972-5-28704261 >> Email: elie...@ngtech.co.il >> >> >> >> -Original Message- >> From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On >> Behalf Of Amos Jeffries >> Sent: Wednesday, July 11, 2018 12:35 AM >> To: squid-users@lists.squid-cache.org >> Subject: Re: [squid-users] Delay pools in squid4 not working with https >> >> On 11/07/18 07:50, Paolo Marzari wrote: >>> My home server just updated from 3.5.27, everything is working fine, >>> but delay pools seems broken to me. >>> I capped some devices to 240kb/s and tried to download a debian ISO >>> with one of them...all good, 240kb/s. >>> Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed. >>> >>> So I tried youtube videos, no cap at all, same problem with facebook. >>> Revert to 3.5.27 and delays works again with every type of traffic. >>> >>> I think there's something wrong with https traffic. >>> >> >> a) is it actually HTTPS traffic? >> >> b) are the bytes going through the proxy 2.2Mbps or 240kbps ? >> >> I ask because Google/YouTube and Facebook are services using HTTP/2 with >> high compression features as much as possible. So while the proxy is set to >> transfer X bytes per second, when hidden inside "HTTPS" those X bytes may >> show up as 90*X bytes of traffic when decompressed by a Browser. >> >> Or the transfer may be QUIC protocol, completely bypassing the HTTP the >> proxy is counting. >> >> Amos >> ___ >> squid-users mailing list >> squid-users@lists.squid-cache.org >> http://lists.squid-cache.org/listinfo/squid-users >> >> ___ >> squid-users mailing list >> squid-users@lists.squid-cache.org >> http://lists.squid-cache.org/listinfo/squid-users > > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users > ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
Just to mention QUIC related wiki links: - https://wiki.squid-cache.org/KnowledgeBase/Block%20QUIC%20protocol?highlight=%28QUIC%29 - https://wiki.squid-cache.org/ConfigExamples/Intercept/CiscoIOSv15Wccp2?highlight=%28QUIC%29#QUIC.2FSPDY_protocol_blocking Eliezer Eliezer Croitoru Linux System Administrator Mobile: +972-5-28704261 Email: elie...@ngtech.co.il -Original Message- From: squid-users [mailto:squid-users-boun...@lists.squid-cache.org] On Behalf Of Amos Jeffries Sent: Wednesday, July 11, 2018 12:35 AM To: squid-users@lists.squid-cache.org Subject: Re: [squid-users] Delay pools in squid4 not working with https On 11/07/18 07:50, Paolo Marzari wrote: > My home server just updated from 3.5.27, everything is working fine, > but delay pools seems broken to me. > I capped some devices to 240kb/s and tried to download a debian ISO > with one of them...all good, 240kb/s. > Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed. > > So I tried youtube videos, no cap at all, same problem with facebook. > Revert to 3.5.27 and delays works again with every type of traffic. > > I think there's something wrong with https traffic. > a) is it actually HTTPS traffic? b) are the bytes going through the proxy 2.2Mbps or 240kbps ? I ask because Google/YouTube and Facebook are services using HTTP/2 with high compression features as much as possible. So while the proxy is set to transfer X bytes per second, when hidden inside "HTTPS" those X bytes may show up as 90*X bytes of traffic when decompressed by a Browser. Or the transfer may be QUIC protocol, completely bypassing the HTTP the proxy is counting. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
> -Mensaje original- > De: Julian Perconti [mailto:vh1...@yahoo.com.ar] > Enviado el: jueves, 12 de julio de 2018 21:24 > Para: 'squid-users@lists.squid-cache.org' cache.org> > Asunto: RE: [squid-users] Delay pools in squid4 not working with https > > > -Mensaje original- > > De: Alex Rousskov [mailto:rouss...@measurement-factory.com] > > Enviado el: jueves, 12 de julio de 2018 21:20 > > Para: Julian Perconti ; squid-users@lists.squid- > > cache.org > > Asunto: Re: [squid-users] Delay pools in squid4 not working with https > > > > On 07/12/2018 06:16 PM, Julian Perconti wrote: > > >> De: Alex Rousskov > > >> If you start splicing/tunneling, it will probably stop working. > > > > > > > Ok, but is not is supposed that this is the normal behaviour? > > > > > > No, Squid should apply delay pools to all traffic. Ok, I did not know that.. > > OK I Will splice https://speed.hetzner.de/ and then tell You what happened > with delay pool. > > An important thing, the delay_pool cfg that Paolo has is pretty complexthan > mine. Confirmed. Splicing.. speed.hetzner.de TCP_TUNNEL/200 4452 CONNECT 88.198.248.254:443 - ORIGINAL_DST/88.198.248.254 - The delay_pool does not work. Download speed never goes down. delay_pool class 2 cfg: delay_pools 1 delay_class 1 2 delay_access 1 allow all delay_parameters 1 -1/-1 10/104857600 Version: Squid Cache: Version 4.1 Service Name: squid This binary uses OpenSSL 1.1.0f 25 May 2017. For legal restrictions on distribution see https://www.openssl.org/source/license.html > > > > > > > > I mean, TCP_TUNNEL = squid forward, so squid can not do nothing > > > about > > the spliced connection. > > > > > > Squid knows how many bytes it is forwarding, and that is all Squid > > needs to know to shape traffic. > > > > Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
> -Mensaje original- > De: Alex Rousskov [mailto:rouss...@measurement-factory.com] > Enviado el: jueves, 12 de julio de 2018 21:20 > Para: Julian Perconti ; squid-users@lists.squid- > cache.org > Asunto: Re: [squid-users] Delay pools in squid4 not working with https > > On 07/12/2018 06:16 PM, Julian Perconti wrote: > >> De: Alex Rousskov > >> If you start splicing/tunneling, it will probably stop working. > > > > Ok, but is not is supposed that this is the normal behaviour? > > > No, Squid should apply delay pools to all traffic. OK I Will splice https://speed.hetzner.de/ and then tell You what happened with delay pool. An important thing, the delay_pool cfg that Paolo has is pretty complexthan mine. > > > > I mean, TCP_TUNNEL = squid forward, so squid can not do nothing about > the spliced connection. > > > Squid knows how many bytes it is forwarding, and that is all Squid needs to > know to shape traffic. > > Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
On 07/12/2018 06:16 PM, Julian Perconti wrote: >> De: Alex Rousskov >> If you start splicing/tunneling, it will probably stop working. > Ok, but is not is supposed that this is the normal behaviour? No, Squid should apply delay pools to all traffic. > I mean, TCP_TUNNEL = squid forward, so squid can not do nothing about the > spliced connection. Squid knows how many bytes it is forwarding, and that is all Squid needs to know to shape traffic. Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
> -Mensaje original- > De: Alex Rousskov [mailto:rouss...@measurement-factory.com] > Enviado el: jueves, 12 de julio de 2018 21:03 > Para: Julian Perconti ; squid-users@lists.squid- > cache.org > Asunto: Re: [squid-users] Delay pools in squid4 not working with https > > On 07/12/2018 05:42 PM, Julian Perconti wrote: > >> De: Alex Rousskov > >> On 07/12/2018 05:19 PM, Julian Perconti wrote: > >> > >>> From my side, the tests were done with full SSL-Bump; downloading a > >>> file from: https://speed.hetzner.de/ > >>> > >>> No splice. > > > >> My "not working" statement was specific to tunneling code. When Squid > >> bumps, it does not tunnel, so your tests did not tickle the broken code. > >> We do not yet know whether prazola is bumping HTTPS traffic. > >> > >> Tunneling happens when handling CONNECT requests without SslBump > and > >> when splicing TLS traffic with SslBump. > > > > My delay_pool cfg is working. > > Yes, I understand. I do not think anybody has claimed that your config should > not be working. The only claim was that delay pools do not work when Squid > tunnels traffic. Your Squid does not tunnel traffic. > > > > Without splice/tunneling the connection. > > ... and that is why it is working. If you start splicing/tunneling, it will > probably > stop working. Ok, but is not is supposed that this is the normal behaviour? I mean, TCP_TUNNEL = squid forward, so squid can not do nothing about the spliced connection. I don't I am just a squid user... and BTW new in squid SSL intercepts. > > > Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
On 07/12/2018 05:42 PM, Julian Perconti wrote: >> De: Alex Rousskov >> On 07/12/2018 05:19 PM, Julian Perconti wrote: >> >>> From my side, the tests were done with full SSL-Bump; downloading a >>> file from: https://speed.hetzner.de/ >>> >>> No splice. >> My "not working" statement was specific to tunneling code. When Squid >> bumps, it does not tunnel, so your tests did not tickle the broken code. >> We do not yet know whether prazola is bumping HTTPS traffic. >> >> Tunneling happens when handling CONNECT requests without SslBump and >> when splicing TLS traffic with SslBump. > My delay_pool cfg is working. Yes, I understand. I do not think anybody has claimed that your config should not be working. The only claim was that delay pools do not work when Squid tunnels traffic. Your Squid does not tunnel traffic. > Without splice/tunneling the connection. ... and that is why it is working. If you start splicing/tunneling, it will probably stop working. Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
> De: Alex Rousskov [mailto:rouss...@measurement-factory.com] > Enviado el: jueves, 12 de julio de 2018 20:31 > Para: Julian Perconti ; squid-users@lists.squid- > cache.org > Asunto: Re: [squid-users] Delay pools in squid4 not working with https > > On 07/12/2018 05:19 PM, Julian Perconti wrote: > > > From my side, the tests were done with full SSL-Bump; downloading a > > file from: https://speed.hetzner.de/ > > > > No splice. > > My "not working" statement was specific to tunneling code. When Squid > bumps, it does not tunnel, so your tests did not tickle the broken code. > We do not yet know whether prazola is bumping HTTPS traffic. > > Tunneling happens when handling CONNECT requests without SslBump and > when splicing TLS traffic with SslBump. > > Alex. My delay_pool cfg is working. Without splice/tunneling the connection. When I download a file from https://speed.hetzner.de/ with the https prefix in the URL downloaded file (without splice anything), the delay slows down the download once the limit is reached. May be I missunderstood something. Regards ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
On 07/12/2018 05:19 PM, Julian Perconti wrote: > From my side, the tests were done with full SSL-Bump; downloading a file > from: https://speed.hetzner.de/ > > No splice. My "not working" statement was specific to tunneling code. When Squid bumps, it does not tunnel, so your tests did not tickle the broken code. We do not yet know whether prazola is bumping HTTPS traffic. Tunneling happens when handling CONNECT requests without SslBump and when splicing TLS traffic with SslBump. Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
On 12/07/18 11:39, Julian Perconti wrote: >>> >>> El martes, 10 de julio de 2018 18:57:43 -03, Alex Rousskov >>> escribió: >>> >>> >>> On 07/10/2018 01:50 PM, Paolo Marzari wrote: My home server just updated from 3.5.27, everything is working fine, but delay pools seems broken to me. >>> Revert to 3.5.27 and delays works again with every type of traffic. I think there's something wrong with https traffic. >>> >>> You are probably right. A few days ago, while working on an unrelated >>> project, we have found a bug in delay pools support for tunneled https >>> traffic. That support was probably broken by v4 commit 6b2b6cf. We have >>> not tested v3.5, so I can only confirm that v4 and v5 are broken. >>> >>> The bug will be fixed as a side effect of "peering support for SslBump" >>> changes that should be ready for the official review soon. If you would >>> like to test our unofficial branch, the code is available at >>> https://github.com/measurement-factory/squid/tree/SQUID-360-peering-for-SslBump >>> >>> >>> HTH, >>> >>> Alex. >>> >>> ___ >>> squid-users mailing list >>> squid-users@lists.squid-cache.org >>> http://lists.squid-cache.org/listinfo/squid-users > > I can confirm that delay_pools works fine both http and https protocols in > squid 4 running debian 9 > > Squid Cache: Version 4.1 When I looked at the code for Paolos report I found there to be a difference between SSL-Bumped and non-Bumped traffic. This hints to me that these opposite reports may due to how the traffic is being handled. So Julian, Paolo; if you don't mind can you please say whether you are using SSL-Bump in your tests and if so whether the test traffic got splice'd, bump'ed or no SSL-Bump feature use at all ? There might also still be bugs specific to pool types. We have had a few in the past that I'm not sure if ever got fixed. Though Paolo's mention that 3.5 worked okay hints that its probably not those exact issues. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
>> >> El martes, 10 de julio de 2018 18:57:43 -03, Alex Rousskov >> escribió: >> >> >> On 07/10/2018 01:50 PM, Paolo Marzari wrote: >>> My home server just updated from 3.5.27, everything is working fine, but >>> delay pools seems broken to me. >> >>> Revert to 3.5.27 and delays works again with every type of traffic. >>> >>> I think there's something wrong with https traffic. >> >> You are probably right. A few days ago, while working on an unrelated >> project, we have found a bug in delay pools support for tunneled https >> traffic. That support was probably broken by v4 commit 6b2b6cf. We have >> not tested v3.5, so I can only confirm that v4 and v5 are broken. >> >> The bug will be fixed as a side effect of "peering support for SslBump" >> changes that should be ready for the official review soon. If you would >> like to test our unofficial branch, the code is available at >> https://github.com/measurement-factory/squid/tree/SQUID-360-peering-for-SslBump >> >> >> HTH, >> >> Alex. >> >> ___ >> squid-users mailing list >> squid-users@lists.squid-cache.org >> http://lists.squid-cache.org/listinfo/squid-users I can confirm that delay_pools works fine both http and https protocols in squid 4 running debian 9 Squid Cache: Version 4.1 Service Name: squid Here the cfg: delay_pools 1 delay_class 1 2 delay_access 1 allow all delay_parameters 1 -1/-1 10/104857600 # ~100KBs/~100MB delay_initial_bucket_level 50 Regards ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
On 07/10/2018 01:50 PM, Paolo Marzari wrote: > My home server just updated from 3.5.27, everything is working fine, but > delay pools seems broken to me. > Revert to 3.5.27 and delays works again with every type of traffic. > > I think there's something wrong with https traffic. You are probably right. A few days ago, while working on an unrelated project, we have found a bug in delay pools support for tunneled https traffic. That support was probably broken by v4 commit 6b2b6cf. We have not tested v3.5, so I can only confirm that v4 and v5 are broken. The bug will be fixed as a side effect of "peering support for SslBump" changes that should be ready for the official review soon. If you would like to test our unofficial branch, the code is available at https://github.com/measurement-factory/squid/tree/SQUID-360-peering-for-SslBump HTH, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
A fast check with nbwmon shows 2.2Mbps when using squid 4.1. -- Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Delay pools in squid4 not working with https
On 11/07/18 07:50, Paolo Marzari wrote: > My home server just updated from 3.5.27, everything is working fine, but > delay pools seems broken to me. > I capped some devices to 240kb/s and tried to download a debian ISO with > one of them...all good, 240kb/s. > Then I tried a speed test, results = 2.2mb/s, that's the whole ADSL speed. > > So I tried youtube videos, no cap at all, same problem with facebook. > Revert to 3.5.27 and delays works again with every type of traffic. > > I think there's something wrong with https traffic. > a) is it actually HTTPS traffic? b) are the bytes going through the proxy 2.2Mbps or 240kbps ? I ask because Google/YouTube and Facebook are services using HTTP/2 with high compression features as much as possible. So while the proxy is set to transfer X bytes per second, when hidden inside "HTTPS" those X bytes may show up as 90*X bytes of traffic when decompressed by a Browser. Or the transfer may be QUIC protocol, completely bypassing the HTTP the proxy is counting. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
