Re: [squid-users] Sudden but sustained high bandwidth usage
OK it's pretty simple to reproduce on any machine what so ever on 3.5.15-2. open two terminals on two machines more or less. Then run on one the next command watch -n 0.2 "http_proxy=http://IP_OP_PROXY:3128/ curl --silent --range 20-40 http://ngtech.co.il/squid/videos/sosp2011_27.mp4 | wc -c" And on the other one: "http_proxy=http://IP_OP_PROXY:3128/ wget http://ngtech.co.il/squid/videos/sosp2011_27.mp4 You will see that when the download is finished and at the same time the download of the partial content is still running, the SWAPFAIL_MISS happens. If for some reason it won't work for someone(I doubt that there is faster then full env RAM only (hypervisor+vms+disks) so lower the 0.2 to 0.1 . Please file a bug report with the above details, Eliezer On 11/03/2016 15:55, Heiler Bemerguy wrote: Hi Eliezer, We usually don't restart it ever. Only recently I've been restarting it because of these issues. The shutdown_lifetime is set to 5 seconds only. We are still getting SWAPFAIL_MISS without any apparent reason, and if it is for a RANGE request, it would multiply it to many parallel connections I'd like to track it down but I'm afraid I don't have the knowledge to do it... root@proxy:/var/log/squid# tail -f access.log |grep SWAPF 1457703376.593383 10.88.100.100 TCP_SWAPFAIL_MISS/206 533 GET http://vdownloader.com/wp-content/uploads/multiwebsite-570x321.png - HIER_DIRECT/104.25.245.28 image/png 1457703376.942726 10.88.100.100 TCP_SWAPFAIL_MISS/206 530 GET http://vdownloader.com/wp-content/uploads/social-media-share-570x321.jpg - HIER_DIRECT/104.25.244.28 image/jpeg 1457703376.964746 10.88.100.100 TCP_SWAPFAIL_MISS/206 529 GET http://vdownloader.com/wp-content/uploads/Cnet-logo.png - HIER_DIRECT/104.25.244.28 image/png 1457703540.685 106669 10.101.1.50 TCP_SWAPFAIL_MISS/200 3342325 GET http://www.rarlab.com/rar/wrar531br.exe - HIER_DIRECT/5.135.104.98 application/octet-stream 1457703631.055 4088 10.101.1.130 TCP_SWAPFAIL_MISS/206 33062 GET http://www.cetapnet.com.br/arquivos_cetap/arquivos/pma_001_2015_anexo_03.pdf - HIER_DIRECT/200.219.214.204 application/pdf 1457703637.471 6407 10.101.1.130 TCP_SWAPFAIL_MISS/206 352287 GET http://www.cetapnet.com.br/arquivos_cetap/arquivos/pma_001_2015_anexo_03.pdf - HIER_DIRECT/200.219.214.204 application/pdf 1457703755.673262 10.72.0.24 TCP_SWAPFAIL_MISS/206 21736 GET http://ciac.ufpa.br/phocadownload/EDITAL_018-2016-RET-HOM_REC_2-CH-SISU-2016.pdf - HIER_DIRECT/200.239.64.160 application/pdf 1457703755.712 35 10.72.0.24 TCP_SWAPFAIL_MISS/206 65938 GET http://ciac.ufpa.br/phocadownload/EDITAL_018-2016-RET-HOM_REC_2-CH-SISU-2016.pdf - HIER_DIRECT/200.239.64.160 application/pdf Best Regards, ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] Sudden but sustained high bandwidth usage
Hey,
Thanks for the debug!.
I do not know the exact reason but I can say for sure that it's not the
NetAPP or any other OS level issue since the AUFS\UFS cache_dir works
fine in the same system and in a similar situation.
I will try to replicate it locally.
I do understand the issue and I will try my best to see what can be done
from my side.
For now I will try to replicate the issue on a RAM only
system(OS,CACHE,DISK,env..).
I think the right way now is to file a BUG in the bugzilla and continue
following here and there since it's a major bug when comparing UFS\AUFS
to ROCK.
If you understand that a patch\fix for such an issue is not a tiny task
then I and you are on the same ship.
(I will try to test today but I cannot promise I will find it in such a
hurry)
All The Bests,
Eliezer
* http://bugs.squid-cache.org/enter_bug.cgi
On 11/03/2016 18:25, Heiler Bemerguy wrote:
I managed to track down with GDB one of these swapfails...
Breakpoint 1, clientReplyContext::cacheHit (this=0x33bff58, result=...)
at client_side_reply.cc:471
471 http->logType = LOG_TCP_SWAPFAIL_MISS;
(gdb) l
466 debugs(88, 3, "clientCacheHit: request aborted");
467 return;
468 } else if (result.flags.error) {
469 /* swap in failure */
470 debugs(88, 3, "clientCacheHit: swapin failure for " <<
http->uri);
471 http->logType = LOG_TCP_SWAPFAIL_MISS;
472 removeClientStoreReference(, http);
473 processMiss();
474 return;
475 }
(gdb) p *this
$2 = { = {_vptr.Lock = 0x84e668, count_ = 1}, =
{_vptr.StoreClient = 0x84e638}, purgeStatus = Http::scNone,
lookingforstore = 5, http = 0x1eadb398, headers_sz = 0, sc =
0x3034028, tempBuffer = {flags = {error = 0}, length = 0, offset = 0,
data = 0x0}, old_reqsize = 0, reqsize = 210, reqofs = 0, tempbuf =
'\000' , flags = {storelogiccomplete = 0,
complete = 0, headersSent = false}, ourNode = 0xaa32198,
holdingBuffer = {flags = {error = 0}, length = 0, offset = 0, data = 0x0},
reply = 0x0, old_entry = 0x0, old_sc = 0x0, deleting = false, static
CBDATA_clientReplyContext = 24}
(gdb) p result
$3 = {flags = {error = 1}, length = 0, offset = 0,
data = 0x9ef18d8 "HTTP/1.1 200 Internal marker object\r\nServer:
squid\r\nMime-Version: 1.0\r\nDate: Fri, 11 Mar 2016 15:57:32
GMT\r\nContent-Type: x-squid-internal/vary\r\nExpires: Sat, 12 Mar 2016
19:44:12 GMT\r\nVary: Accept-En"...}
It already came to client_side_reply.cc with "ERROR" set to 1...
(gdb) bt
#0 clientReplyContext::cacheHit (this=0x33bff58, result=...) at
client_side_reply.cc:471
#1 0x0064dcbd in store_client::callback (this=0x3034028,
sz=, error=) at store_client.cc:130
#2 0x0064e6c1 in store_client::startSwapin
(this=this@entry=0x3034028) at store_client.cc:382
#3 0x006501d3 in store_client::doCopy
(this=this@entry=0x3034028, anEntry=anEntry@entry=0xab07eb0) at
store_client.cc:359
#4 0x0065033c in storeClientCopy2 (e=0xab07eb0,
sc=sc@entry=0x3034028) at store_client.cc:315
#5 0x00650db9 in storeClientCopy2 (sc=0x3034028, e=) at store_client.cc:281
#6 store_client::copy (this=0x3034028, anEntry=0xab07eb0, copyRequest=...,
callback_fn=0x55b250 , data=0x33bff58) at store_client.cc:232
#7 0x00558111 in clientReplyContext::doGetMoreData
(this=this@entry=0x33bff58) at client_side_reply.cc:1799
#8 0x00558422 in clientReplyContext::identifyFoundObject
(this=0x33bff58, newEntry=) at client_side_reply.cc:1649
#9 0x0055a8b8 in clientReplyContext::cacheHit (this=0x33bff58,
result=...) at client_side_reply.cc:525
#10 0x0064dcbd in store_client::callback (this=0x51d5b08,
sz=, error=) at store_client.cc:130
#11 0x0064e1ba in store_client::readBody
(this=this@entry=0x51d5b08, buf=, len=len@entry=210) at
store_client.cc:497
#12 0x0064f76b in store_client::readHeader (this=0x51d5b08,
buf=, len=) at store_client.cc:611
#13 0x00732790 in Rock::IoState::callReaderBack (this=,
buf=0x9ef18d8 "HTTP/1.1 200 Internal marker object\r\nServer:
squid\r\nMime-Version: 1.0\r\nDate: Fri, 11 Mar 2016 15:57:32
GMT\r\nContent-Type: x-squid-internal/vary\r\nExpires: Sat, 12 Mar 2016
19:44:12 GMT\r\nVary: Accept-En"..., rlen=428) at rock/RockIoState.cc:143
#14 0x00726ec1 in Rock::SwapDir::readCompleted (this=, buf=, rlen=428, errflag=0, r=...)
at rock/RockSwapDir.cc:822
#15 0x006f22c0 in IpcIoFile::readCompleted (this=, readRequest=0x5417008, response=)
at DiskIO/IpcIo/IpcIoFile.cc:255
#16 0x006f6bcc in IpcIoFile::handleResponse (this=, ipcIo=...) at DiskIO/IpcIo/IpcIoFile.cc:462
#17 0x006f6fde in IpcIoFile::HandleResponses
(when=when@entry=0x851324 "after notification") at
DiskIO/IpcIo/IpcIoFile.cc:449
...help!
___
squid-users mailing list
Re: [squid-users] HTTPS interception and filtering?
On Sun, 2016-03-13 at 00:09 +1100, Tim Bates wrote: > Is it possible to do this: > > * Intercept HTTPS and send it via Squid? > * Apply ACLs to the intercepted HTTPS traffic based on host/domain name? > * Not change any configuration on clients? > > Should I keep researching how this peeking and bumping and splicing and > such works, or is it impossible? > > TB > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users Search for my previous posts...I've posted full configs on how to do exactly this. James ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] HTTPS interception and filtering?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 http://wiki.squid-cache.org/ConfigExamples/Intercept http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit http://wiki.squid-cache.org/SquidFaq/SquidAcl 12.03.16 19:09, Tim Bates пишет: > Is it possible to do this: > > * Intercept HTTPS and send it via Squid? > * Apply ACLs to the intercepted HTTPS traffic based on host/domain name? > * Not change any configuration on clients? > > Should I keep researching how this peeking and bumping and splicing and such works, or is it impossible? > > TB > ___ > squid-users mailing list > squid-users@lists.squid-cache.org > http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJW5BxOAAoJENNXIZxhPexG7U4IAMnDDnd9CewdJQMpANFPJv5v kX7uR69qDhvmhFVlLNO5TceYWOfh9+Cp36kzS9sWhhKJg1stUkqCUINz1/4t41Ym rfBT5jWAjkwSI20EyyBHJo/L3u1NnVQk2ie+e0ZaGfo6ta6xeZovQEv2TupDMtPp Y6lYBLMH14aNu8rNH7YxAAkXhSxdLcKZOE62+9Aezk3qaI8IQXchoVTBVvWCYAkH AkbnAJJPt/FYFW935CDlum8zAZ4LhuC5t//7afIw/Xz9GlbbRk8xbNLzlSjSv7lP 9eDmXvkVbH89Xp/bpIUiXChoscmc/Vqcv+IymGfVxo3vsZRURdrAAKSc4Exm2nY= =zEUk -END PGP SIGNATURE- 0x613DEC46.asc Description: application/pgp-keys ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
[squid-users] HTTPS interception and filtering?
Is it possible to do this: * Intercept HTTPS and send it via Squid? * Apply ACLs to the intercepted HTTPS traffic based on host/domain name? * Not change any configuration on clients? Should I keep researching how this peeking and bumping and splicing and such works, or is it impossible? TB ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
