Hi, Please annwer my question, I sent you my test ________________________________ From: varnish-misc <[email protected]> on behalf of hamidreza hosseini <[email protected]> Sent: Wednesday, December 11, 2019 11:49 PM To: Guillaume Quintard <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: Varnish doesnt purge
I do all steps now again, this is the resault: STEP ONE: First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl, varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 -H "newtrack: yes" HTTP/1.1 200 OK Server: nginx Date: Thu, 12 Dec 2019 07:29:32 GMT Content-Type: text/plain Content-Length: 23521499 Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT ETag: "3a1794b0-166e8db" X-Varnish: 2 Age: 0 Via: 1.1 varnish (Varnish/6.0) Accept-Ranges: bytes Connection: keep-alive varnish cach that object and then i purge the url: varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes" * Trying 192.168.200.12... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0) > PURGE /Naserfeiz.mp4 HTTP/1.1 > Host: 192.168.200.12:8080 > User-Agent: curl/7.47.0 > Accept: */* > newtrack: yes > < HTTP/1.1 200 Purged < Date: Thu, 12 Dec 2019 07:29:16 GMT < Server: Varnish < X-Varnish: 32770 < Content-Type: text/html; charset=utf-8 < Retry-After: 5 < Content-Length: 240 < Accept-Ranges: bytes < Connection: keep-alive < { [240 bytes data] 100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k * Connection #0 to host 192.168.200.12 left intact The object purged successfully and it didnt exist in cach anymore. This is varnishlog resault: varnish-file$ sudo varnishlog -d -g request * << Request >> 2 - Begin req 1 rxreq - Timestamp Start: 1576135743.442646 0.000000 0.000000 - Timestamp Req: 1576135743.442646 0.000000 0.000000 - ReqStart 192.168.200.12 5432 a0 - ReqMethod HEAD - ReqURL /Naserfeiz.mp4 - ReqProtocol HTTP/1.1 - ReqHeader Host: 192.168.200.12:8080 - ReqHeader User-Agent: curl/7.47.0 - ReqHeader Accept: */* - ReqHeader newtrack: yes - ReqHeader X-Forwarded-For: 192.168.200.12 - VCL_call RECV - VCL_return hash - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 3 fetch - Timestamp Fetch: 1576135743.444347 0.001701 0.001701 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Server: nginx - RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT - RespHeader Content-Type: text/plain - RespHeader Content-Length: 23521499 - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT - RespHeader ETag: "3a1794b0-166e8db" - RespHeader X-Varnish: 2 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish (Varnish/6.0) - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1576135743.444382 0.001736 0.000036 - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1576135743.444474 0.001828 0.000092 - ReqAcct 112 0 112 295 0 295 - End ** << BeReq >> 3 -- Begin bereq 2 fetch -- VCL_use boot -- Timestamp Start: 1576135743.442807 0.000000 0.000000 -- BereqMethod HEAD -- BereqURL /Naserfeiz.mp4 -- BereqProtocol HTTP/1.1 -- BereqHeader Host: 192.168.200.12:8080 -- BereqHeader User-Agent: curl/7.47.0 -- BereqHeader Accept: */* -- BereqHeader newtrack: yes -- BereqHeader X-Forwarded-For: 192.168.200.12 -- BereqMethod GET -- BereqHeader Accept-Encoding: gzip -- BereqHeader X-Varnish: 3 -- VCL_call BACKEND_FETCH -- VCL_return fetch -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846 -- BackendStart 192.168.200.11 8080 -- Timestamp Bereq: 1576135743.443586 0.000779 0.000779 -- Timestamp Beresp: 1576135743.444017 0.001210 0.000431 -- BerespProtocol HTTP/1.1 -- BerespStatus 200 -- BerespReason OK -- BerespHeader Server: nginx -- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT -- BerespHeader Content-Type: text/plain -- BerespHeader Content-Length: 23521499 -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT -- BerespHeader Connection: keep-alive -- BerespHeader ETag: "3a1794b0-166e8db" -- BerespHeader Accept-Ranges: bytes -- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable -- VCL_call BACKEND_RESPONSE -- TTL VCL 86400 10 0 1576135743 cacheable -- VCL_return deliver -- Storage file s0 -- Fetch_Body 3 length stream -- BackendReuse 27 boot.swift_proxy_1 -- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818 -- Length 23521499 -- BereqAcct 181 0 181 241 23521499 23521740 -- End * << Request >> 32770 - Begin req 32769 rxreq - Timestamp Start: 1576135756.680792 0.000000 0.000000 - Timestamp Req: 1576135756.680792 0.000000 0.000000 - ReqStart 192.168.200.12 5438 a0 - ReqMethod PURGE - ReqURL /Naserfeiz.mp4 - ReqProtocol HTTP/1.1 - ReqHeader Host: 192.168.200.12:8080 - ReqHeader User-Agent: curl/7.47.0 - ReqHeader Accept: */* - ReqHeader newtrack: yes - ReqHeader X-Forwarded-For: 192.168.200.12 - VCL_call RECV - VCL_return purge - VCL_call HASH - VCL_return lookup - VCL_call PURGE - VCL_return synth - Timestamp Process: 1576135756.680982 0.000190 0.000190 - RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 32770 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespReason Purged - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - RespHeader Content-Length: 240 - Storage malloc Transient - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1576135756.681190 0.000399 0.000208 - ReqAcct 113 0 113 218 240 458 - End * << Request >> 32772 - Begin req 32771 rxreq - Timestamp Start: 1576135769.304679 0.000000 0.000000 - Timestamp Req: 1576135769.304679 0.000000 0.000000 - ReqStart 192.168.200.12 5440 a0 - ReqMethod HEAD - ReqURL /Naserfeiz.mp4 - ReqProtocol HTTP/1.1 - ReqHeader Host: 192.168.200.12:8080 - ReqHeader User-Agent: curl/7.47.0 - ReqHeader Accept: */* - ReqHeader newtrack: yes - ReqHeader X-Forwarded-For: 192.168.200.12 - VCL_call RECV - VCL_return hash - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 32773 fetch - Timestamp Fetch: 1576135769.306630 0.001951 0.001951 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Server: nginx - RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT - RespHeader Content-Type: text/plain - RespHeader Content-Length: 23521499 - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT - RespHeader ETag: "3a1794b0-166e8db" - RespHeader X-Varnish: 32772 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish (Varnish/6.0) - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1576135769.306657 0.001978 0.000027 - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1576135769.306748 0.002069 0.000091 - ReqAcct 112 0 112 299 0 299 - End ** << BeReq >> 32773 -- Begin bereq 32772 fetch -- VCL_use boot -- Timestamp Start: 1576135769.304834 0.000000 0.000000 -- BereqMethod HEAD -- BereqURL /Naserfeiz.mp4 -- BereqProtocol HTTP/1.1 -- BereqHeader Host: 192.168.200.12:8080 -- BereqHeader User-Agent: curl/7.47.0 -- BereqHeader Accept: */* -- BereqHeader newtrack: yes -- BereqHeader X-Forwarded-For: 192.168.200.12 -- BereqMethod GET -- BereqHeader Accept-Encoding: gzip -- BereqHeader X-Varnish: 32773 -- VCL_call BACKEND_FETCH -- VCL_return fetch -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846 -- BackendStart 192.168.200.11 8080 -- Timestamp Bereq: 1576135769.304993 0.000160 0.000160 -- Timestamp Beresp: 1576135769.306170 0.001336 0.001176 -- BerespProtocol HTTP/1.1 -- BerespStatus 200 -- BerespReason OK -- BerespHeader Server: nginx -- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT -- BerespHeader Content-Type: text/plain -- BerespHeader Content-Length: 23521499 -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT -- BerespHeader Connection: keep-alive -- BerespHeader ETag: "3a1794b0-166e8db" -- BerespHeader Accept-Ranges: bytes -- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable -- VCL_call BACKEND_RESPONSE -- TTL VCL 86400 10 0 1576135769 cacheable -- VCL_return deliver -- Storage file s0 -- Fetch_Body 3 length stream -- BackendReuse 27 boot.swift_proxy_1 -- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321 -- Length 23521499 -- BereqAcct 185 0 185 241 23521499 23521740 -- End ######## STEP TWO: First i restart varnish file and varnishlog then: I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server): varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 -H "newtrack: yes" HTTP/1.1 200 OK Server: nginx Date: Thu, 12 Dec 2019 07:33:05 GMT Content-Type: text/plain Content-Length: 23521499 Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT ETag: "3a1794b0-166e8db" X-Varnish: 2 Via: 1.1 varnish (Varnish/6.0) X-Varnish: 2 Age: 0 Via: 1.1 varnish-v4 Accept-Ranges: bytes Connection: keep-alive First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached: varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes" * Trying 192.168.200.13... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0) > PURGE /Naserfeiz.mp4 HTTP/1.1 > Host: 192.168.200.13 > User-Agent: curl/7.47.0 > Accept: */* > newtrack: yes > < HTTP/1.1 200 Purged < Date: Thu, 12 Dec 2019 07:33:46 GMT < Server: Varnish < X-Varnish: 32770 < Content-Type: text/html; charset=utf-8 < Retry-After: 5 < Content-Length: 240 < Accept-Ranges: bytes < Connection: keep-alive < { [240 bytes data] 100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k * Connection #0 to host 192.168.200.13 left intact Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!: varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes" * Trying 192.168.200.12... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0) > PURGE /Naserfeiz.mp4 HTTP/1.1 > Host: 192.168.200.12:8080 > User-Agent: curl/7.47.0 > Accept: */* > newtrack: yes > < HTTP/1.1 200 Purged < Date: Thu, 12 Dec 2019 07:33:39 GMT < Server: Varnish < X-Varnish: 5 < Content-Type: text/html; charset=utf-8 < Retry-After: 5 < Content-Length: 236 < Accept-Ranges: bytes < Connection: keep-alive < { [236 bytes data] 100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k * Connection #0 to host 192.168.200.12 left intact This is varnishlog resault in varnish file: varnish-file:~$ sudo varnishlog -d -g request * << Request >> 2 - Begin req 1 rxreq - Timestamp Start: 1576135956.460768 0.000000 0.000000 - Timestamp Req: 1576135956.460768 0.000000 0.000000 - ReqStart 192.168.200.13 22300 a0 - ReqMethod GET - ReqURL /Naserfeiz.mp4 - ReqProtocol HTTP/1.1 - ReqHeader Host: 192.168.200.13 - ReqHeader User-Agent: curl/7.47.0 - ReqHeader Accept: */* - ReqHeader newtrack: yes - ReqHeader X-Forwarded-For: 192.168.200.13 - ReqHeader Accept-Encoding: gzip - ReqHeader X-Varnish: 3 - ReqUnset X-Forwarded-For: 192.168.200.13 - ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13 - VCL_call RECV - VCL_return hash - VCL_call HASH - VCL_return lookup - VCL_call MISS - VCL_return fetch - Link bereq 3 fetch - Timestamp Fetch: 1576135956.463065 0.002297 0.002297 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Server: nginx - RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT - RespHeader Content-Type: text/plain - RespHeader Content-Length: 23521499 - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT - RespHeader ETag: "3a1794b0-166e8db" - RespHeader X-Varnish: 2 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish (Varnish/6.0) - VCL_call DELIVER - VCL_return deliver - Timestamp Process: 1576135956.463114 0.002346 0.000049 - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1576135956.758920 0.298152 0.295806 - ReqAcct 176 0 176 295 23521499 23521794 - End ** << BeReq >> 3 -- Begin bereq 2 fetch -- VCL_use boot -- Timestamp Start: 1576135956.461099 0.000000 0.000000 -- BereqMethod GET -- BereqURL /Naserfeiz.mp4 -- BereqProtocol HTTP/1.1 -- BereqHeader Host: 192.168.200.13 -- BereqHeader User-Agent: curl/7.47.0 -- BereqHeader Accept: */* -- BereqHeader newtrack: yes -- BereqHeader Accept-Encoding: gzip -- BereqHeader X-Varnish: 3 -- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13 -- BereqHeader X-Varnish: 3 -- VCL_call BACKEND_FETCH -- VCL_return fetch -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860 -- BackendStart 192.168.200.11 8080 -- Timestamp Bereq: 1576135956.461955 0.000856 0.000856 -- Timestamp Beresp: 1576135956.462607 0.001508 0.000652 -- BerespProtocol HTTP/1.1 -- BerespStatus 200 -- BerespReason OK -- BerespHeader Server: nginx -- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT -- BerespHeader Content-Type: text/plain -- BerespHeader Content-Length: 23521499 -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT -- BerespHeader Connection: keep-alive -- BerespHeader ETag: "3a1794b0-166e8db" -- BerespHeader Accept-Ranges: bytes -- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable -- VCL_call BACKEND_RESPONSE -- TTL VCL 86400 10 0 1576135956 cacheable -- VCL_return deliver -- Storage file s0 -- Fetch_Body 3 length stream -- BackendReuse 27 boot.swift_proxy_1 -- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349 -- Length 23521499 -- BereqAcct 206 0 206 241 23521499 23521740 -- End * << Request >> 5 - Begin req 4 rxreq - Timestamp Start: 1576136019.230666 0.000000 0.000000 - Timestamp Req: 1576136019.230666 0.000000 0.000000 - ReqStart 192.168.200.12 5452 a0 - ReqMethod PURGE - ReqURL /Naserfeiz.mp4 - ReqProtocol HTTP/1.1 - ReqHeader Host: 192.168.200.12:8080 - ReqHeader User-Agent: curl/7.47.0 - ReqHeader Accept: */* - ReqHeader newtrack: yes - ReqHeader X-Forwarded-For: 192.168.200.12 - VCL_call RECV - VCL_return purge - VCL_call HASH - VCL_return lookup - VCL_call PURGE - VCL_return synth - Timestamp Process: 1576136019.230863 0.000198 0.000198 - RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 5 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespReason Purged - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - RespHeader Content-Length: 236 - Storage malloc Transient - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1576136019.231138 0.000473 0.000275 - ReqAcct 113 0 113 214 236 450 - End ######## PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file Please do this scenario for yourself ________________________________ From: Guillaume Quintard <[email protected]> Sent: Wednesday, December 11, 2019 9:44 AM To: hamidreza hosseini <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: Varnish doesnt purge Yes it does, we have already proved it./ Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat. You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that. I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help. -- Guillaume Quintard On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <[email protected]<mailto:[email protected]>> wrote: Hi, please help me to solve this problem, this is the link that i explaine my question completely: https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html _______________________________________________ varnish-misc mailing list [email protected]<mailto:[email protected]> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
