when you purge, varnish creates a synthetic response, and doesn't go to the backend, so if you wish to purge that one too, you need to send a request to it too.
-- Guillaume Quintard On Mon, Dec 9, 2019 at 4:08 AM hamidreza hosseini <[email protected]> wrote: > I test it again and in this test the fron server was purged but the > backend didnt purge > what is the problem? > ------------------------------ > *From:* varnish-misc <varnish-misc-bounces+hrhosseini= > [email protected]> on behalf of hamidreza hosseini < > [email protected]> > *Sent:* Monday, December 9, 2019 3:34 AM > *To:* Guillaume Quintard <[email protected]> > *Cc:* [email protected] <[email protected]> > *Subject:* Re: Varnish doesnt PURGE urls > > this is my test servers, I purge with ip in production but steel it doesnt > work! > ------------------------------ > *From:* Guillaume Quintard <[email protected]> > *Sent:* Monday, December 9, 2019 12:31 PM > *To:* hamidreza hosseini <[email protected]> > *Cc:* shafeeque aslam <[email protected]>; > [email protected] <[email protected]> > *Subject:* Re: Varnish doesnt PURGE urls > > Purge uses the hash of the objects to clear them. In turn, the hash, by > default depends on the host of the requests that created the object. > > So, when you purge using localhost/some/path.mp4, you dont purge the > object referring to my.website.com/some/path.mp4 > > -- > Guillaume Quintard > > > On Mon, Dec 9, 2019 at 3:24 AM hamidreza hosseini <[email protected]> > wrote: > > This is purge to another server: > > $curl -v -k -X PURGE http://localhost:8080/Naserfeiz.mp4 > > * Trying ::1... > * Connected to localhost (::1) port 8080 (#0) > > PURGE /Naserfeiz.mp4 HTTP/1.1 > > Host: localhost:8080 > > User-Agent: curl/7.47.0 > > Accept: */* > > > < HTTP/1.1 200 Purged > < Date: Mon, 09 Dec 2019 11:20:52 GMT > < Server: Varnish > < X-Varnish: 10 > < Content-Type: text/html; charset=utf-8 > < Retry-After: 5 > < Content-Length: 237 > < Accept-Ranges: bytes > < Connection: keep-alive > < > <!DOCTYPE html> > <html> > <head> > <title>200 Purged</title> > </head> > <body> > <h1>Error 200 Purged</h1> > <p>Purged</p> > <h3>Guru Meditation:</h3> > <p>XID: 10</p> > <hr> > <p>Varnish cache server</p> > </body> > </html> > * Connection #0 to host localhost left intact > > And varnishlog resault: > > $ sudo varnishlog > > * << Request >> 32773 > - Begin req 32772 rxreq > - Timestamp Start: 1575890593.662132 0.000000 0.000000 > - Timestamp Req: 1575890593.662132 0.000000 0.000000 > - ReqStart ::1 63184 > - ReqMethod PURGE > - ReqURL /Naserfeiz.mp4 > - ReqProtocol HTTP/1.1 > - ReqHeader Host: localhost:8080 > - ReqHeader User-Agent: curl/7.47.0 > - ReqHeader Accept: */* > - ReqHeader X-Forwarded-For: ::1 > - VCL_call RECV > - VCL_acl MATCH purge "localhost" > - VCL_return purge > - VCL_call HASH > - VCL_return lookup > - VCL_call PURGE > - VCL_return synth > - Timestamp Process: 1575890593.662268 0.000135 0.000135 > - RespHeader Date: Mon, 09 Dec 2019 11:23:13 GMT > - RespHeader Server: Varnish > - RespHeader X-Varnish: 32773 > - RespProtocol HTTP/1.1 > - RespStatus 200 > - RespReason OK > - RespReason Purged > - VCL_call SYNTH > - RespHeader Content-Type: text/html; charset=utf-8 > - RespHeader Retry-After: 5 > - VCL_return deliver > - RespHeader Content-Length: 240 > - Storage malloc Transient > - RespHeader Accept-Ranges: bytes > - Debug "RES_MODE 2" > - RespHeader Connection: keep-alive > - Timestamp Resp: 1575890593.662484 0.000352 0.000216 > - ReqAcct 93 0 93 218 240 458 > - End > > * << Session >> 32772 > - Begin sess 0 HTTP/1 > - SessOpen ::1 63184 :8080 ::1 8080 1575890593.662044 16 > - Link req 32773 rxreq > - SessClose REM_CLOSE 0.001 > - End > > * << Request >> 12 > - Begin req 11 rxreq > - Timestamp Start: 1575890600.799371 0.000000 0.000000 > - Timestamp Req: 1575890600.799371 0.000000 0.000000 > - ReqStart ::1 63186 > - ReqMethod PURGE > - ReqURL /Naserfeiz.mp4 > - ReqProtocol HTTP/1.1 > - ReqHeader Host: localhost:8080 > - ReqHeader User-Agent: curl/7.47.0 > - ReqHeader Accept: */* > - ReqHeader X-Forwarded-For: ::1 > - VCL_call RECV > - VCL_acl MATCH purge "localhost" > - VCL_return purge > - VCL_call HASH > - VCL_return lookup > - VCL_call PURGE > - VCL_return synth > - Timestamp Process: 1575890600.799463 0.000092 0.000092 > - RespHeader Date: Mon, 09 Dec 2019 11:23:20 GMT > - RespHeader Server: Varnish > - RespHeader X-Varnish: 12 > - RespProtocol HTTP/1.1 > - RespStatus 200 > - RespReason OK > - RespReason Purged > - VCL_call SYNTH > - RespHeader Content-Type: text/html; charset=utf-8 > - RespHeader Retry-After: 5 > - VCL_return deliver > - RespHeader Content-Length: 237 > - Storage malloc Transient > - RespHeader Accept-Ranges: bytes > - Debug "RES_MODE 2" > - RespHeader Connection: keep-alive > - Timestamp Resp: 1575890600.799611 0.000240 0.000147 > - ReqAcct 93 0 93 215 237 452 > - End > > * << Session >> 11 > - Begin sess 0 HTTP/1 > - SessOpen ::1 63186 :8080 ::1 8080 1575890600.799261 16 > - Link req 12 rxreq > - SessClose REM_CLOSE 0.001 > - End > > ------------------------------ > *From:* shafeeque aslam <[email protected]> > *Sent:* Monday, December 9, 2019 3:19 AM > *To:* hamidreza hosseini <[email protected]> > *Cc:* Guillaume Quintard <[email protected]>; > [email protected] <[email protected]> > *Subject:* Re: Varnish doesnt PURGE urls > > This is purge to 1st Varnish server only. Give another purge request to > 2nd Varnish server giving it's port number. > > On Mon 9 Dec, 2019, 4:45 PM hamidreza hosseini, <[email protected]> > wrote: > > Yes, surely i purge both servers > This is my purge request: > > $curl -v -k -X PURGE http://localhost/Naserfeiz.mp4 > > * Trying ::1... > * Connected to localhost (::1) port 80 (#0) > > PURGE /Naserfeiz.mp4 HTTP/1.1 > > Host: localhost > > User-Agent: curl/7.47.0 > > Accept: */* > > > < HTTP/1.1 200 Purged > < Date: Mon, 09 Dec 2019 11:12:54 GMT > < Server: Varnish > < X-Varnish: 98306 > < Content-Type: text/html; charset=utf-8 > < Retry-After: 5 > < Content-Length: 240 > < Accept-Ranges: bytes > < Connection: keep-alive > < > <!DOCTYPE html> > <html> > <head> > <title>200 Purged</title> > </head> > <body> > <h1>Error 200 Purged</h1> > <p>Purged</p> > <h3>Guru Meditation:</h3> > <p>XID: 98306</p> > <hr> > <p>Varnish cache server</p> > </body> > </html> > * Connection #0 to host localhost left intact > > > And this is varnishlog resault: > > * << BeReq >> 32771 > - Begin bereq 32770 fetch > - Timestamp Start: 1575889894.491843 0.000000 0.000000 > - BereqMethod GET > - BereqURL / > - BereqProtocol HTTP/1.1 > - BereqHeader Host: 192.168.200.13 > - BereqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; > rv:70.0) Gecko/20100101 Firefox/70.0 > - BereqHeader Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > - BereqHeader Accept-Language: en-US,en;q=0.5 > - BereqHeader Upgrade-Insecure-Requests: 1 > - BereqHeader X-Forwarded-For: 172.16.1.3 > - BereqHeader Accept-Encoding: gzip > - BereqHeader X-Varnish: 32771 > - VCL_call BACKEND_FETCH > - VCL_return fetch > - BackendOpen 18 boot.varnish_malloc_01 192.168.200.12 8080 > 192.168.200.13 14088 > - Timestamp Bereq: 1575889894.492444 0.000601 0.000601 > - Timestamp Beresp: 1575889894.493276 0.001433 0.000832 > - BerespProtocol HTTP/1.1 > - BerespStatus 200 > - BerespReason OK > - BerespHeader Server: nginx > - BerespHeader Date: Mon, 09 Dec 2019 10:00:29 GMT > - BerespHeader Content-Type: text/html > - BerespHeader Content-Length: 340 > - BerespHeader Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT > - BerespHeader ETag: "5dcaad34-154" > - BerespHeader X-Varnish: 8 6 > - BerespHeader Age: 4270 > - BerespHeader Via: 1.1 varnish-v4 > - BerespHeader Accept-Ranges: bytes > - BerespHeader Connection: keep-alive > - TTL RFC 120 10 -1 1575889894 1575885624 1575885629 0 0 > - VCL_call BACKEND_RESPONSE > - TTL VCL 177070 10 0 1575885624 > - VCL_return deliver > - Storage malloc s0 > - ObjProtocol HTTP/1.1 > - ObjStatus 200 > - ObjReason OK > - ObjHeader Server: nginx > - ObjHeader Date: Mon, 09 Dec 2019 10:00:29 GMT > - ObjHeader Content-Type: text/html > - ObjHeader Content-Length: 340 > - ObjHeader Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT > - ObjHeader ETag: "5dcaad34-154" > - ObjHeader X-Varnish: 8 6 > - ObjHeader Via: 1.1 varnish-v4 > - Fetch_Body 3 length stream > - BackendReuse 18 boot.varnish_malloc_01 > - Timestamp BerespBody: 1575889894.493474 0.001631 0.000198 > - Length 340 > - BereqAcct 336 0 336 279 340 619 > - End > > * << Request >> 32770 > - Begin req 32769 rxreq > - Timestamp Start: 1575889894.491659 0.000000 0.000000 > - Timestamp Req: 1575889894.491659 0.000000 0.000000 > - ReqStart 172.16.1.3 42502 > - ReqMethod GET > - ReqURL / > - ReqProtocol HTTP/1.1 > - ReqHeader Host: 192.168.200.13 > - ReqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; > rv:70.0) Gecko/20100101 Firefox/70.0 > - ReqHeader Accept: > text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > - ReqHeader Accept-Language: en-US,en;q=0.5 > - ReqHeader Accept-Encoding: gzip, deflate > - ReqHeader Connection: keep-alive > - ReqHeader Cookie: _ga=GA1.1.1476719154.1572941620; > _gid=GA1.1.117112441.1575877103 > - ReqHeader Upgrade-Insecure-Requests: 1 > - ReqHeader If-Modified-Since: Tue, 12 Nov 2019 13:01:40 GMT > - ReqHeader If-None-Match: "5dcaad34-154" > - ReqHeader Cache-Control: max-age=0 > - ReqHeader X-Forwarded-For: 172.16.1.3 > - VCL_call RECV > - ReqUnset Cookie: _ga=GA1.1.1476719154.1572941620; > _gid=GA1.1.117112441.1575877103 > - VCL_return hash > - ReqUnset Accept-Encoding: gzip, deflate > - ReqHeader Accept-Encoding: gzip > - VCL_call HASH > - VCL_return lookup > - VCL_call MISS > - VCL_return fetch > - Link bereq 32771 fetch > - Timestamp Fetch: 1575889894.493501 0.001842 0.001842 > - RespProtocol HTTP/1.1 > - RespStatus 200 > - RespReason OK > - RespHeader Server: nginx > - RespHeader Date: Mon, 09 Dec 2019 10:00:29 GMT > - RespHeader Content-Type: text/html > - RespHeader Content-Length: 340 > - RespHeader Last-Modified: Tue, 12 Nov 2019 13:01:40 GMT > - RespHeader ETag: "5dcaad34-154" > - RespHeader X-Varnish: 8 6 > - RespHeader Via: 1.1 varnish-v4 > - RespHeader X-Varnish: 32770 > - RespHeader Age: 4270 > - RespHeader Via: 1.1 varnish-v4 > - VCL_call DELIVER > - VCL_return deliver > - Timestamp Process: 1575889894.493541 0.001881 0.000039 > - RespProtocol HTTP/1.1 > - RespStatus 304 > - RespReason Not Modified > - RespReason Not Modified > - RespUnset Content-Length: 340 > - Debug "RES_MODE 0" > - RespHeader Connection: keep-alive > - Timestamp Resp: 1575889894.493599 0.001940 0.000058 > - ReqAcct 503 0 503 285 0 285 > - End > > * << Request >> 65538 > - Begin req 65537 rxreq > - Timestamp Start: 1575889962.124164 0.000000 0.000000 > - Timestamp Req: 1575889962.124164 0.000000 0.000000 > - ReqStart ::1 62064 > - ReqMethod PURGE > - ReqURL /Naserfeiz.mp4 > - ReqProtocol HTTP/1.1 > - ReqHeader Host: localhost > - ReqHeader User-Agent: curl/7.47.0 > - ReqHeader Accept: */* > - ReqHeader X-Forwarded-For: ::1 > - VCL_call RECV > - VCL_acl MATCH purge "localhost" > - VCL_return purge > - VCL_call HASH > - VCL_return lookup > - VCL_call PURGE > - VCL_return synth > - Timestamp Process: 1575889962.124300 0.000136 0.000136 > - RespHeader Date: Mon, 09 Dec 2019 11:12:42 GMT > - RespHeader Server: Varnish > - RespHeader X-Varnish: 65538 > - RespProtocol HTTP/1.1 > - RespStatus 200 > - RespReason OK > - RespReason Purged > - VCL_call SYNTH > - RespHeader Content-Type: text/html; charset=utf-8 > - RespHeader Retry-After: 5 > - VCL_return deliver > - RespHeader Content-Length: 240 > - Storage malloc Transient > - RespHeader Accept-Ranges: bytes > - Debug "RES_MODE 2" > - RespHeader Connection: keep-alive > - Timestamp Resp: 1575889962.124593 0.000428 0.000292 > - ReqAcct 88 0 88 218 240 458 > - End > > * << Session >> 65537 > - Begin sess 0 HTTP/1 > - SessOpen ::1 62064 :80 ::1 80 1575889962.124081 13 > - Link req 65538 rxreq > - SessClose REM_CLOSE 0.001 > - End > > * << Request >> 98306 > - Begin req 98305 rxreq > - Timestamp Start: 1575889974.116017 0.000000 0.000000 > - Timestamp Req: 1575889974.116017 0.000000 0.000000 > - ReqStart ::1 62066 > - ReqMethod PURGE > - ReqURL /Naserfeiz.mp4 > - ReqProtocol HTTP/1.1 > - ReqHeader Host: localhost > - ReqHeader User-Agent: curl/7.47.0 > - ReqHeader Accept: */* > - ReqHeader X-Forwarded-For: ::1 > - VCL_call RECV > - VCL_acl MATCH purge "localhost" > - VCL_return purge > - VCL_call HASH > - VCL_return lookup > - VCL_call PURGE > - VCL_return synth > - Timestamp Process: 1575889974.116128 0.000111 0.000111 > - RespHeader Date: Mon, 09 Dec 2019 11:12:54 GMT > - RespHeader Server: Varnish > - RespHeader X-Varnish: 98306 > - RespProtocol HTTP/1.1 > - RespStatus 200 > - RespReason OK > - RespReason Purged > - VCL_call SYNTH > - RespHeader Content-Type: text/html; charset=utf-8 > - RespHeader Retry-After: 5 > - VCL_return deliver > - RespHeader Content-Length: 240 > - Storage malloc Transient > - RespHeader Accept-Ranges: bytes > - Debug "RES_MODE 2" > - RespHeader Connection: keep-alive > - Timestamp Resp: 1575889974.116288 0.000271 0.000160 > - ReqAcct 88 0 88 218 240 458 > - End > > * << Session >> 98305 > - Begin sess 0 HTTP/1 > - SessOpen ::1 62066 :80 ::1 80 1575889974.115935 15 > - Link req 98306 rxreq > - SessClose REM_CLOSE 0.002 > - End > > * << Session >> 32769 > - Begin sess 0 HTTP/1 > - SessOpen 172.16.1.3 42502 :80 192.168.200.13 80 > 1575889894.491572 14 > - Link req 32770 rxreq > - Link req 32772 rxreq > - VSL timeout > - End synth > > * << Request >> 32772 > - Begin req 32769 rxreq > - Timestamp Start: 1575889894.845556 0.000000 0.000000 > - Timestamp Req: 1575889894.845556 0.000000 0.000000 > - ReqStart 172.16.1.3 42502 > - ReqMethod GET > - ReqURL /Naserfeiz.mp4 > - ReqProtocol HTTP/1.1 > - ReqHeader Host: 192.168.200.13 > - ReqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; > rv:70.0) Gecko/20100101 Firefox/70.0 > - ReqHeader Accept: > video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5 > - ReqHeader Accept-Language: en-US,en;q=0.5 > - ReqHeader Range: bytes=205560- > - ReqHeader Connection: keep-alive > - ReqHeader Referer: http://192.168.200.13/ > - ReqHeader Cookie: _ga=GA1.1.1476719154.1572941620; > _gid=GA1.1.117112441.1575877103 > - ReqHeader X-Forwarded-For: 172.16.1.3 > - VCL_call RECV > - ReqUnset Cookie: _ga=GA1.1.1476719154.1572941620; > _gid=GA1.1.117112441.1575877103 > - VCL_return hash > - VCL_call HASH > - VCL_return lookup > - Hit 3 > - VCL_call HIT > - VCL_return deliver > - RespProtocol HTTP/1.1 > - RespStatus 200 > - RespReason OK > - RespHeader Server: nginx > - RespHeader Date: Mon, 09 Dec 2019 11:06:51 GMT > - RespHeader Content-Type: text/plain > - RespHeader Content-Length: 23521499 > - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT > - RespHeader ETag: "3a1794b0-166e8db" > - RespHeader X-Varnish: 32770 > - RespHeader Via: 1.1 varnish-v4 > - RespHeader X-Varnish: 32772 3 > - RespHeader Age: 288 > - RespHeader Via: 1.1 varnish-v4 > - VCL_call DELIVER > - VCL_return deliver > - Timestamp Process: 1575889894.845683 0.000127 0.000127 > - RespHeader Accept-Ranges: bytes > - RespHeader Content-Range: bytes 205560-23521498/23521499 > - RespProtocol HTTP/1.1 > - RespStatus 206 > - RespReason Partial Content > - RespReason Partial Content > - RespUnset Content-Length: 23521499 > - RespHeader Content-Length: 23315939 > - Debug "RES_MODE 2" > - RespHeader Connection: keep-alive > - Debug "Hit idle send timeout, wrote = 10899908/23316330; > retrying" > - Debug "Hit idle send timeout, wrote = 258432/12416422; > retrying" > - Debug "Write error, retval = -1, len = 12157990, errno = > Resource temporarily unavailable" > - Timestamp Resp: 1575890074.846999 180.001443 180.001316 > - ReqAcct 419 0 419 391 23315939 23316330 > - End > > ########################## > ------------------------------ > *From:* Guillaume Quintard <[email protected]> > *Sent:* Monday, December 9, 2019 11:57 AM > *To:* hamidreza hosseini <[email protected]> > *Cc:* [email protected] <[email protected]> > *Subject:* Re: Varnish doesnt PURGE urls > > Are you sure that you purge both Varnish layers (file and RAM)? > > can you pastebin a varnishlog of the purge and of the subsequent request? > > -- > Guillaume Quintard > > > On Mon, Dec 9, 2019 at 2:55 AM hamidreza hosseini <[email protected]> > wrote: > > because its steel in cach and i can download and see th urls that i purged > before! > its Interesting that it return 200 code from varnish!!! > > Best regards. > ------------------------------ > *From:* Guillaume Quintard <[email protected]> > *Sent:* Monday, December 9, 2019 11:48 AM > *To:* hamidreza hosseini <[email protected]> > *Cc:* [email protected] <[email protected]> > *Subject:* Re: Varnish doesnt PURGE urls > > Hi, > > How do you tell that it's not purging? > > Cheers, > > -- > Guillaume Quintard > > > On Mon, Dec 9, 2019 at 2:45 AM hamidreza hosseini <[email protected]> > wrote: > > Hi, > > I've two varnish, that one of them use ram as backend and another file as > backend, requests first come to varnish(ram) and then go to another server > in varnish(file) and then go to nginx, this is my diagram: > > > Req=====> [varnish ram] =======> [varnish file] =====> [Nginx] > > And following file is my vcl configuration file that i'm using, > but when i want to purge a url it doesnt work and it steel remains in > cache, > Should i chage my config file? > What's wrong with varnish? > > Vcl config file: > > ### Naming note: use [0-9a-zA-Z_] only. no -. Or use camelCase ;-) > vcl 4.0; > > import directors; > > > backend varnish_malloc_01 { .host = "192.168.200.12"; .port = "8080"; } > > > sub vcl_init { > new varnish_cluster = directors.round_robin(); > varnish_cluster.add_backend(varnish_malloc_01); > > } > > > acl purge { > "localhost"; > "192.168.200.0"/24; > ; > } > > > sub vcl_recv { > > sub vcl_recv { > #set req.backend_hint = varnish_cluster.backend(); > unset req.http.Cookie; > if (req.method == "PURGE") { > ### TODO: also purge from the backend > if (!client.ip ~ purge) { > return(synth(405,"Not allowed.")); > } > return (purge); > } > } > > sub vcl_backend_response { > set beresp.ttl=48h; > # Don't cache 404 responses > if ( beresp.status == 404 ) { > set beresp.ttl = 120s; > set beresp.uncacheable = true; > return (deliver); > } > } > > > _______________________________________________ > varnish-misc mailing list > [email protected] > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > _______________________________________________ > varnish-misc mailing list > [email protected] > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > >
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
