Hi Andrei, Here are the results:
T *****:56538 -> ******:8080 [AP] POST /wp-admin/admin-ajax.php HTTP/1.1. Host: *******.com. Accept-Language: tr-tr. User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 10_0 like Mac OS X) AppleWebKit/6 02.1.40 (KHTML, like Gecko) Version/10.0 Mobile/14A5309d Safari/602.1. X-Requested-With: XMLHttpRequest. Accept: application/json, text/javascript, */*; q=0.01. Referer: http://******/. Content-Type: application/x-www-form-urlencoded; charset=UTF-8. Content-Length: 130. Origin: http://****.com. X-Actual-IP: 5.46.70.115. Cookie: pps_show_100=Tue%20Aug%2002%202016%2018%3A50%3A21%20GMT+0300%20%28EEST%2 9; pps_times_showed_100=1. X-Forwarded-For: 5.46.70.115, 5.46.70.115, 5.46.70.115. Accept-Encoding: gzip. X-Varnish: 39754073. . mod=statistics&action=add&id=100&type=show&sm_type=0&is_unique=1&connect_hash=c1 532a201e2ee25540c61d3199e0e960&pl=pps&reqType=ajax T *****:56652 -> ******:8080 [AP] POST /wp-admin/admin-ajax.php HTTP/1.1. Host: *****.com. Content-Length: 130. Origin: http://*****.com. X-Requested-With: XMLHttpRequest. User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:46.0) Gecko/20100101 Firefox/46.19. Content-Type: application/x-www-form-urlencoded; charset=UTF-8. Accept: application/json, text/javascript, */*; q=0.01. Referer: https://www.yandex.com.tr/. Accept-Language: en-us,en. Accept-Charset: iso-8859-1,*,utf-8. X-Actual-IP: 78.173.206.179. Cookie: PHPSESSID=br4avjrtmgd0e2j04eoelev4r6; pps_show_100=Tue%20Aug%2002%202016%2018%3A51%3A14%20GMT+0300%20%28Turkey%20Daylight%20 Time%29; pps_times_showed_100=2. X-Forwarded-For: 78.173.206.179, 78.173.206.179, 78.173.206.179. Accept-Encoding: gzip. X-Varnish: 37456291. . mod=statistics&action=add&id=100&type=show&sm_type=0&is_unique=0&connect_hash=c1532a201e2ee25540c61d3199e0e960&pl=pps&reqType=ajax T *****:56630 -> *****.10:8080 [AP] POST /wp-admin/admin-ajax.php HTTP/1.1. Host: *****.com. Content-Length: 130. Origin: http://*****.com. X-Requested-With: XMLHttpRequest. User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:46.0) Gecko/20100101 Firefox/46.8. Content-Type: application/x-www-form-urlencoded; charset=UTF-8. Accept: application/json, text/javascript, */*; q=0.01. Referer: https://google.com.tr/. Accept-Language: en-us,en. Accept-Charset: iso-8859-1,*,utf-8. X-Actual-IP: 46.197.96.35. Cookie: pps_show_100=Tue%20Aug%2002%202016%2018%3A51%3A27%20GMT+0300%20%28T%FCrkiye%20Yaz%20Saati%29; pps_times_showed_100=1. X-Forwarded-For: 46.197.96.35, 46.197.96.35, 46.197.96.35. Accept-Encoding: gzip. X-Varnish: 35061669. Heartbeat plugin didn't fix it by the way. 2016-08-02 18:45 GMT+03:00 Andrei <[email protected]>: > Those admin-ajax.php POST requests won't get cached, and are likely > related to WordPress heartbeats, or plugins. The quickest way to see what > those requests actually are, which will help you identify the plugin/theme > option is using ngrep: ngrep 'admin-ajax' -d any dst port 8080 -W byline -q > > On Tue, Aug 2, 2016 at 9:43 AM, MAGNIEN, Thierry <[email protected]> > wrote: > >> Hi, >> >> >> >> There are known CPU issues due to WordPress HearBeat API, for example, or >> misbehaving plugins. >> >> >> >> See >> http://www.inmotionhosting.com/support/website/wordpress/heartbeat-ajax-php-usage >> for example. >> >> >> >> Maybe one of your plugins has some difficulties going through varnish. >> >> >> >> If you enable debug on your browser, can you see specific calls to >> /wp-admin/admin-ajax.php that take a very long time with varnish and not >> when varnish is disabled ? >> >> >> >> Thierry >> >> >> >> *De :* [email protected] >> [mailto:[email protected]] *De >> la part de* Ayberk Kimsesiz >> *Envoyé :* mardi 2 août 2016 15:18 >> *À :* Stig Bakken >> *Cc :* varnish-misc >> *Objet :* Re: Varnish CPU Usage >> >> >> >> I used Default.VCL in two different ways. The first with default settings >> and the second with settings customized for Wordpress (either case, the CPU >> usage increases). That is as follows: >> >> >> >> Also i couldn't find a Centos 6 installation guide for Prometheus. >> >> >> >> */* SET THE HOST AND PORT OF WORDPRESS* >> >> * * *********************************************************/* >> >> *vcl 4.0;* >> >> *import std;* >> >> >> >> *backend default {* >> >> * .host = "SERVER IP";* >> >> * .port = "8080";* >> >> * .first_byte_timeout = 60s;* >> >> * .connect_timeout = 300s;* >> >> *}* >> >> >> >> *# SET THE ALLOWED IP OF PURGE REQUESTS* >> >> *# ##########################################################* >> >> *acl purge {* >> >> * "localhost";* >> >> * "127.0.0.1";* >> >> *}* >> >> >> >> *#THE RECV FUNCTION* >> >> *# ##########################################################* >> >> *sub vcl_recv {* >> >> >> >> *# set realIP by trimming CloudFlare IP which will be used for various >> checks* >> >> *set req.http.X-Actual-IP = regsub(req.http.X-Forwarded-For, "[, ].*$", >> ""); * >> >> >> >> * # FORWARD THE IP OF THE REQUEST* >> >> * if (req.restarts == 0) {* >> >> * if (req.http.x-forwarded-for) {* >> >> * set req.http.X-Forwarded-For =* >> >> * req.http.X-Forwarded-For + ", " + client.ip;* >> >> * } else {* >> >> * set req.http.X-Forwarded-For = client.ip;* >> >> * }* >> >> * }* >> >> >> >> * # Purge request check sections for hash_always_miss, purge and ban* >> >> * # BLOCK IF NOT IP is not in purge acl* >> >> * # ##########################################################* >> >> >> >> * # Enable smart refreshing using hash_always_miss* >> >> *if (req.http.Cache-Control ~ "no-cache") {* >> >> * if (client.ip ~ purge || !std.ip(req.http.X-Actual-IP, "1.2.3.4") ~ >> purge) {* >> >> * set req.hash_always_miss = true;* >> >> * }* >> >> *}* >> >> >> >> *if (req.method == "PURGE") {* >> >> * if (!client.ip ~ purge || !std.ip(req.http.X-Actual-IP, "1.2.3.4") ~ >> purge) {* >> >> * return(synth(405,"Not allowed."));* >> >> * }* >> >> * return (purge);* >> >> >> >> * }* >> >> *if (req.method == "BAN") {* >> >> * # Same ACL check as above:* >> >> * if (!client.ip ~ purge || !std.ip(req.http.X-Actual-IP, >> "1.2.3.4") ~ purge) {* >> >> * return(synth(403, "Not allowed."));* >> >> * }* >> >> * ban("req.http.host == " + req.http.host +* >> >> * " && req.url == " + req.url);* >> >> >> >> * # Throw a synthetic page so the* >> >> * # request won't go to the backend.* >> >> * return(synth(200, "Ban added"));* >> >> *}* >> >> >> >> *# Unset cloudflare cookies* >> >> *# Remove has_js and CloudFlare/Google Analytics __* cookies.* >> >> * set req.http.Cookie = regsuball(req.http.Cookie, >> "(^|;\s*)(_[_a-z]+|has_js)=[^;]*", "");* >> >> * # Remove a ";" prefix, if present.* >> >> * set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");* >> >> >> >> * # For Testing: If you want to test with Varnish passing (not caching) >> uncomment* >> >> * # return( pass );* >> >> >> >> * # FORWARD THE IP OF THE REQUEST* >> >> * if (req.restarts == 0) {* >> >> * if (req.http.x-forwarded-for) {* >> >> * set req.http.X-Forwarded-For =* >> >> * req.http.X-Forwarded-For + ", " + client.ip;* >> >> * } else {* >> >> * set req.http.X-Forwarded-For = client.ip;* >> >> * }* >> >> * }* >> >> >> >> *# DO NOT CACHE RSS FEED* >> >> * if (req.url ~ "/feed(/)?") {* >> >> * return ( pass ); * >> >> *}* >> >> >> >> *## Do not cache search results, comment these 3 lines if you do want to >> cache them* >> >> >> >> *if (req.url ~ "/\?s\=") {* >> >> * return ( pass ); * >> >> *}* >> >> >> >> *# CLEAN UP THE ENCODING HEADER.* >> >> * # SET TO GZIP, DEFLATE, OR REMOVE ENTIRELY. WITH VARY ACCEPT-ENCODING* >> >> * # VARNISH WILL CREATE SEPARATE CACHES FOR EACH* >> >> * # DO NOT ACCEPT-ENCODING IMAGES, ZIPPED FILES, AUDIO, ETC.* >> >> * # ##########################################################* >> >> * if (req.http.Accept-Encoding) {* >> >> * if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {* >> >> * # No point in compressing these* >> >> * unset req.http.Accept-Encoding;* >> >> * } elsif (req.http.Accept-Encoding ~ "gzip") {* >> >> * set req.http.Accept-Encoding = "gzip";* >> >> * } elsif (req.http.Accept-Encoding ~ "deflate") {* >> >> * set req.http.Accept-Encoding = "deflate";* >> >> * } else {* >> >> * # unknown algorithm* >> >> * unset req.http.Accept-Encoding;* >> >> * }* >> >> * }* >> >> >> >> * # PIPE ALL NON-STANDARD REQUESTS* >> >> * # ##########################################################* >> >> * if (req.method != "GET" &&* >> >> * req.method != "HEAD" &&* >> >> * req.method != "PUT" && * >> >> * req.method != "POST" &&* >> >> * req.method != "TRACE" &&* >> >> * req.method != "OPTIONS" &&* >> >> * req.method != "DELETE") {* >> >> * return (pipe);* >> >> * }* >> >> >> >> * # ONLY CACHE GET AND HEAD REQUESTS* >> >> * # ##########################################################* >> >> * if (req.method != "GET" && req.method != "HEAD") {* >> >> * return (pass);* >> >> * }* >> >> >> >> * # OPTIONAL: DO NOT CACHE LOGGED IN USERS (THIS OCCURS IN FETCH TOO, >> EITHER* >> >> * # COMMENT OR UNCOMMENT BOTH* >> >> * # ##########################################################* >> >> * if ( req.http.cookie ~ "wordpress_logged_in" ) {* >> >> * return( pass );* >> >> * }* >> >> >> >> * # IF THE REQUEST IS NOT FOR A PREVIEW, WP-ADMIN OR WP-LOGIN* >> >> * # THEN UNSET THE COOKIES* >> >> * # ##########################################################* >> >> * if (!(req.url ~ "wp-(login|admin)") * >> >> * && !(req.url ~ "&preview=true" ) * >> >> * ){* >> >> * unset req.http.cookie;* >> >> * }* >> >> >> >> * # IF BASIC AUTH IS ON THEN DO NOT CACHE* >> >> * # ##########################################################* >> >> * if (req.http.Authorization || req.http.Cookie) {* >> >> * return (pass);* >> >> * }* >> >> >> >> * # IF YOU GET HERE THEN THIS REQUEST SHOULD BE CACHED* >> >> * # ##########################################################* >> >> * return (hash);* >> >> * # This is for phpmyadmin* >> >> *if (req.http.Host == "ki1.org <http://ki1.org>") {* >> >> *return (pass);* >> >> *}* >> >> >> >> *if (req.http.Host == "mysql.ki1.org <http://mysql.ki1.org>") {* >> >> *return (pass);* >> >> *}* >> >> >> >> *}* >> >> >> >> *# HIT FUNCTION* >> >> *# ##########################################################* >> >> *sub vcl_hit {* >> >> * # IF THIS IS A PURGE REQUEST THEN DO THE PURGE* >> >> * # ##########################################################* >> >> * if (req.method == "PURGE") {* >> >> * #* >> >> * # This is now handled in vcl_recv.* >> >> * #* >> >> * # purge;* >> >> * return (synth(200, "Purged."));* >> >> * }* >> >> * return (deliver);* >> >> *}* >> >> >> >> *# MISS FUNCTION* >> >> *# ##########################################################* >> >> *sub vcl_miss {* >> >> * if (req.method == "PURGE") {* >> >> * #* >> >> * # This is now handled in vcl_recv.* >> >> * #* >> >> * # purge;* >> >> * return (synth(200, "Purged."));* >> >> * }* >> >> * return (fetch);* >> >> *}* >> >> >> >> *# FETCH FUNCTION* >> >> *# ##########################################################* >> >> *sub vcl_backend_response {* >> >> * # I SET THE VARY TO ACCEPT-ENCODING, THIS OVERRIDES W3TC * >> >> * # TENDANCY TO SET VARY USER-AGENT. YOU MAY OR MAY NOT WANT* >> >> * # TO DO THIS* >> >> * # ##########################################################* >> >> * set beresp.http.Vary = "Accept-Encoding";* >> >> >> >> * # IF NOT WP-ADMIN THEN UNSET COOKIES AND SET THE AMOUNT OF * >> >> * # TIME THIS PAGE WILL STAY CACHED (TTL)* >> >> * # ##########################################################* >> >> * if (!(bereq.url ~ "wp-(login|admin)") && !bereq.http.cookie ~ >> "wordpress_logged_in" ) {* >> >> * unset beresp.http.set-cookie;* >> >> * set beresp.ttl = 52w;* >> >> *# set beresp.grace =1w;* >> >> * }* >> >> >> >> * if (beresp.ttl <= 0s ||* >> >> * beresp.http.Set-Cookie ||* >> >> * beresp.http.Vary == "*") {* >> >> * set beresp.ttl = 120 s;* >> >> * # set beresp.ttl = 120s;* >> >> * set beresp.uncacheable = true;* >> >> * return (deliver);* >> >> * }* >> >> >> >> * return (deliver);* >> >> *}* >> >> >> >> *# DELIVER FUNCTION* >> >> *# ##########################################################* >> >> *sub vcl_deliver {* >> >> * # IF THIS PAGE IS ALREADY CACHED THEN RETURN A 'HIT' TEXT * >> >> * # IN THE HEADER (GREAT FOR DEBUGGING)* >> >> * # ##########################################################* >> >> * if (obj.hits > 0) {* >> >> * set resp.http.X-Cache = "HIT";* >> >> * # IF THIS IS A MISS RETURN THAT IN THE HEADER* >> >> * # ##########################################################* >> >> * } else {* >> >> * set resp.http.X-Cache = "MISS";* >> >> * }* >> >> *}* >> >> >> >> >> >> >> >> 2016-08-02 15:07 GMT+03:00 Stig Bakken <[email protected]>: >> >> Could it be that Varnish is not caching those POST requests, and that >> they are what makes Apache consume a lot of CPU? >> >> >> >> On MPMs: with PHP prefork is the safest choice, and it will work just >> fine as long as you have a reasonable config, especially since you have >> Varnish in front offloading the job of shuffling bytes back to the users. >> >> >> >> If I were you I'd try to get more data on which requests take a long >> time, some variant of "varnishncsa -F %D".. >> >> >> >> If you're into using Prometheus for metrics, you can try out this >> exporter which will give you a lot of insight into which requests take a >> long time to process: https://github.com/stigsb/varnish_request_exporter >> (Prometheus is awesome!) >> >> >> >> - Stig >> >> >> >> >> >> On Thu, Jul 28, 2016 at 6:57 PM, Ayberk Kimsesiz < >> [email protected]> wrote: >> >> Hi, >> >> >> >> *CPU Monitor: * >> >> >> >> http://i.imgur.com/5KT1xRu.jpg >> >> >> >> *Apache status:* >> >> >> >> *Srv* >> >> *PID* >> >> *Acc* >> >> *M* >> >> *CPU* >> >> *SS* >> >> *Req* >> >> *Conn* >> >> *Child* >> >> *Slot* >> >> *Client* >> >> *Protocol* >> >> *VHost* >> >> *Request* >> >> *0-0* >> >> - >> >> 0/0/9766 >> >> . >> >> 134.59 >> >> 37 >> >> 0 >> >> 0.0 >> >> 0.00 >> >> 64.40 >> >> ::1 >> >> http/1.1 >> >> ns1.***com:8080 >> >> OPTIONS * HTTP/1.0 >> >> *1-0* >> >> 14612 >> >> 0/16/9058 >> >> _ >> >> 17.83 >> >> 13 >> >> 1498 >> >> 0.0 >> >> 0.02 >> >> 53.29 >> >> 176.***.10 >> >> http/1.1 >> >> www.***.com:8080 >> >> POST /wp-admin/admin-ajax.php HTTP/1.1 >> >> *2-0* >> >> 10863 >> >> 0/179/9795 >> >> _ >> >> 185.14 >> >> 6 >> >> 1424 >> >> 0.0 >> >> 0.58 >> >> 60.32 >> >> 176.***.10 >> >> http/1.1 >> >> www.***.com:8080 >> >> POST /wp-admin/admin-ajax.php HTTP/1.1 >> >> *3-0* >> >> 13127 >> >> 0/127/9435 >> >> _ >> >> 119.80 >> >> 4 >> >> 1419 >> >> 0.0 >> >> 0.42 >> >> 56.51 >> >> 176.***.10 >> >> http/1.1 >> >> www.***.com:8080 >> >> POST /wp-admin/admin-ajax.php HTTP/1.1 >> >> *4-0* >> >> - >> >> 0/0/9187 >> >> . >> >> 0.00 >> >> 50 >> >> 0 >> >> 0.0 >> >> 0.00 >> >> 56.60 >> >> ::1 >> >> http/1.1 >> >> ns1.***.com:8080 >> >> OPTIONS * HTTP/1.0 >> >> *5-0* >> >> 14851 >> >> 0/9/8761 >> >> _ >> >> 8.95 >> >> 13 >> >> 1559 >> >> 0.0 >> >> 0.01 >> >> 57.90 >> >> 176.***.10 >> >> http/1.1 >> >> www.***.com:8080 >> >> POST /wp-admin/admin-ajax.php HTTP/1.1 >> >> *6-0* >> >> 14852 >> >> 0/6/8130 >> >> _ >> >> 6.67 >> >> 4 >> >> 1482 >> >> 0.0 >> >> 0.01 >> >> 51.88 >> >> 176.***.10 >> >> http/1.1 >> >> www.***.com:8080 >> >> POST /wp-admin/admin-ajax.php HTTP/1.1 >> >> *7-0* >> >> 14192 >> >> 11/57/8355 >> >> *K* >> >> 72.73 >> >> 0 >> >> 1363 >> >> 106.6 >> >> 0.44 >> >> 52.79 >> >> 176.***.10 >> >> http/1.1 >> >> www.***.com:8080 >> >> POST /wp-admin/admin-ajax.php HTTP/1.1 >> >> *8-0* >> >> 13067 >> >> 0/125/7795 >> >> _ >> >> 121.19 >> >> 13 >> >> 1759 >> >> 0.0 >> >> 0.65 >> >> 68.27 >> >> 176.***.10 >> >> http/1.1 >> >> www.***.com:8080 >> >> POST /wp-admin/admin-ajax.php HTTP/1.1 >> >> >> >> Apache error logs don't show anything about CPU. >> >> >> >> >> >> 2016-07-28 19:31 GMT+03:00 Guillaume Quintard < >> [email protected]>: >> >> You seem to have a good hit ratio, are you seeing anything on the apache >> logs that would explain the cpu usage? >> >> >> -- >> >> Guillaume Quintard >> >> >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> [email protected] >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> >> >> >> >> -- >> >> Stig Bakken >> CTO, Zedge.net - free your phone! >> >> >> >> _______________________________________________ >> varnish-misc mailing list >> [email protected] >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > >
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
