Hi Charles, So, if I'm reading this right, there's no discrepancy. Varnish will malloc the full storage (malloc), and will mmap a file the size of the full storage (file). So even though the storage is not used, it's allocated.
Does it make sense, or did I miss something? -- Guillaume Quintard On Mon, Jul 10, 2017 at 1:23 PM, Bender, Charles <[email protected]> wrote: > Hi, > > > > There is a large discrepancy between Varnish resident memory reported by > top vs reported by varnishstat. Varnish is configured with both malloc and > file storage; 20G malloc and 75G file storage. > > > > These are the startup parameters- > > > > VARNISH_STORAGE="memcache=malloc,20G -s filecache=file,/mnt/xvdf1/ > varnish/varnish_storage.bin,75G" > > > > After running for a few days top is reporting more than twice amount of > memory used for varnishd process than varnishstat. > > > > From top- > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ > COMMAND > > > 1083 varnish 20 0 82.235g 0.014t 7.765g S 75.7 51.4 2159:07 > varnishd > > > > > From varnishstat- > > > > SMA.memcache.g_bytes > 5.20G 39.65K . 5.20G 5.20G 5.20G > > SMA.memcache.g_space > 14.80G -39.65K . 14.80G 14.80G > 14.80G > > SMF.filecache.g_bytes > 7.78G 27.97K . 7.78G 7.78G > 7.78G > > SMF.filecache.g_space > 67.22G -27.97K . 67.22G 67.22G 67.22G > > > > This is the relevant part of the VCL regarding storage backend selection- > > > > sub vcl_backend_response { > > # define separate cache storage groups > > if (bereq.http.host ~ > "^(encore|thereal|static)\.(beachcamera|buydig)\.com") > { > > set beresp.storage_hint = "filecache"; > > set beresp.http.X-Cache-Storage = "disk"; > > } elsif (bereq.url ~ > "(?i)\.(jpg|jpeg|gif|ico|pdf|swf|png|zip)") > { > > set beresp.storage_hint = "filecache"; > > set beresp.http.X-Cache-Storage = "disk"; > > } elsif (bereq.url ~ "(?i)product\-image\.aspx") { > > set beresp.storage_hint = "filecache"; > > set beresp.http.X-Cache-Storage = "disk"; > > } else { > > set beresp.storage_hint = "memcache"; > > set beresp.http.X-Cache-Storage = > "memory"; > > } > > > > Can post entire VCL if needed. > > > > Would think that since varnishstat reports 5.20G RAM used the resident > memory should be around 6-7G, 14G seems excessively high. File storage > should use minimal resident memory, correct? > > > > Varnish was installed from Varnish Cache 4.1 repo. No VMODs loaded except > std and directors. Using latest 4.1.7 > > > > Anything else you need please let me know. > > > > > > _______________________________________________ > varnish-misc mailing list > [email protected] > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
