Self reply: Does varnish rewrite the /var/lib/varnish/<VARNISH_NAME>/_.vsm file periodically?
<googling> http://www.gossamer-threads.com/lists/varnish/misc/24678 <d'oh!> Looks like the big control-knob for this is vm.dirty_expire_centisecs for modern linux kernels. Every interval, an async copy of the shared memory log goes to disk. I'll be evaluating just tweaking this, or moving it to a tmpfs volume. yay. -Jason On Mon, Apr 27, 2015 at 11:50 AM, Jason Price <[email protected]> wrote: > Varnish invocation: > > /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -n vorigin -f > /etc/varnish/vorigin.vcl -T 127.0.0.1:6082 -t 120 -w 2000,8000,300 -p > thread_pools=2 -p thread_pool_workspace=100000 -p > sess_workspace=1000000 -u varnish -g varnish -S /etc/varnish/secret -s > malloc,90G > > We've had a production issue and I've been asserting that varnish > doesn't write anything to disk, so the root volume IO spike we're > seeing CAN'T be varnish. > > Of course, I look into iotop (specifically 'iotop -obPk') I see > periodic times where the output is as follows: > > PID PRIO USER DISK READ DISK WRITE SWAPIN IO COMMAND > 6 be/4 root 0.00 K/s 0.00 K/s 0.00 % 33.26 % [kworker/u128:0] > 4185 be/4 varnish 0.00 K/s 6673.66 K/s 0.00 % 0.00 % varnishd > -P /var/run/varnish.pid -a :80 -n vorigin -f /etc/varnish/vorigin.vcl > -T 127.0.0.1:6082 -t 120 -w 2000,8000,300 -p thread_pools=2 -p > thread_pool_workspace=100000 -p sess_workspace=1000000 -u varnish -g > varnish -S /etc/varnish/secret -s malloc,90G > Total DISK READ: 0.00 K/s | Total DISK WRITE: 0.00 K/s > PID PRIO USER DISK READ DISK WRITE SWAPIN IO COMMAND > 4185 be/4 varnish 0.00 K/s 12659.41 K/s 0.00 % 0.00 % varnishd > -P /var/run/varnish.pid -a :80 -n vorigin -f /etc/varnish/vorigin.vcl > -T 127.0.0.1:6082 -t 120 -w 2000,8000,300 -p thread_pools=2 -p > thread_pool_workspace=100000 -p sess_workspace=1000000 -u varnish -g > varnish -S /etc/varnish/secret -s malloc,90G > > I've dug into it with LSOF and sundry, and I can't figure out > what/where varnish is doing the writing. It's not constant... 10-20 > seconds will pass before I see it again for several seconds (2-8ish). > > NOTE: normal load for these boxes is 3-10k req/sec. > > Any thoughts? Why is varnish writing to disk (more to the point: > writing tens megabytes at a time)? > > varnish version 3.0.5 > Amazon Linux > Kernel 3.14.35-28 > > Thank you for any help; > -Jason _______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
