On Mon, Mar 2, 2020 at 11:15 AM Yassine Aouadi <[email protected]> wrote: > > > Hello, > > I am sending my Varnish log to to remote SAAS solution I and want to > improve logs costs by implementing a server side sampling solution . > > First I splitted Varnishnncsa into two service one for error logs and the > other for acces logs : > > CGroup: /system.slice/varnishncsa-error.service > └─18458 /usr/bin/varnishncsa -c -b -a -w > /var/log/varnish/varnishncsa-error.log -D -P > /run/varnishncsa/varnishncsa-error.pid -f > /etc/varnish/varnishncsa_logmatic.format -q *Status > 399 > > CGroup: /system.slice/varnishncsa.service > └─18347 /usr/bin/varnishncsa -c -b -a -w > /var/log/varnish/varnishncsa-access.log -D -P > /run/varnishncsa/varnishncsa-access.pid -f > /etc/varnish/varnishncsa_logmatic.format -q *Status < 400 > > > Is there Any way to go further with varnishncsa and perform and random > sampling of my access logs ? for example write only 10 % of access logs > > If it's not possible with varnishncsa any Suggestion ? I tried rsyslog > random sampling but I am facing memory leaks while stress testing server > with high load
Hi, I think the closest to what you want is rate limiting, see the documentation for the varnishstat -R option. Otherwise you can always do the sampling one step downstream and instead of sending varnishncsa-<something>.log whenever logrotate triggers a rotation you run script that sends 1 line every 10 lines. But I think rate limiting with -R is simpler and instead of a percentage that depends highly on your traffic you can actually get a limit according to a budget since you wish to reduce costs. Dridi _______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
