Hello, I see that varnish 4.0.2 does not free duplicate objects when ttl is expired but grace is not expired. Every time when ttl expired it will fetch a new object and increase memory usage. I set ttl to 1ms and grace to 1 day and did stress test. Varnish memory usage continued to increase and soon exceeded the max memory limit, but remained stable at a point.
I made a testing patch on 4.0.2 source release and tried to purge all expired matching objects except the most recent one during hash lookup, so the memory of duplicate graced objects will be freed. Any comments? Thanks, Jingyi Wei
From 3c7b83a2919f361d990d7aafe2b487f6cccd0a3c Mon Sep 17 00:00:00 2001 From: Jingyi Wei <[email protected]> Date: Sun, 14 Dec 2014 01:05:04 -0500 Subject: [PATCH] Purge all expired matching objects except the most recent one during hash lookup. The memory of duplicate graced objects will be freed. --- bin/varnishd/cache/cache_hash.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c index 0a94a3d..0811926 100644 --- a/bin/varnishd/cache/cache_hash.c +++ b/bin/varnishd/cache/cache_hash.c @@ -444,10 +444,22 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp, } if (o->exp.t_origin > exp_t_origin && !(oc->flags & OC_F_PASS)) { + VSLb(req->vsl, SLT_Debug, "Object recent: ttl %.3f", + EXP_Ttl(req, o) - req->t_prev); + /* free the previous object */ + if (exp_o) { + VSLb(req->vsl, SLT_Debug, "Object recent purged"); + EXP_Rearm(exp_o, exp_o->exp.t_origin, 0, 0, 0); // Fake now + } /* record the newest object */ exp_oc = oc; exp_o = o; exp_t_origin = o->exp.t_origin; + } else if (!(oc->flags & OC_F_PASS)) { + VSLb(req->vsl, SLT_Debug, "Object old purged: ttl %.3f", + EXP_Ttl(req, o) - req->t_prev); + /* free the current object */ + EXP_Rearm(o, o->exp.t_origin, 0, 0, 0); // Fake now } } -- 1.7.9.5
_______________________________________________ varnish-dev mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
