> 966403669 . . SMA outstanding bytes There's your 1GB cache buffer, which is what you're setting. Obviously, Varnish needs memory to do other things, so you'll see a fixed couple of hundred extra megs. Also, each thread (e.g., worker) will allocate its own memory, including a stack. This memory usage will scale and contract as load spawns and reaps threads. This is true of any server, especially examples such as Squid and MySQLd.
Linux (including 10.04) sets a default stack size of 8MB: -s: stack size (kbytes) 8192 With 500 threads, you could in theory see up to 4GB of VIRT. Run "pmap -x PID" on the child process (or grandchildren), and you will see where the memory goes. It should reflect the above -- you'll see 500+ 8MB stacks, plus a lot of other 1MB allocations, which is the block size used by jemalloc IIRC. If you really only want to use 1G, specify something like "malloc,800M". Hope it helps, -- Ken On Jun 9, 2010, at 5:29 PM, Michael Lenaghan wrote: > What a way to start! Obviously, I accidentally hit "send." Let's try > this again... > > I'm running Varnish 2.1 on Ubuntu 10.04. We ran a test to purposely > fill Varnish's cache. We've defined that cache as "malloc,1G". After > running our test, "top" shows: > > 1486m VIRT memory > 1.3g RES memory > 81m SHR memory > > Either I've mis-configured something or mis-understood something. > Either way, help is appreciated. > > In case it's of use the output of varnishstat follows. > > 0+05:22:49 > domU-12-31-38-00-9D-27 > Hitrate ratio: 10 100 1000 > Hitrate avg: 0.0381 0.0813 0.0705 > > 3889 0.00 0.20 Client connections accepted > 227735 0.99 11.76 Client requests received > 10916 0.00 0.56 Cache hits > 215592 0.00 11.13 Cache misses > 211160 0.00 10.90 Backend conn. success > 12 0.00 0.00 Backend conn. failures > 5646 0.99 0.29 Backend conn. reuses > 211144 0.00 10.90 Backend conn. was closed > 216791 0.99 11.19 Backend conn. recycles > 4575 0.99 0.24 Fetch with Length > 212231 0.00 10.96 Fetch chunked > 31 . . N struct sess_mem > 2 . . N struct sess > 205011 . . N struct object > 205014 . . N struct objectcore > 205014 . . N struct objecthead > 1 . . N struct vbe_conn > 500 . . N worker threads > 500 0.00 0.03 N worker threads created > 1259710 0.00 65.04 N worker threads limited > 2 . . N backends > 10569 . . N expired objects > 4425 . . N LRU moved objects > 216589 0.99 11.18 Objects sent with write > 3889 0.00 0.20 Total Sessions > 227735 0.99 11.76 Total Requests > 1226 0.99 0.06 Total pass > 216806 0.99 11.19 Total fetch > 97606086 221.79 5039.29 Total header bytes > 833650072 2.97 43040.43 Total body bytes > 13 0.00 0.00 Session Closed > 231611 0.99 11.96 Session herd > 21199614 59.41 1094.51 SHM records > 697327 2.97 36.00 SHM writes > 38 0.00 0.00 SHM MTX contention > 8 0.00 0.00 SHM cycles through buffer > 432162 0.99 22.31 SMA allocator requests > 410022 . . SMA outstanding allocations > 966403669 . . SMA outstanding bytes > 27988956353 . . SMA bytes allocated > 27022552684 . . SMA bytes free > 13 0.00 0.00 SMS allocator requests > 3670 . . SMS bytes allocated > 3670 . . SMS bytes freed > 216806 0.99 11.19 Backend requests made > 1 0.00 0.00 N vcl total > 1 0.00 0.00 N vcl available > 1 . . N total active purges > 1 0.00 0.00 N new purges added > 19369 1.98 1.00 Client uptime > > _______________________________________________ > varnish-misc mailing list > [email protected] > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc _______________________________________________ varnish-misc mailing list [email protected] http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
