On 11/02/2012 15:43, Amos Jeffries wrote:
On 11/02/2012 8:34 a.m., Pieter De Wit wrote:
Hi Guys,
So I saw on the mailing list the question about quotas came up again.
I thought I would give it a shot (planning etc) and I was wondering,
does "Squid" have a way to store data ?
In Quota you will have Bandwidth (bytes, not per second) that has to
be checked/adjusted and updated to disk. Instead of writing my own
routines to store this in my "own" format, it would be nice to have
something that grows with Squid.
Discard the idea of disk formats for quota control. You are dealing
with individual packet read/writes at this level. Everything of
importance needs to be in RAM. Squid has a client_db memory cache
which stores statistics and details about each unique client. Several
transaction state controllers (ConnStateData, TunnelStateData,
*ServerData) already interact with that for per-client bandwidth
reporting.
At the more abstracted level, semi-accurate quotas would need the
client_db to be backed up on disk or somewhere periodically. (oh yay,
yet another event to cause "squid keeps pausing" bugs).
Or alternatively, the design I worked out a years or so ago uses a
helper process to query some database or system managing client
quotas. There is a lot of interest in RADIUS and ActiveDirectory
backends controlling this type of thing, and a helper interface is
much more flexible than pre-determined config formats. You can query
this helper on each new client to receive a quota value which gets
used up then re-checked. We still get some slowdown from the lookups,
but it is up to the admin to configure how much quota bytes the helper
requests each cycle and thus how much overhead they add.
Amos
Hi Amos,
Thanks - I second the idea of a helper process, I like the "milter" type
idea where this can run on another box, away from Squid. I have been
messing around with ICAP to do this, what are your thoughts on that ?
IMHO, the time part of quotas should be handled by ACL's, it is
impossible for Squid to know how long someone has been on the net.
Back to the "sync" of client_db. I agree that the working set of the DB
should be in memory, but would a threaded approach slow squid down "that
much" ? I am thinking along the lines of a pthread that just sits there
and lazy writes dirty objects to disk. A quick lock of the global mutex,
copy the object, unlock mutex, write object, rinse repeat. There will
also have to be a shutdown hook...the helper process is looking pretty
good round about now :)
For some reason, version control systems and I have never been friends.
I can't get a copy of the source code using BZR (using the instructions
on the Squid wiki), so I will build against the latest .tar.gz that I
can find. I also suspect this will be new files and very little
modifying of current files.
Cheers,
Pieter
