On 12/01/2011 09:55 PM, Simo Sorce wrote:
On Wed, 2011-11-30 at 17:33 -0500, John Dennis wrote:
Comments? Suggestions?
Sorry for the late reply.
First of all, excellent write-up John, it is very comprehensive and lays
down things very clearly.
Thank you.
I agree that using ipa:ipa for memcached and wsgi would be a better
proposition for us. Although we need to explore how this would affect
credential caches created by mod_auth_kerb and our ability to use them,
which is crucial*.
You say that object sized for the stuff we will store in memcached
should be limited. What is a reasonable size for those objects ?
I was thinking we may want to store the krb ccaches in memcached in
order to be able to keep them around. The reason I ask is that Krb
ccaches can become quite big if PACs are attached to tickets although
they are normally quite small.
I had to do a little digging to get the answer.
The maximum size of a single entry is 1 MB. That includes some metadata
bound to the item for internal use, but it isn't a lot, so technically a
single item's max size is just under one megabyte. This limit can be
increased but the memcached developers do not recommend it (I believe
because it reduces memory usage efficiency due to page hits).
By default the memcached daemon initializes with 64 MB of total memory
the vast majority of which is allocated to item storage and a minor
amount allocated to bookkeeping. The total memory size is easily
adjusted by a command line parameter when the daemon initializes.
In addition the Python library supports compression using zlib. You
specify a minimum compression length. The value is converted into it's
storage form (python objects other than integers and strings are
"pickled") and if the length of the value in it's final storage form is
greater than the minimum compression length zib is invoked to compress
it. This way zlib compression is avoided for small items (or compression
can be completely disabled). Thus we have the option to play around with
compressing our entries if we discover we have some need to. I think
initially we would disable compression sacrificing memory consumption in
favor of performance.
My guess is we're not likely to be bumping up against the 1 MB per item
threshold (nor would it be smart to anywhere be close to that). I think
I recalled you mentioning that PAC data would max out around 16 KB. So I
don't see the limit as being something we realistically need to worry
about (or at least I hope not :-)
One nice feature of memcached is you can query the statistics from a
live daemon at any time. While we're tuning we might need to do this.
Aside for these minor details I totally agree with the direction you are
proposing and I can't wait to see it implemented :)
Simo.
*In the long term we may even decide to stop using mod_auth_kerb and do
our own handling if this simplifies things, but I guess we will need an
interim transition period in any case, because we can't depend on too
many changes to be done at once as a dependency to introduce sessions.
--
John Dennis <jden...@redhat.com>
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
_______________________________________________
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel
