On 09.03.2011 11:56, Philip Martin wrote:
Stefan Fuhrmann<eq...@web.de> writes:
On 08.03.2011 14:46, Philip Martin wrote:
stef...@apache.org writes:
Author: stefan2
Date: Mon Mar 7 22:57:04 2011
New Revision: 1079008
URL: http://svn.apache.org/viewvc?rev=1079008&view=rev
Log:
Set FSFS cache default size to 16 MB. This is the same default as
for everybody else, namely the server processes. Thus, it should
be reasonable value on the same machines.
16MB may be "reasonable" for normal usage but it still has a significant
effect on the testsuite. The testsuite is an unusual access pattern, it
runs thousands of commands on small repositories. Using a 1MB cache I
can run the testsuite for ra_local/FSFS in 10m15s, with 16MB cache that
increases to 12m30s, a 20% increase. There is a similar increase when
testing ra_svn/FSFS using the default Linux svnserve (a threaded
svnserve has a shared cache so isn't affected).
The patch attached fixes the performance penalty
for "make check" by disabling membuffer caches
over ra_local by default. Since svn_ra_initialize is
being called early in main(), the setting can later
be overwritten by settings from e.g. a config file.
Could you verify that / whether this fixes your
performance issue?
It goes some way, but it's not as good as setting the cache size to 1MB
since it doesn't affect the svnadmin dump/load commands used by most of
the tests.
I think we should commit that change, then. But I'm not
perfectly sure that the ra_loader initialization code it is
a good place to put it. It seems to be one of the central
places where it could be done consistently.
svnserve and svnadmin both have a -M option, was there any discussion
about using a single letter option for this feature?
No.
This could be used
by the testuite to restrict the cache size, but for ra_local/FSFS
testing there is no way to restrict the svn client.
As already shown in a different thread, the memory
consumption should actually go down for any non-
trivial repository.
Should we add the -M option to the svn client?
Due to its limited applicability (FSFS via file:/// only),
There may well be large numbers of people using that for private
repositories.
With the cache statistics feature we talked about lately
on IRC, we should get reliable data whether ra_local
will benefit from it at all. Having thought about that for
some time now, it seems that even diff and annotate
might not benefit.
The TSVN repository browser, in fact, might be one
of the few applications that causes significant numbers
of cache hits.
As I undertand it cache initialisation includes setting the memory to
zero, and that this is partly to ensure that all the allocated memory is
really available to the process on a Linux system with overcommit
enabled.
Exactly. I wanted to prevent a behavior that looks like
a memory leak. Non-threaded servers should now
be more likely to detect the OOM condition and fall
back to the hopefully smaller 1.6 caching scheme.
If the cache had a more lightweight initialisation then a
cache size of 16MB would probably not be a problem.
The index part needs to be initialized but covers only
1/16th of the whole cache memory. So, yes, no zeroing
the data memory will bring performance back.
Was there any
discussion about this overcommit behaviour? Linux overcommit is
configurable, why should Subversion override this?
It would a service to all these LAMP V-server users
that might not be aware about the overcommit feature
at all.
It doesn't apply to
all the other memory allocation in Subversion, so what guarantee are you
trying to provide and what are you actually providing?
Good point. You got me convinced: the 1.6 caches
will grow slowly while getting filled and without zeroing,
the membuffer cache will apparently do the same.
Committed in r1081082.
-- Stefan^2.
