Re: [ccache] CCACHE_NLEVELS , locking and cache size

[Pierre Tardy]

On Thu, Aug 22, 2013 at 2:18 AM, tal zilcer  wrote:

> hi,
>
> i recently started using ccache and i have a couple of questions.
>
> 1)i did a little experiment with different values of CCACHE_NLEVELS  [..]

 according to the build time results it seems it is better to use low
> NLEVELS value. when should this value be high?
> i dont understand the true meaning of this variable and what is he good
> for.
>

It is rather simple: it spreads the cached object in different folders. So
the overead you have is probably the time needed to create the folder.
The number of level is depending on the overall number of cache files you
want to store.

Each level will divide by 16 the maximum number of file found in the cache
directories.
Depending on the filesystem where you store your cache the time to access a
file can explode when the number of file in a directory is high.

Reiserfs is notoriously good at storing a lot of files in the same
directory. not sure of what happens if you put nfs in the whole story.


> 2)when using the cache for multiple users(about 30 people) i noticed that i
> get much slower compilation time because cache/NUMBER/stats.lock is locked
> a lot.
> is there an option to lock a more "deep" file(when using high NLEVELS
> value)? for instance cache/NUMBER/NUMBER/NUMBER/stats.lock.
> if not is there a solution to this problem?
>

Yeah NFS does not scale too much for such a high load a big ccache database
can generate.

You can try my memcached backend for ccache:
http://permalink.gmane.org/gmane.comp.compilers.ccache/1116

memcached is a real database, so it will scale much better than the
traditional ccache adhoc db + NFS

Regards,
Pierre
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache

[ccache] CCACHE_NLEVELS , locking and cache size

[tal zilcer]

hi,

i recently started using ccache and i have a couple of questions.

1)i did a little experiment with different values of CCACHE_NLEVELS  and i
got the following build time results:
NLEVELS 2: 1m:33s
NLEVELS 4: 1m:45s
NLEVELS 6: 1m:50s
NLEVELS 8: 1m:59s
some info:
a) i always build the same projects from clean cache
b) rebuild of projects(only cache hits) resulted always in 30s build time
c) im using the cache on NFS
d) im using the latest ccache -  version 3.1.9

 according to the build time results it seems it is better to use low
NLEVELS value. when should this value be high?
i dont understand the true meaning of this variable and what is he good for.

2)when using the cache for multiple users(about 30 people) i noticed that i
get much slower compilation time because cache/NUMBER/stats.lock is locked
a lot.
is there an option to lock a more "deep" file(when using high NLEVELS
value)? for instance cache/NUMBER/NUMBER/NUMBER/stats.lock.
if not is there a solution to this problem?

3) my cache size is about 13G(his limit is 50G).
does caching become slower if cache size is big?
do you recommend cleaning it often?

thanks a lot,
tal
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache