Re: [ccache] Shared cache (Ubuntu 14, VMWare, TeamCity, gcc)

[Pierre Tardy]

Hi Mats,

There is a memcached branch here: https://github.com/ccache/ccache/pull/58
I've been running a previous version of this for years with great success,
and you can see that others also had success with it.
Event for 16 agent, you will probably begin to see performance improvment
compared to NFS single point of failure.

At the moment, it is kept as a separate branch as we haven't figured out a
way to cleanly disable it for people who do not want it while still being
maximum performance for people who want it. See the PR discussion.

BR
Pierre


Le lun. 3 oct. 2016 à 15:56, Mats Nilsson  a écrit :

> Hi,
>
> I'm a newcomer and I'm just about to start some tests with ccache and I'm
> unsure of how to arrange a shared cache for my builds in TeamCity.
>
> We usually compile the same git commit in several ways (eg. shared or
> unshared libs) and with several cross compilers on our 16 build agents.
>
> I wonder if the shared cache needs to be on a NFS-fileserver or if it's
> better to use a shared virtual disk?
>
> BR,
>
> /Mats
>
> ___
> ccache mailing list
> ccache@lists.samba.org
> https://lists.samba.org/mailman/listinfo/ccache
>
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache

Re: [ccache] Using ccache with memcached

[Pierre Tardy]

>
>
> i don't think getting rid out of the fs makes sense, but having memcache
> be available dynamically as an additional layer sounds fine.
>

It does make a lot of sense for me as I have a high performance network,
which is faster than local harddrive. So I would insist on keeping an
option for memcached only.

Regards,
Pierre
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache

Re: [ccache] CCACHE_NLEVELS , locking and cache size

[Pierre Tardy]

On Thu, Aug 22, 2013 at 2:18 AM, tal zilcer tal.zil...@gmail.com wrote:

 hi,

 i recently started using ccache and i have a couple of questions.

 1)i did a little experiment with different values of CCACHE_NLEVELS  [..]

 according to the build time results it seems it is better to use low
 NLEVELS value. when should this value be high?
 i dont understand the true meaning of this variable and what is he good
 for.


It is rather simple: it spreads the cached object in different folders. So
the overead you have is probably the time needed to create the folder.
The number of level is depending on the overall number of cache files you
want to store.

Each level will divide by 16 the maximum number of file found in the cache
directories.
Depending on the filesystem where you store your cache the time to access a
file can explode when the number of file in a directory is high.

Reiserfs is notoriously good at storing a lot of files in the same
directory. not sure of what happens if you put nfs in the whole story.


 2)when using the cache for multiple users(about 30 people) i noticed that i
 get much slower compilation time because cache/NUMBER/stats.lock is locked
 a lot.
 is there an option to lock a more deep file(when using high NLEVELS
 value)? for instance cache/NUMBER/NUMBER/NUMBER/stats.lock.
 if not is there a solution to this problem?


Yeah NFS does not scale too much for such a high load a big ccache database
can generate.

You can try my memcached backend for ccache:
http://permalink.gmane.org/gmane.comp.compilers.ccache/1116

memcached is a real database, so it will scale much better than the
traditional ccache adhoc db + NFS

Regards,
Pierre
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache