Re: [ccache] why is limit_multiple ignored?

[Joel Rosdahl via ccache]

Hi Scott,

On 19 December 2017 at 02:16, Scott Bennett via ccache <
ccache@lists.samba.org> wrote:

>  I set "limit_multiple = 0.95" in ccache.conf and "max_size = 30.0G"
> in ccache.conf, but cleanups are triggered when space usage reaches 24 GB,
> which is the default of 0.8.  Why is this happening with ccache 3.3.4?
>

The ccache manual is not very good at describing what actually happens at
cleanup. I'll try to improve it.

Here's how cleanup works: After a cache miss, ccache stores the object file
in (a subdirectory of) one of the 16 top level directories in the cache
(0-9, a-f). It then checks if that top level directory holds more than
max_cache_size/16 bytes (and similar for max_files). If yes, ccache removes
files from that top level directory until it contains at most
limit_multiple*max_cache_size/16 bytes. This means that if limit_multiple
is 0.8, the total cache size is expected to hover around 0.9*max_cache_size
when it has filled up. But due to the pseudo-randomness of the hash
algorithm, the cache size can be closer to 0.8*max_cache_size or
1.0*max_cache_size.

The above should be true for any serial usage of ccache. However, ccache is
of course very often called in parallel, and then there is a race condition
since several ccache processes that have stored an object to the same top
level directory may start the cleanup process simultaneously. Since
performing cleanup in a large cache with a low limit_multiple can take a
lot of time, more ccache processes may start to perform cleanup of the same
directory. The race can lead to the final cache size being below
limit_multiple*max_cache_size, perhaps very much so. This is a known
problem. We have had some ideas to improve the admittedly naive cleanup
logic, but nothing has been done yet.

Maybe the above described problem is why you get a 24 GB cache size?

Or maybe you ran "ccache -c"? Unlike what the manual indicates, "ccache -c"
will delete files until each top level directory holds at most
limit_multiple*max_size/16...

why is limit_multiple ignored?


It isn't. Or don't you see a difference if you e.g. set it to 0.5?

-- Joel
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache

Re: [ccache] base_dir and symbolic links

[Joel Rosdahl via ccache]

On 2 January 2018 at 13:49, Andreas Wettstein via ccache <
ccache@lists.samba.org> wrote:

> Hello Joel,
>
> > https://www.mail-archive.com/ccache@lists.samba.org/msg00802.html
>
> Thank you.  It is trickier than I thought.
>
> > ccache has a test suite and it fails like this with your patch:
>
> Sorry, I missed the test directory.  Attached is a modified patch, which
> fixes this issue and includes tests as well.
>

Thanks, but that unfortunately doesn't normalize "../" parts like the
current code with realpath does. For example, the following change to the
test suite makes it fail with your proposal:

--- a/test.sh
+++ b/test.sh
@@ -2355,8 +2355,8 @@ SUITE_basedir() {
 expect_stat 'cache miss' 1

 # Rewriting triggered by CCACHE_BASEDIR should handle paths with
multiple
-# slashes correctly:
-CCACHE_BASEDIR=`pwd` $CCACHE_COMPILE -I`pwd`//include -c
`pwd`//src/test.c
+# slashes, redundant "/." parts and "foo/.." parts correctly:
+CCACHE_BASEDIR=`pwd` $CCACHE_COMPILE -I`pwd`//./include/../include -c
`pwd`/src/test.c
 expect_stat 'cache hit (direct)' 1
 expect_stat 'cache hit (preprocessed)' 0
 expect_stat 'cache miss' 1


I would prefer not to lose that ability.

-- Joel
___
ccache mailing list
ccache@lists.samba.org
https://lists.samba.org/mailman/listinfo/ccache