Re: [ccache] Why not cache compile failures?
On 24 September 2012 10:44, Andrew Stubbs a...@codesourcery.com wrote: On 22/09/12 13:13, Joel Rosdahl wrote: I would be OK with caching failures if it's off by default and doesn't affect the default's performance. Have you thought about how to store the exit code in the cache? The simplest way would be a tiny file in the cache that just contained the code. Say with extension .error. I'd prefer it to be on by default One reason for having it off by default is that it will decrease performance, especially on NFS, to let ccache stat a .error file in the cache for each compilation. Side note: I have previously toyed with an idea to store a compilation result as a single file in the cache instead of up to three different files (as it's done currently) to potentially improve performance (this will make the hardlink mode impossible, however). Another reason is that I feel that there are many subtleties about when a result can be reliably cached, as discussed in this mail thread already. if only to prevent other devs breaking it, but that's your decision to make, of course. That's what automatic test cases are for. :-) Printing an extra warning message would break ccache's transparency, but perhaps that would be acceptable for failures. Have to think more about it. [...] Otherwise, I don't know of any cases where scripts check error/warning messages beyond whether they exist or not. Since I would have it only emit the warning when there are already error messages, it would only break in unusual cases. Right, that sounds reasonable. -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Why not cache compile failures?
On 04/10/12 01:50, Shentino wrote: Would snooping on the compiler's own error messages help weed out transient failures like out of memory, disk full, and so on? I recently compiled webkit-gtk and it managed to exhaust all of my memory AND swap. This does raise an interesting choice! On the one hand, the scheme Martin has suggested for not caching failures due to signals should mean that this result would not be cached (an out-of-memory condition *could* be reported by the compiler as a malloc failure, but overcommit makes it more likely to show up as a SIGTERM/SIGKILL). But on the other hand, if you're not sharing a cache with others, and it's physically impossible to compile a given file on your machine, it might be better to fail instantly rather than kill your machine everytime you run it by mistake! On balance, I prefer the first answer though: not caching memory or disk exhaustion. (Disk full is harder to detect though, as it's not a signal.) Andrew ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Why not cache compile failures?
On Fri, Oct 5, 2012 at 2:17 AM, Andrew Stubbs a...@codesourcery.com wrote: On 04/10/12 01:50, Shentino wrote: Would snooping on the compiler's own error messages help weed out transient failures like out of memory, disk full, and so on? I recently compiled webkit-gtk and it managed to exhaust all of my memory AND swap. This does raise an interesting choice! On the one hand, the scheme Martin has suggested for not caching failures due to signals should mean that this result would not be cached (an out-of-memory condition *could* be reported by the compiler as a malloc failure, but overcommit makes it more likely to show up as a SIGTERM/SIGKILL). It's indeed unpredictable how ccache could be notified of it. And overcommit is configurable at the whim of the end user anyhow, so I'd consider it unwise to make any assumptions about it. IMHO better to be robust and handle both cases as well as possible. But on the other hand, if you're not sharing a cache with others, and it's physically impossible to compile a given file on your machine, it might be better to fail instantly rather than kill your machine everytime you run it by mistake! A good special case, but it's IMHO none of ccache's business what the user is doing with his memory, and most likely a prudent user will be upping his resource allocations before trying again. Or at the very least, failures of this type ought to be cached for shorter periods of time if at all, since the underlying cause is something the user should hopefully be strongly motivated to repair before attempting again. As an example, my system was able to digest the compilation successfully after I fiddled with lvm and upped my swap space to a whopping 64G. After that the compilation cleared successfully. I did have to wait awhile, and I did fall victim to a nasty swap handling bug in the 3.3.8 kernel and have to start over again after upgrading to 3.5.3, but I was eventually able to deal with what made it choke to begin with. A happy medium might be to detect transient failures of this sort and allow an easy way for the user to flush those out of the cache on demand. On balance, I prefer the first answer though: not caching memory or disk exhaustion. (Disk full is harder to detect though, as it's not a signal.) This was indeed my motivation for suggesting that ccache eavesdrop on gcc's stderr output. If the compiler wants to use stderr to bitch about something that puts cacheability in doubt, ccache may as well pay attention. Andrew ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Why not cache compile failures?
On Sat, Sep 29, 2012 at 6:23 PM, Martin Pool m...@sourcefrog.net wrote: I think caching errors would be worth trying. It may help with configure, and it may also help with people rebuilding trees that from time to time have errors in them. Depending on the compiler, it may be possible to see from the waitstatus that it was interrupted, and so to avoid caching the result. Or, perhaps you can do something with process groups to let ccache observe the interrupt signal itself and so be sure not to cache it. If you emit a message when printing a cached error it ought to be safe enough to experiment with. On 29 September 2012 20:07, Shentino shent...@gmail.com wrote: I'd have to agree that caching failures is tricky. Transient errors are temporary by definition and shouldn't be cached. What if compile failures were only cached for a limited amount of time, say, an hour or so? I suppose you are proposing this because you think the cache will often be wrong, Not really. I did mean to add TTL as an attribute to cache entries for compile failures, not for compile successes. It's rather a strange assumption to make about my reasons for suggesting it. Call it (optional?) insurance against a transient failure being cached. and limiting the TTL will limit the damage. But, if the cache is unreliable, adding in an additional time-based factor will make things worse: suppose someone is trying to debug it and things suddenly start working again? More like failures are inherently trickier to cache because of them being more apt to be transient. A success is rather solid. It wouldn't be a small step to imagine the TTL being ignorable for debugging purposes or for whatever other reason. As much as possible we want things to be pure functions of the input, both in the input to ccache, and in how ccache itself behaves. True enough, and I agree wiht that. My suggestion was more how to deal with external interference from transient conditions that likely have nothing to do with said input. The other comment regarding having ccache test for nuisance signals like SIGINT and/or looking for memory or filespace shortage errors would probably help weed out transient failures before they are considered eligible for caching. The vast majority of errors are due to actual errors in the source code in the context of the environment and flags. Including configure scripts testing for features, which would make it worthwile to consider how to cache them without mopping up transient failures caused by things such as resource constraints. If the compiler is intermittently giving errors for other reasons, the user really needs to stabilize them before thinking about using ccache. I agree mostly but it would seem wise to be robust against surprises that the user cannot predict, hopefully without tainting the cache in the process. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Why not cache compile failures?
On Mon, Sep 24, 2012 at 1:44 AM, Andrew Stubbs a...@codesourcery.com wrote: On 22/09/12 13:13, Joel Rosdahl wrote: On 18 September 2012 14:16, Andrew Stubbs a...@codesourcery.com wrote: I'm aware that there's some danger here that we can end up caching Ctrl-C interrupts, SIGTERM/SIGKILL terminations, out-of-memory failures, and all manner of other non-reproducible failures, but these are the unusual case, and nothing that can't be fixed with CCACHE_RECACHE. I would be OK with caching failures if it's off by default and doesn't affect the default's performance. Have you thought about how to store the exit code in the cache? The simplest way would be a tiny file in the cache that just contained the code. Say with extension .error. I'd prefer it to be on by default, if only to prevent other devs breaking it, but that's your decision to make, of course. I might suggest emitting an extra warning message that informs the user that they are seeing a cached failure. Printing an extra warning message would break ccache's transparency, but perhaps that would be acceptable for failures. Have to think more about it. It would definitely break the compiler testsuite, but if you're testing that through ccache you have bigger problems. Otherwise, I don't know of any cases where scripts check error/warning messages beyond whether they exist or not. Since I would have it only emit the warning when there are already error messages, it would only break in unusual cases. Thanks for your feedback, I'll have a bash at it soonish. Andrew ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache I'd have to agree that caching failures is tricky. Transient errors are temporary by definition and shouldn't be cached. What if compile failures were only cached for a limited amount of time, say, an hour or so? Assuming that cache entries are timestamped to allow for LRU purges if the cache overflows, implementing a TTL mechanism to flush transient failures out of the cache might be an easy enhancement. ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Why not cache compile failures?
I think caching errors would be worth trying. It may help with configure, and it may also help with people rebuilding trees that from time to time have errors in them. Depending on the compiler, it may be possible to see from the waitstatus that it was interrupted, and so to avoid caching the result. Or, perhaps you can do something with process groups to let ccache observe the interrupt signal itself and so be sure not to cache it. If you emit a message when printing a cached error it ought to be safe enough to experiment with. On 29 September 2012 20:07, Shentino shent...@gmail.com wrote: I'd have to agree that caching failures is tricky. Transient errors are temporary by definition and shouldn't be cached. What if compile failures were only cached for a limited amount of time, say, an hour or so? I suppose you are proposing this because you think the cache will often be wrong, and limiting the TTL will limit the damage. But, if the cache is unreliable, adding in an additional time-based factor will make things worse: suppose someone is trying to debug it and things suddenly start working again? As much as possible we want things to be pure functions of the input, both in the input to ccache, and in how ccache itself behaves. The vast majority of errors are due to actual errors in the source code in the context of the environment and flags. If the compiler is intermittently giving errors for other reasons, the user really needs to stabilize them before thinking about using ccache. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] Why not cache compile failures?
On 22/09/12 13:13, Joel Rosdahl wrote: On 18 September 2012 14:16, Andrew Stubbs a...@codesourcery.com wrote: I'm aware that there's some danger here that we can end up caching Ctrl-C interrupts, SIGTERM/SIGKILL terminations, out-of-memory failures, and all manner of other non-reproducible failures, but these are the unusual case, and nothing that can't be fixed with CCACHE_RECACHE. I would be OK with caching failures if it's off by default and doesn't affect the default's performance. Have you thought about how to store the exit code in the cache? The simplest way would be a tiny file in the cache that just contained the code. Say with extension .error. I'd prefer it to be on by default, if only to prevent other devs breaking it, but that's your decision to make, of course. I might suggest emitting an extra warning message that informs the user that they are seeing a cached failure. Printing an extra warning message would break ccache's transparency, but perhaps that would be acceptable for failures. Have to think more about it. It would definitely break the compiler testsuite, but if you're testing that through ccache you have bigger problems. Otherwise, I don't know of any cases where scripts check error/warning messages beyond whether they exist or not. Since I would have it only emit the warning when there are already error messages, it would only break in unusual cases. Thanks for your feedback, I'll have a bash at it soonish. Andrew ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
