Re: [ccache] What to do when there are problems writing to the cache directory?
On 7 February 2012 19:55, Frank Klotz wrote: > [...] inability to write to the stats file should be a fatal error (it will > get attention FAST, and so fixed quickly), but [...] when the stats file is > writable, then placing an indication of the problem there is "not silent", so > it would be acceptable to document the failure in the stats and continue with > a regular compilation that is "otherwise silent". One problem with putting such an error log in the stats files: who will trim the error log from the stats files? The files are rewritten for each cache hit/miss, which means that it's not good if they grow. Another minor but valid problem with the approach is that it's not backward compatible - old ccache versions (which may use the same ccache directory as a new version) throw away anything they don't understand from the stats files. I guess it would be possible to write the log to some other file in the ccache directory, though, unless that's not possible either... > If other people check "ccache -s" as often as I do [...] I doubt many people check "ccache -s" often, I'm afraid. -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] What to do when there are problems writing to the cache directory?
After letting this percolate in my mind for a few weeks, I am now feeling a little more strongly that inability to write to the stats file should be a fatal error (it will get attention FAST, and so fixed quickly), but that when the stats file is writable, then placing an indication of the problem there is "not silent", so it would be acceptable to document the failure in the stats and continue with a regular compilation that is "otherwise silent". If other people check "ccache -s" as often as I do, and the error indicator in that output is at all eye-catching, they will know soon enough that there is a problem to be addressed, and someone with adequate privileges will address it "soon", while others (including perhaps people without the privileges to fix the problem) are able to continue compiling and get successful builds rather than failures. If I were looking at "ccache -s" output and saw a line like: Unable to write to $CCACHE_DIR/a/b/ - permission denied:303 I would certainly know that I needed to fix the permissions on that directory. And with that scheme, I think we could avoid adding any new CCACHE environment variables to control different failure handling options. Thanks, Frank On 01/10/2012 02:45 PM, Frank Klotz wrote: I agree that "tools should not fail silently", but the problem here is the definition of "fail". And that definition is going to depend on context. Ccache is not a compiler, but a compiler wrapper whose purpose is to keep the functionality of the compiler it wraps, but merely speed it up. Under some conditions (e.g., a nightly build where performance is nice, but successful compilation if possible is crucial), getting a compilation failure because the wrapper couldn't do what it wants would be undesirable. For this case, silently falling back to a non-ccache compilation would AVOID a failure, by successfully doing the compile despite wrapper issues. Under other circumstances (e.g., a small interactive build), you would WANT to know right away if there is some issue with the wrapper, so you can fix it immediately and all subsequent builds do get the performance improvement offered by the wrapper. So here silently falling back to a non-ccache compilation would BE a failure. I hate to say it, but I kinda think you need to add another environment variable - CCACHE_FALLBACK - with at least two possible values: hard fail and fallback (there may be other possibilities as well - say "email f...@bar.com", which would say "mail details of the problem to foo, but go ahead and compile the dang file already!" Not actually recommending that, but ) Then users can decide for themselves (on a per-task basis if necessary) whether they care more about getting a successful compile whatever that takes, or about having all compiles be as fast as possible, even if the tool has to fail to compile something it COULD have compiled, in order to get attention on a problem with the tool as fast as possible. Which simply moves your question back a step to "what should be the DEFAULT behaviour for this? My suggestion on that would be: If you can write to the ccache stats file(s) (cases 3 and 4), add an error indication there, and silently fall back to non-ccache compile. If you cannot even write to the stats file (cases 1 and 2), fail. Maybe add a further refinement depending on whether the compile is done in an interactive shell or not - people might want their nightly cronjobs to succeed even if slower than ideal, but interactive builds to fail so they can solve the issue. But I don't have a really strong feeling on any of these. Thanks, Frank On 01/09/2012 01:34 PM, Joel Rosdahl wrote: Hi, I would like to get some feedback on what ccache users think ccache's behavior should be when it fails to create files or directories in the cache directory. Here are some different cases to consider: 1. CCACHE_DIR is set to a directory that doesn't exist and can't be created. 2. CCACHE_DIR is set to a directory that exists but is unwritable. 3. CCACHE_DIR is set to a directory that exists, but one of the subdirectories storing the cache data (e.g. $CCACHE_DIR/a/b) is unwritable or not possible to create. 4. CCACHE_DIR is set to a directory that exists, but CCACHE_TEMPDIR (defaults to $CCACHE_DIR/tmp for ccache 3 and $CCACHE_DIR for ccache 2) is unwritable or not possible to create. Here's what happens with ccache 2.4: 1. Fatal error, non-zero exit code. 2. Silent failure, falling back to running the compiler. 3. Silent failure, falling back to running the compiler. 4. Silent failure, falling back to running the compiler. ccache 3.0-3.1.6 (I think, haven't checked all versions): 1. Fatal error, non-zero exit code. 2. Fatal error, non-zero exit code. 3. Silent failure, falling back to running the compiler. 4. Silent failure, falling back to running the compiler. ccache 3.1.7: 1. Fatal error, non-zero exit code. 2. Fatal error, non-zero exit code. 3. Fatal
Re: [ccache] What to do when there are problems writing to the cache directory?
I agree that "tools should not fail silently", but the problem here is the definition of "fail". And that definition is going to depend on context. Ccache is not a compiler, but a compiler wrapper whose purpose is to keep the functionality of the compiler it wraps, but merely speed it up. Under some conditions (e.g., a nightly build where performance is nice, but successful compilation if possible is crucial), getting a compilation failure because the wrapper couldn't do what it wants would be undesirable. For this case, silently falling back to a non-ccache compilation would AVOID a failure, by successfully doing the compile despite wrapper issues. Under other circumstances (e.g., a small interactive build), you would WANT to know right away if there is some issue with the wrapper, so you can fix it immediately and all subsequent builds do get the performance improvement offered by the wrapper. So here silently falling back to a non-ccache compilation would BE a failure. I hate to say it, but I kinda think you need to add another environment variable - CCACHE_FALLBACK - with at least two possible values: hard fail and fallback (there may be other possibilities as well - say "email f...@bar.com", which would say "mail details of the problem to foo, but go ahead and compile the dang file already!" Not actually recommending that, but ) Then users can decide for themselves (on a per-task basis if necessary) whether they care more about getting a successful compile whatever that takes, or about having all compiles be as fast as possible, even if the tool has to fail to compile something it COULD have compiled, in order to get attention on a problem with the tool as fast as possible. Which simply moves your question back a step to "what should be the DEFAULT behaviour for this? My suggestion on that would be: If you can write to the ccache stats file(s) (cases 3 and 4), add an error indication there, and silently fall back to non-ccache compile. If you cannot even write to the stats file (cases 1 and 2), fail. Maybe add a further refinement depending on whether the compile is done in an interactive shell or not - people might want their nightly cronjobs to succeed even if slower than ideal, but interactive builds to fail so they can solve the issue. But I don't have a really strong feeling on any of these. Thanks, Frank On 01/09/2012 01:34 PM, Joel Rosdahl wrote: Hi, I would like to get some feedback on what ccache users think ccache's behavior should be when it fails to create files or directories in the cache directory. Here are some different cases to consider: 1. CCACHE_DIR is set to a directory that doesn't exist and can't be created. 2. CCACHE_DIR is set to a directory that exists but is unwritable. 3. CCACHE_DIR is set to a directory that exists, but one of the subdirectories storing the cache data (e.g. $CCACHE_DIR/a/b) is unwritable or not possible to create. 4. CCACHE_DIR is set to a directory that exists, but CCACHE_TEMPDIR (defaults to $CCACHE_DIR/tmp for ccache 3 and $CCACHE_DIR for ccache 2) is unwritable or not possible to create. Here's what happens with ccache 2.4: 1. Fatal error, non-zero exit code. 2. Silent failure, falling back to running the compiler. 3. Silent failure, falling back to running the compiler. 4. Silent failure, falling back to running the compiler. ccache 3.0-3.1.6 (I think, haven't checked all versions): 1. Fatal error, non-zero exit code. 2. Fatal error, non-zero exit code. 3. Silent failure, falling back to running the compiler. 4. Silent failure, falling back to running the compiler. ccache 3.1.7: 1. Fatal error, non-zero exit code. 2. Fatal error, non-zero exit code. 3. Fatal error, non-zero exit code. 4. Silent failure, falling back to running the compiler. ccache has previously been more forgiving, so one might argue that ccache 3.x more strict behavior is a regression from version 2.4 that should be fixed. Also, one might argue that ccache should never fail fatally so that case 1 should silently fall back to running the compiler as well. On the other hand, my general opinion is that tools should not fail silently. As a ccache user, I would like to be informed if something is badly configured, for instance if the cache directory is unwritable by me, so that I get alerted that something is wrong. What do you think? -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] What to do when there are problems writing to the cache directory?
On 10 January 2012 05:32, Martin Pool wrote: > I would be inclined to at least give a suppressible warning rather > than just silently falling back. As far as I know, there's no good way of notifying the user other than stopping with a non-zero error code. Just printing a warning message to stderr is not acceptable since there are things like autoconf tests that consider a compiler writing to stderr as non-functional. -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
Re: [ccache] What to do when there are problems writing to the cache directory?
On 9 January 2012 22:34, Joel Rosdahl wrote: > On the other hand, my general opinion is that tools should not fail > silently. As a ccache user, I would like to be informed if something > is badly configured, for instance if the cache directory is unwritable > by me, so that I get alerted that something is wrong. > > What do you think? I agree with the general approach. I suspect a common case where people hit this is something like 'make && sudo make install' which can end up with some files being owned by a different user. In a sense they ought to fix this but perhaps many people would prefer to just have some stuff uncached. I would be inclined to at least give a suppressible warning rather than just silently falling back. -- Martin ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
[ccache] What to do when there are problems writing to the cache directory?
Hi, I would like to get some feedback on what ccache users think ccache's behavior should be when it fails to create files or directories in the cache directory. Here are some different cases to consider: 1. CCACHE_DIR is set to a directory that doesn't exist and can't be created. 2. CCACHE_DIR is set to a directory that exists but is unwritable. 3. CCACHE_DIR is set to a directory that exists, but one of the subdirectories storing the cache data (e.g. $CCACHE_DIR/a/b) is unwritable or not possible to create. 4. CCACHE_DIR is set to a directory that exists, but CCACHE_TEMPDIR (defaults to $CCACHE_DIR/tmp for ccache 3 and $CCACHE_DIR for ccache 2) is unwritable or not possible to create. Here's what happens with ccache 2.4: 1. Fatal error, non-zero exit code. 2. Silent failure, falling back to running the compiler. 3. Silent failure, falling back to running the compiler. 4. Silent failure, falling back to running the compiler. ccache 3.0-3.1.6 (I think, haven't checked all versions): 1. Fatal error, non-zero exit code. 2. Fatal error, non-zero exit code. 3. Silent failure, falling back to running the compiler. 4. Silent failure, falling back to running the compiler. ccache 3.1.7: 1. Fatal error, non-zero exit code. 2. Fatal error, non-zero exit code. 3. Fatal error, non-zero exit code. 4. Silent failure, falling back to running the compiler. ccache has previously been more forgiving, so one might argue that ccache 3.x more strict behavior is a regression from version 2.4 that should be fixed. Also, one might argue that ccache should never fail fatally so that case 1 should silently fall back to running the compiler as well. On the other hand, my general opinion is that tools should not fail silently. As a ccache user, I would like to be informed if something is badly configured, for instance if the cache directory is unwritable by me, so that I get alerted that something is wrong. What do you think? -- Joel ___ ccache mailing list ccache@lists.samba.org https://lists.samba.org/mailman/listinfo/ccache
