Re: [CMake] Cannot restore timestamp error on Windows
Hi, I read the content of the change in the link you provided, and I have a newbie question: Why not using one timestamp (different filename) per library/binary/project ? Looking forward to having this issue solved in the next official release! Best, Raffi Enficiaud -- View this message in context: http://cmake.3232098.n2.nabble.com/Cannot-restore-timestamp-error-on-Windows-tp7584120p7584273.html Sent from the CMake mailing list archive at Nabble.com. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On 04/29/2013 10:53 AM, Raffi Enficiaud wrote: I read the content of the change in the link you provided, and I have a newbie question: Why not using one timestamp (different filename) per library/binary/project? The current approach evolved historically. A per-target approach would probably work too but things should be working now anyway. Looking forward to having this issue solved in the next official release! It is already solved in 2.8.11-rc*. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
Hi Brad,
Thank you for your reply.
As you pointed out, the problem seems to come from a race condition on the
file generate.stamp. Now the operations being atomic, the problem should
be solved, in practice.
However, I do not fully understand in what extent the atomicity of renaming
a file would solve this. The log says:
Write the stamp file to a random temporary name and then atomically rename
it to the real stamp file.
The operations take basically less time; hence the problem should occur
less. However:
- process 1 is accessing generate.stamp for reading
- process 2 is moving some tmpfile to generate.stamp - race
The probability of race increases with the number of projects in
CMakeList.txt (since they share generate.stamp) and the number of cores
used for building (and of course the structure and dependencies among the
projects).
What if, in .\Source\cmLocalVisualStudio7Generator.cxx, we replace:
cmSourceFile* cmLocalVisualStudio7Generator::CreateVCProjBuildRule()
{
std::string stampName = this-Makefile-GetCurrentOutputDirectory();
stampName += /;
stampName += cmake::GetCMakeFilesDirectoryPostSlash();
stampName += generate.stamp;
const char* dsprule =
this-Makefile-GetRequiredDefinition(CMAKE_COMMAND);
by
cmSourceFile* cmLocalVisualStudio7Generator::CreateVCProjBuildRule()
{
std::string stampName = this-Makefile-GetCurrentOutputDirectory();
stampName += /;
stampName += cmake::GetCMakeFilesDirectoryPostSlash();
stampName += std::string(this-Makefile-GetProjectName()) +
_generate.stamp;
const char* dsprule =
this-Makefile-GetRequiredDefinition(CMAKE_COMMAND);
I do not know the code that much, but I guess Makefile::SetProjectName is
called much earlier than
cmLocalVisualStudio7Generator::CreateVCProjBuildRule, right?
What do you think?
Best,
Raffi Enficiaud
-Original Message-
From: Brad King [mailto:brad.k...@kitware.com]
Sent: lundi 29 avril 2013 18:56
To: Raffi Enficiaud
Cc: cmake@cmake.org
Subject: Re: [CMake] Cannot restore timestamp error on Windows
On 04/29/2013 10:53 AM, Raffi Enficiaud wrote:
I read the content of the change in the link you provided, and I have
a newbie question:
Why not using one timestamp (different filename) per
library/binary/project?
The current approach evolved historically. A per-target approach would
probably work too but things should be working now anyway.
Looking forward to having this issue solved in the next official release!
It is already solved in 2.8.11-rc*.
-Brad
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On 04/29/2013 04:37 PM, Raffi Enficiaud wrote: - process 1 is accessing generate.stamp for reading Nothing ever reads the file. Only its existence and modification time matter. - process 2 is moving some tmpfile to generate.stamp - race The only race was for multiple processes simultaneously deciding they need to create the (currently missing) file and then trying to open the file for write at the same time. That race is gone because replacement is now atomic. stampName += std::string(this-Makefile-GetProjectName()) CMake has a different meaning for project than .vcxproj so this will still be the same name for all targets in a directory. Each .vcxproj file corresponds to what CMake calls a target. In order to use a different stamp file for each .vcxproj file then the custom command produced by CreateVCProjBuildRule would not be able to be re-used across multiple targets. Instead each generator would have to hand-code the custom command for its own .vcxproj file. While possible it is a lot of work and this bug has already been solved. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
The only race was for multiple processes simultaneously deciding they need
to create the (currently missing) file and then trying to open the file for
write at the same time.
To what I see on the logs, this is not what I observe:
build 26-avr.-2013 13:11:12 CMake does not need to re-run because
XXX\tmp\src\\CMakeFiles\generate.stamp is up-to-date.
build 26-avr.-2013 13:11:12 Building Custom Rule CCC/CMakeLists.txt
build 26-avr.-2013 13:11:12 Building Custom Rule CCC/CMakeLists.txt
build 26-avr.-2013 13:11:12 Building Custom Rule CCC/CMakeLists.txt
build 26-avr.-2013 13:11:12 CMake does not need to re-run because
XXX\tmp\src\\CMakeFiles\generate.stamp is up-to-date.
then
build 26-avr.-2013 13:11:24 Building Custom Rule CCC/CMakeLists.txt
build 26-avr.-2013 13:11:24 CUSTOMBUILD : CMake error : Cannot restore
timestamp XXX\tmp\src\\CMakeFiles\generate.stamp
[XXX\tmp\src\\SSS.vcxproj]
The file XXX\tmp\src\\CMakeFiles\generate.stamp already existed long
before the error, and many projects were built between. But if by missing
you are saying that it is missing because moved to another filename (I am a
bit ignorant of how it works), then we still have a race condition.
CMake has a different meaning for project than .vcxproj so this will
still be the same name for all targets in a directory.
Each .vcxproj file corresponds to what CMake calls a target.
Indeed, sorry for that mistake.
That race is gone because replacement is now atomic
This is true for unices, but does not seem to be true on Windows. There is a
fallback that tries to move the file 5 times before triggering an error.
This is not atomic.
In order to use a different stamp file for each .vcxproj file then the
custom command produced by CreateVCProjBuildRule would not be able to be
re-used across multiple targets. Instead each generator would have to
hand-code the custom command for its own .vcxproj file.
While possible it is a lot of work and this bug has already been solved.
True, and more true now I have a closer look at the code. I do not
understand why it was so (historically), and not understanding that does not
help me to provide you a patch. For instance, in
cmLocalVisualStudio10Generator::Generate(), all vcxproj are generated, and
then one unique timestamp file is written, hence for the current
CMakeLists.txt. If I understand well, this is the only location (VC10) where
the timestamp is written/made dirty.
cmLocalVisualStudio7Generator::CreateVCProjBuildRule() adds a rule for
checking the timestamp, but I do not understand why this is not a per-target
custom build rule in cmLocalVisualStudio7Generator::AddCMakeListsRules()
(this could easily be done in cmLocalVisualStudio10Generator::Generate()),
nor to what entity the this-Makefile-AddCustomCommandToOutput (in
cmLocalVisualStudio7Generator::CreateVCProjBuildRule() ) refers to.
But if we have something like:
void cmLocalVisualStudio7Generator::AddCMakeListsRules()
{
cmTargets tgts = this-Makefile-GetTargets();
// Create the regeneration custom rule.
if(!this-Makefile-IsOn(CMAKE_SUPPRESS_REGENERATION))
{
// HERE: just returning the files that should be included
if(cmSourceFile* sf = this-Get_CMAKELISTFILENAME())
{
// Add the rule to targets that need it.
for(cmTargets::iterator l = tgts.begin(); l != tgts.end(); ++l)
{
if(l-first != CMAKE_CHECK_BUILD_SYSTEM_TARGET)
{
l-second.AddSourceFile(sf);
// HERE adding the custom build rule checking the timestamps
l-ADD_CUSTOM_BUILD_RULE_WITH_STAMPS();
}
this should be closer to what I thought initially.
For an automatic build daemon, I think setting CMAKE_SUPPRESS_REGENERATION
should be sufficient workaround (until the next true release with the
proposed workaround).
Best,
Raffi Enficiaud
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On Sun, Apr 21, 2013 at 12:32 AM, Paul Smith p...@mad-scientist.net wrote: I looked up this message and found another report of this error which was purported to be fixed in cmake by this patch: commit 2dc17f88dd2de900154f153f521b803ec9b7c377 Author: Brad King brad.k...@kitware.com Date: 2013-02-12 10:46:22 -0500 For reference, here is a link to that change: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=2dc17f88 So, I've built a new cmake from 2.8.10 and cherry-picked that patch and I'm using that cmake now. However, even so I'm still seeing the above error. The above change fixed the problem only because it was based on a CMake that also included a fix to RenameFile: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=59b568e5 Please try 2.8.11-rc3 which has both changes. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On Tue, 2013-04-23 at 21:50 -0400, Bill Hoffman wrote: On 4/23/2013 8:57 PM, J Decker wrote: I've seen this also, and it is intermittant; once upon a time there were several AV programs that held new files open too long, so subsequent accesses would fail... sorry to be no help Can you try the current RC and see if it is fixed in that version of CMake? I installed it and it ran one time and I didn't see the error, but it's intermittent so that's not definitive. Unfortunately one of my cmake files failed (later on; this is a cmake file from a smaller sub-project): CMake Error at CMakeLists.txt:190 (TARGET_LINK_LIBRARIES): Error evaluating generator expression: $TARGET_PROPERTY:$$CONFIG:DEBUG:php5ts_debug,INTERFACE_INCLUDE_DIRECTORIES $TARGET_PROPERTY:tgt,prop expression requires a non-empty target name. I have to back out the upgrade until I can figure out what this error means: there's no reference to INTERFACE_INCLUDE_DIRECTORIES anywhere in my CMakeLists.txt and this project is not one I'm familiar with. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
I installed it and it ran one time and I didn't see the error, but it's intermittent so that's not definitive. Unfortunately one of my cmake files failed (later on; this is a cmake file from a smaller sub-project): CMake Error at CMakeLists.txt:190 (TARGET_LINK_LIBRARIES): Error evaluating generator expression: $TARGET_PROPERTY:$$CONFIG:DEBUG:php5ts_debug,INTERFACE_INCLUDE_DIRECTORIES $TARGET_PROPERTY:tgt,prop expression requires a non-empty target name. I have to back out the upgrade until I can figure out what this error means: there's no reference to INTERFACE_INCLUDE_DIRECTORIES anywhere in my CMakeLists.txt and this project is not one I'm familiar with. I have seen this as well with the 2.8.11-rc releases. I am not exactly what causes that. John -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On Wed, 2013-04-24 at 09:02 -0400, John Drescher wrote: I installed it and it ran one time and I didn't see the error, but it's intermittent so that's not definitive. Unfortunately one of my cmake files failed (later on; this is a cmake file from a smaller sub-project): CMake Error at CMakeLists.txt:190 (TARGET_LINK_LIBRARIES): Error evaluating generator expression: $TARGET_PROPERTY:$$CONFIG:DEBUG:php5ts_debug,INTERFACE_INCLUDE_DIRECTORIES $TARGET_PROPERTY:tgt,prop expression requires a non-empty target name. I have to back out the upgrade until I can figure out what this error means: there's no reference to INTERFACE_INCLUDE_DIRECTORIES anywhere in my CMakeLists.txt and this project is not one I'm familiar with. I have seen this as well with the 2.8.11-rc releases. I am not exactly what causes that. The package I'm trying to build is here, FWIW: https://github.com/nuodb/nuodb-php-pdo The CMakeLists.txt file is here: https://github.com/nuodb/nuodb-php-pdo/blob/master/CMakeLists.txt -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On Wed, 2013-04-24 at 09:11 -0400, Paul Smith wrote: I have seen this as well with the 2.8.11-rc releases. I am not exactly what causes that. The package I'm trying to build is here, FWIW: https://github.com/nuodb/nuodb-php-pdo The CMakeLists.txt file is here: https://github.com/nuodb/nuodb-php-pdo/blob/master/CMakeLists.txt Should I file a bug about this? Or is this an expected change in 2.8.11? Maybe the CMakeLists.txt file is incorrect in a way that was previously ignored? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On Sun, 2013-04-21 at 00:32 -0400, Paul Smith wrote: Hi all; I'm getting this error most, but not every, time I run cmake on my main Windows build system: 28CUSTOMBUILD : CMake error : Cannot restore timestamp D:\build-dir\MASTER-BRANCHES30-WINDOWS\BaseTest\CMakeFiles\generate.stamp It happens for different targets as well, not always the same one. commit 2dc17f88dd2de900154f153f521b803ec9b7c377 Author: Brad King brad.k...@kitware.com Date: 2013-02-12 10:46:22 -0500 We're still seeing this, pretty often. This is a very serious problem for us as it's causing spurious build failures. I've looked into the code and added debugging. For example if the RenameFile() fails, I've printed the system error. The error always appears to be No such file or directory. I've also added a check on the open of the temp file so that if it fails, I print the reason: previously there was no check for that. However, that code never executes so it seems that's not the problem. Currently my suspicion is that the name of the temp file is not unique enough: somehow multiple concurrent runs of this function are using the same temp file name. I've added output to print the temp file name always, not just when a failure happens, but it will take a bit of time to get into our build system. Anyone have any other ideas? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
do you have antivirus software that you can disable? On Tue, Apr 23, 2013 at 2:43 PM, Paul Smith p...@mad-scientist.net wrote: On Sun, 2013-04-21 at 00:32 -0400, Paul Smith wrote: Hi all; I'm getting this error most, but not every, time I run cmake on my main Windows build system: 28CUSTOMBUILD : CMake error : Cannot restore timestamp D:\build-dir\MASTER-BRANCHES30-WINDOWS\BaseTest\CMakeFiles\generate.stamp It happens for different targets as well, not always the same one. commit 2dc17f88dd2de900154f153f521b803ec9b7c377 Author: Brad King brad.k...@kitware.com Date: 2013-02-12 10:46:22 -0500 We're still seeing this, pretty often. This is a very serious problem for us as it's causing spurious build failures. I've looked into the code and added debugging. For example if the RenameFile() fails, I've printed the system error. The error always appears to be No such file or directory. I've also added a check on the open of the temp file so that if it fails, I print the reason: previously there was no check for that. However, that code never executes so it seems that's not the problem. Currently my suspicion is that the name of the temp file is not unique enough: somehow multiple concurrent runs of this function are using the same temp file name. I've added output to print the temp file name always, not just when a failure happens, but it will take a bit of time to get into our build system. Anyone have any other ideas? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On Tue, 2013-04-23 at 15:54 -0700, J Decker wrote: do you have antivirus software that you can disable? No, there's no special antivirus software on these build servers. It's hard to imagine what kind of issue antivirus software would cause, that would be so intermittent. Our builds perform this generate.stamp operation over 100 times per build, according to our logs, and only once in every build or two does it cause a problem. Cheers! -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
I've seen this also, and it is intermittant; once upon a time there were several AV programs that held new files open too long, so subsequent accesses would fail... sorry to be no help On Tue, Apr 23, 2013 at 5:34 PM, Paul Smith p...@mad-scientist.net wrote: On Tue, 2013-04-23 at 15:54 -0700, J Decker wrote: do you have antivirus software that you can disable? No, there's no special antivirus software on these build servers. It's hard to imagine what kind of issue antivirus software would cause, that would be so intermittent. Our builds perform this generate.stamp operation over 100 times per build, according to our logs, and only once in every build or two does it cause a problem. Cheers! -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] Cannot restore timestamp error on Windows
On 4/23/2013 8:57 PM, J Decker wrote: I've seen this also, and it is intermittant; once upon a time there were several AV programs that held new files open too long, so subsequent accesses would fail... sorry to be no help Can you try the current RC and see if it is fixed in that version of CMake? -Bill -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
[CMake] Cannot restore timestamp error on Windows
Hi all; I'm getting this error most, but not every, time I run cmake on my main Windows build system: 28CUSTOMBUILD : CMake error : Cannot restore timestamp D:\build-dir\MASTER-BRANCHES30-WINDOWS\BaseTest\CMakeFiles\generate.stamp It happens for different targets as well, not always the same one. I looked up this message and found another report of this error which was purported to be fixed in cmake by this patch: commit 2dc17f88dd2de900154f153f521b803ec9b7c377 Author: Brad King brad.k...@kitware.com Date: 2013-02-12 10:46:22 -0500 So, I've built a new cmake from 2.8.10 and cherry-picked that patch and I'm using that cmake now. However, even so I'm still seeing the above error. Anyone have any thoughts on this? Cheers! -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
