Re: [CMake] "Cannot restore timestamp" error on Windows
> The only race was for multiple processes simultaneously deciding they need to create the (currently missing) file and then trying to open the file for > write at the same time. To what I see on the logs, this is not what I observe: build 26-avr.-2013 13:11:12 CMake does not need to re-run because XXX\tmp\src\\CMakeFiles\generate.stamp is up-to-date. build 26-avr.-2013 13:11:12 Building Custom Rule CCC/CMakeLists.txt build 26-avr.-2013 13:11:12 Building Custom Rule CCC/CMakeLists.txt build 26-avr.-2013 13:11:12 Building Custom Rule CCC/CMakeLists.txt build 26-avr.-2013 13:11:12 CMake does not need to re-run because XXX\tmp\src\\CMakeFiles\generate.stamp is up-to-date. then build 26-avr.-2013 13:11:24 Building Custom Rule CCC/CMakeLists.txt build 26-avr.-2013 13:11:24 CUSTOMBUILD : CMake error : Cannot restore timestamp XXX\tmp\src\\CMakeFiles\generate.stamp [XXX\tmp\src\\SSS.vcxproj] The file XXX\tmp\src\\CMakeFiles\generate.stamp already existed long before the error, and many projects were built between. But if by "missing" you are saying that it is missing because moved to another filename (I am a bit ignorant of how it works), then we still have a race condition. > CMake has a different meaning for "project" than .vcxproj so this will still be the same name for all targets in a directory. > Each .vcxproj file corresponds to what CMake calls a "target". Indeed, sorry for that mistake. > That race is gone because replacement is now atomic This is true for unices, but does not seem to be true on Windows. There is a fallback that tries to move the file 5 times before triggering an error. This is not atomic. > In order to use a different stamp file for each .vcxproj file then the custom command produced by CreateVCProjBuildRule would not be able to be > re-used across multiple targets. Instead each generator would have to hand-code the custom command for its own .vcxproj file. > While possible it is a lot of work and this bug has already been solved. True, and more true now I have a closer look at the code. I do not understand why it was so (historically), and not understanding that does not help me to provide you a patch. For instance, in " cmLocalVisualStudio10Generator::Generate()", all vcxproj are generated, and then one unique timestamp file is written, hence for the current CMakeLists.txt. If I understand well, this is the only location (VC10) where the timestamp is written/made dirty. "cmLocalVisualStudio7Generator::CreateVCProjBuildRule()" adds a rule for checking the timestamp, but I do not understand why this is not a per-target custom build rule in "cmLocalVisualStudio7Generator::AddCMakeListsRules()" (this could easily be done in cmLocalVisualStudio10Generator::Generate()), nor to what entity the " this->Makefile->AddCustomCommandToOutput" (in "cmLocalVisualStudio7Generator::CreateVCProjBuildRule()" ) refers to. But if we have something like: void cmLocalVisualStudio7Generator::AddCMakeListsRules() { cmTargets &tgts = this->Makefile->GetTargets(); // Create the regeneration custom rule. if(!this->Makefile->IsOn("CMAKE_SUPPRESS_REGENERATION")) { // HERE: just returning the files that should be included if(cmSourceFile* sf = this->Get_CMAKELISTFILENAME()) { // Add the rule to targets that need it. for(cmTargets::iterator l = tgts.begin(); l != tgts.end(); ++l) { if(l->first != CMAKE_CHECK_BUILD_SYSTEM_TARGET) { l->second.AddSourceFile(sf); // HERE adding the custom build rule checking the timestamps l->ADD_CUSTOM_BUILD_RULE_WITH_STAMPS(); } this should be closer to what I thought initially. For an automatic build daemon, I think setting CMAKE_SUPPRESS_REGENERATION should be sufficient workaround (until the next "true" release with the proposed workaround). Best, Raffi Enficiaud -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On 04/29/2013 04:37 PM, Raffi Enficiaud wrote: > - process 1 is accessing generate.stamp for reading Nothing ever reads the file. Only its existence and modification time matter. > - process 2 is moving some "tmpfile" to generate.stamp -> race The only race was for multiple processes simultaneously deciding they need to create the (currently missing) file and then trying to open the file for write at the same time. That race is gone because replacement is now atomic. > stampName += std::string(this->Makefile->GetProjectName()) CMake has a different meaning for "project" than .vcxproj so this will still be the same name for all targets in a directory. Each .vcxproj file corresponds to what CMake calls a "target". In order to use a different stamp file for each .vcxproj file then the custom command produced by CreateVCProjBuildRule would not be able to be re-used across multiple targets. Instead each generator would have to hand-code the custom command for its own .vcxproj file. While possible it is a lot of work and this bug has already been solved. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
Hi Brad, Thank you for your reply. As you pointed out, the problem seems to come from a race condition on the file "generate.stamp". Now the operations being atomic, the problem should be solved, in practice. However, I do not fully understand in what extent the atomicity of renaming a file would solve this. The log says: "Write the stamp file to a random temporary name and then atomically rename it to the real stamp file." The operations take basically less time; hence the problem should occur less. However: - process 1 is accessing generate.stamp for reading - process 2 is moving some "tmpfile" to generate.stamp -> race The probability of race increases with the number of projects in CMakeList.txt (since they share "generate.stamp") and the number of cores used for building (and of course the structure and dependencies among the projects). What if, in .\Source\cmLocalVisualStudio7Generator.cxx, we replace: cmSourceFile* cmLocalVisualStudio7Generator::CreateVCProjBuildRule() { std::string stampName = this->Makefile->GetCurrentOutputDirectory(); stampName += "/"; stampName += cmake::GetCMakeFilesDirectoryPostSlash(); stampName += "generate.stamp"; const char* dsprule = this->Makefile->GetRequiredDefinition("CMAKE_COMMAND"); by cmSourceFile* cmLocalVisualStudio7Generator::CreateVCProjBuildRule() { std::string stampName = this->Makefile->GetCurrentOutputDirectory(); stampName += "/"; stampName += cmake::GetCMakeFilesDirectoryPostSlash(); stampName += std::string(this->Makefile->GetProjectName()) + "_generate.stamp"; const char* dsprule = this->Makefile->GetRequiredDefinition("CMAKE_COMMAND"); I do not know the code that much, but I guess "Makefile::SetProjectName" is called much earlier than cmLocalVisualStudio7Generator::CreateVCProjBuildRule, right? What do you think? Best, Raffi Enficiaud -Original Message----- From: Brad King [mailto:brad.k...@kitware.com] Sent: lundi 29 avril 2013 18:56 To: Raffi Enficiaud Cc: cmake@cmake.org Subject: Re: [CMake] "Cannot restore timestamp" error on Windows On 04/29/2013 10:53 AM, Raffi Enficiaud wrote: > I read the content of the change in the link you provided, and I have > a newbie question: > Why not using one timestamp (different filename) per library/binary/project? The current approach evolved historically. A per-target approach would probably work too but things should be working now anyway. > Looking forward to having this issue solved in the next official release! It is already solved in 2.8.11-rc*. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On 04/29/2013 10:53 AM, Raffi Enficiaud wrote: > I read the content of the change in the link you provided, and I have a > newbie question: > Why not using one timestamp (different filename) per library/binary/project? The current approach evolved historically. A per-target approach would probably work too but things should be working now anyway. > Looking forward to having this issue solved in the next official release! It is already solved in 2.8.11-rc*. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
Hi, I read the content of the change in the link you provided, and I have a newbie question: Why not using one timestamp (different filename) per library/binary/project ? Looking forward to having this issue solved in the next official release! Best, Raffi Enficiaud -- View this message in context: http://cmake.3232098.n2.nabble.com/Cannot-restore-timestamp-error-on-Windows-tp7584120p7584273.html Sent from the CMake mailing list archive at Nabble.com. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On Wed, 2013-04-24 at 09:11 -0400, Paul Smith wrote: > > I have seen this as well with the 2.8.11-rc releases. I am not exactly > > what causes that. > > The package I'm trying to build is here, FWIW: > > https://github.com/nuodb/nuodb-php-pdo > > The CMakeLists.txt file is here: > > https://github.com/nuodb/nuodb-php-pdo/blob/master/CMakeLists.txt Should I file a bug about this? Or is this an expected change in 2.8.11? Maybe the CMakeLists.txt file is incorrect in a way that was previously ignored? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On Wed, 2013-04-24 at 09:02 -0400, John Drescher wrote: > > I installed it and it ran one time and I didn't see the error, but it's > > intermittent so that's not definitive. Unfortunately one of my cmake > > files failed (later on; this is a cmake file from a smaller > > sub-project): > > > > CMake Error at CMakeLists.txt:190 (TARGET_LINK_LIBRARIES): > > Error evaluating generator expression: > > > > > > $:php5ts_debug>,INTERFACE_INCLUDE_DIRECTORIES> > > > > $ expression requires a non-empty target > > name. > > > > I have to back out the upgrade until I can figure out what this error > > means: there's no reference to INTERFACE_INCLUDE_DIRECTORIES anywhere in > > my CMakeLists.txt and this project is not one I'm familiar with. > > > > I have seen this as well with the 2.8.11-rc releases. I am not exactly > what causes that. The package I'm trying to build is here, FWIW: https://github.com/nuodb/nuodb-php-pdo The CMakeLists.txt file is here: https://github.com/nuodb/nuodb-php-pdo/blob/master/CMakeLists.txt -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
> I installed it and it ran one time and I didn't see the error, but it's > intermittent so that's not definitive. Unfortunately one of my cmake > files failed (later on; this is a cmake file from a smaller > sub-project): > > CMake Error at CMakeLists.txt:190 (TARGET_LINK_LIBRARIES): > Error evaluating generator expression: > > > $:php5ts_debug>,INTERFACE_INCLUDE_DIRECTORIES> > > $ expression requires a non-empty target > name. > > I have to back out the upgrade until I can figure out what this error > means: there's no reference to INTERFACE_INCLUDE_DIRECTORIES anywhere in > my CMakeLists.txt and this project is not one I'm familiar with. > I have seen this as well with the 2.8.11-rc releases. I am not exactly what causes that. John -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On Tue, 2013-04-23 at 21:50 -0400, Bill Hoffman wrote: > On 4/23/2013 8:57 PM, J Decker wrote: > > I've seen this also, and it is intermittant; once upon a time there were > > several AV programs that held new files open too long, so subsequent > > accesses would fail... sorry to be no help > Can you try the current RC and see if it is fixed in that version of CMake? I installed it and it ran one time and I didn't see the error, but it's intermittent so that's not definitive. Unfortunately one of my cmake files failed (later on; this is a cmake file from a smaller sub-project): CMake Error at CMakeLists.txt:190 (TARGET_LINK_LIBRARIES): Error evaluating generator expression: $:php5ts_debug>,INTERFACE_INCLUDE_DIRECTORIES> $ expression requires a non-empty target name. I have to back out the upgrade until I can figure out what this error means: there's no reference to INTERFACE_INCLUDE_DIRECTORIES anywhere in my CMakeLists.txt and this project is not one I'm familiar with. -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On Sun, Apr 21, 2013 at 12:32 AM, Paul Smith wrote: > I looked up this message and found another report of this error which > was purported to be fixed in cmake by this patch: > > commit 2dc17f88dd2de900154f153f521b803ec9b7c377 > Author: Brad King > Date: 2013-02-12 10:46:22 -0500 For reference, here is a link to that change: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=2dc17f88 > So, I've built a new cmake from 2.8.10 and cherry-picked that patch and > I'm using that cmake now. However, even so I'm still seeing the above > error. The above change fixed the problem only because it was based on a CMake that also included a fix to RenameFile: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=59b568e5 Please try 2.8.11-rc3 which has both changes. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On 4/23/2013 8:57 PM, J Decker wrote: I've seen this also, and it is intermittant; once upon a time there were several AV programs that held new files open too long, so subsequent accesses would fail... sorry to be no help Can you try the current RC and see if it is fixed in that version of CMake? -Bill -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
I've seen this also, and it is intermittant; once upon a time there were several AV programs that held new files open too long, so subsequent accesses would fail... sorry to be no help On Tue, Apr 23, 2013 at 5:34 PM, Paul Smith wrote: > On Tue, 2013-04-23 at 15:54 -0700, J Decker wrote: > > do you have antivirus software that you can disable? > > No, there's no special antivirus software on these build servers. It's > hard to imagine what kind of issue antivirus software would cause, that > would be so intermittent. Our builds perform this "generate.stamp" > operation over 100 times per build, according to our logs, and only once > in every build or two does it cause a problem. > > Cheers! > > -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On Tue, 2013-04-23 at 15:54 -0700, J Decker wrote: > do you have antivirus software that you can disable? No, there's no special antivirus software on these build servers. It's hard to imagine what kind of issue antivirus software would cause, that would be so intermittent. Our builds perform this "generate.stamp" operation over 100 times per build, according to our logs, and only once in every build or two does it cause a problem. Cheers! -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
do you have antivirus software that you can disable? On Tue, Apr 23, 2013 at 2:43 PM, Paul Smith wrote: > On Sun, 2013-04-21 at 00:32 -0400, Paul Smith wrote: > > Hi all; I'm getting this error most, but not every, time I run cmake on > > my main Windows build system: > > > > 28>CUSTOMBUILD : CMake error : Cannot restore timestamp > D:\build-dir\MASTER-BRANCHES30-WINDOWS\BaseTest\CMakeFiles\generate.stamp > > > > It happens for different targets as well, not always the same one. > > > commit 2dc17f88dd2de900154f153f521b803ec9b7c377 > > Author: Brad King > > Date: 2013-02-12 10:46:22 -0500 > > We're still seeing this, pretty often. This is a very serious problem > for us as it's causing spurious build failures. > > I've looked into the code and added debugging. For example if the > RenameFile() fails, I've printed the system error. The error always > appears to be "No such file or directory". > > I've also added a check on the open of the temp file so that if it > fails, I print the reason: previously there was no check for that. > However, that code never executes so it seems that's not the problem. > > Currently my suspicion is that the name of the temp file is not unique > enough: somehow multiple concurrent runs of this function are using the > same temp file name. I've added output to print the temp file name > always, not just when a failure happens, but it will take a bit of time > to get into our build system. > > Anyone have any other ideas? > > -- > > Powered by www.kitware.com > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Follow this link to subscribe/unsubscribe: > http://www.cmake.org/mailman/listinfo/cmake > -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] "Cannot restore timestamp" error on Windows
On Sun, 2013-04-21 at 00:32 -0400, Paul Smith wrote: > Hi all; I'm getting this error most, but not every, time I run cmake on > my main Windows build system: > > 28>CUSTOMBUILD : CMake error : Cannot restore timestamp > D:\build-dir\MASTER-BRANCHES30-WINDOWS\BaseTest\CMakeFiles\generate.stamp > > It happens for different targets as well, not always the same one. > commit 2dc17f88dd2de900154f153f521b803ec9b7c377 > Author: Brad King > Date: 2013-02-12 10:46:22 -0500 We're still seeing this, pretty often. This is a very serious problem for us as it's causing spurious build failures. I've looked into the code and added debugging. For example if the RenameFile() fails, I've printed the system error. The error always appears to be "No such file or directory". I've also added a check on the open of the temp file so that if it fails, I print the reason: previously there was no check for that. However, that code never executes so it seems that's not the problem. Currently my suspicion is that the name of the temp file is not unique enough: somehow multiple concurrent runs of this function are using the same temp file name. I've added output to print the temp file name always, not just when a failure happens, but it will take a bit of time to get into our build system. Anyone have any other ideas? -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
[CMake] "Cannot restore timestamp" error on Windows
Hi all; I'm getting this error most, but not every, time I run cmake on my main Windows build system: 28>CUSTOMBUILD : CMake error : Cannot restore timestamp D:\build-dir\MASTER-BRANCHES30-WINDOWS\BaseTest\CMakeFiles\generate.stamp It happens for different targets as well, not always the same one. I looked up this message and found another report of this error which was purported to be fixed in cmake by this patch: commit 2dc17f88dd2de900154f153f521b803ec9b7c377 Author: Brad King Date: 2013-02-12 10:46:22 -0500 So, I've built a new cmake from 2.8.10 and cherry-picked that patch and I'm using that cmake now. However, even so I'm still seeing the above error. Anyone have any thoughts on this? Cheers! -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake