On Thu, 28 Jun 2012, Chetan Jhurani wrote: > > Look. It does not do that. Ever. In any universe. I don't know what you > > think you are doing, > > > but it is definitely not what you wrote. Look at the code. It can't do that. > > > > I don't think you understand what your code is doing in a new > > environment of the same universe. > > > > It works fine on Linux with the steps you had given. > > But on the cygwin side, no dependency files like > > ./lib/libpetsc-obj/adaptbasic.d are created. I found this after > > running the python build on Linux (I didn't know before this > > where the dependencies were stored). > > > > Here is some debug info in the python code: > > > > In the buildLibraries function on the Linux box (when building in the second > step) > > > > self.sourceDatabase = DirectedGraph with 1172 vertices and 21894 edges > > > > However, on the cygwin side (again, when building the second time) > > > > self.sourceDatabase = <builder.NullSourceDatabase object at 0xffd833ac> > > > > So of course it is totalRebuild = true on cygwin.
Hm - eventhough you added 'argparse.py' to python-2.6 on cygwin to get builder.py working - perhaps Matt is using some other python-2.7 feature - and thats causing this difference? Satish > > > > I don't mind extra debugging on cygwin/Windows before reporting issues > > since I know the petsc project has many other priorities. But a few useful > > debugging pointers will be serve both of us better. > > > > Chetan > > > > > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > mcs.anl.gov] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 6:00 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani at gmail.com> > wrote: > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > > > ./config/builder.py > > > (The same happens (existing .o are deleted) if ./config/builder2.py build > > > is used) > > > This is not what I wrote. You used builder.py, which is python 2.4 > > compatible, but has to do everything without options, > > so yes it deletes all the objects and rebuilds everything. That is why I > > did not suggest you use it. > > > I said to use builder2.py, which does proper dependency tracking. > > > > That is what I used and I mentioned it above. I've tried both separately. > > > > Both give the same result that libpetsc.lib is not created. And > > whether I use builder.py or builder2.py, the existing objects > > files are deleted in the second build. I did not call for a cleanup. > > > > Look. It does not do that. Ever. In any universe. I don't know what you think > you are doing, > > but it is definitely not what you wrote. Look at the code. It can't do that. > > > > matt > > > > To be sure, I tried the whole thing in a new ARCH with a fresh > > configure step, and this time builder.py was not used at all (by me) > > but it led to the same issues with builder2.py. > > > > I understand that this is not the most used platform, so let me > > know if I can be of help. And I forgot to mention that the > > python build was 4-5 times faster than "ar" based build and > > that's great... > > > > Chetan > > > > > > > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > mcs.anl.gov] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 3:21 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani at gmail.com> > wrote: > > Matt, > > > > I was surprised too on both counts. Couldn't find any libpetsc.lib after the > python build > > and I did not run "clean" in between that could have deleted the .o files. > > > > Here's one more try below to show it. > > > > $ ls test_8/lib/ > > libpetsc-obj > > > > $ ls test_8/lib/libpetsc-obj/ | wc -l > > 560 > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > ./config/builder.py > > (The same happens (existing .o are deleted) if ./config/builder2.py build is > used) > > > > This is not what I wrote. You used builder.py, which is python 2.4 > compatible, but has to do everything without options, > > so yes it deletes all the objects and rebuilds everything. That is why I did > not suggest you use it. > > > > I said to use builder2.py, which does proper dependency tracking. > > > > Matt > > > > > > Rebuilding Dependencies > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll > > > > I stopped the build using control-c. It would have deleted all files first. > > > > Traceback (most recent call last): > > File "./config/builder.py", line 1534, in <module> > > PETScMaker().run() > > File "./config/builder.py", line 1526, in run > > if self.buildLibraries('libpetsc', self.rootDir): > > File "./config/builder.py", line 1331, in buildLibraries > > objects += self.buildDir(root, files, objDir) > > File "./config/builder.py", line 1251, in buildDir > > objects.extend(getattr(self, 'compile'+language)(sourceMap[language], > objDir)) > > File "./config/builder.py", line 1025, in compileC > > return self.compile(self.configInfo.languages.clanguage, source, objDir) > > File "./config/builder.py", line 1012, in compile > > (output, error, status) = self.executeShellCommand(cmd, checkCommand = > noCheckCommand, log=self.log) > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line > 254, in executeShellCommand > > (output, error, status) = runInShell(command, log, cwd) > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line > 243, in runInShell > > thread.join(timeout) > > File "/usr/lib/python2.6/threading.py", line 655, in join > > self.__block.wait(delay) > > File "/usr/lib/python2.6/threading.py", line 258, in wait > > _sleep(delay) > > KeyboardInterrupt > > > > $ ls test_8/lib/libpetsc-obj/ > > bag.o > > > > All the previous .o were deleted. Perhaps I'm not doing something right? > > > > Chetan > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > mcs.anl.gov] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 2:46 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani at gmail.com> > wrote: > > Thanks Matt, Satish. A few issues though. > > > > Like you said, the latest cygwin has python 2.6.8 which > > does not have argparse. When I place the following > > in path http://argparse.googlecode.com/svn/trunk/argparse.py > > I can run the python build with 2.6.8. > > > > However, the "ar" based build had worked fine and the python > > based build fails for these two files. > > > > src/snes/impls/MULTIB~1/multiblock.c > > src/snes/impls/vi/rsaug/virsaug.c > > > > Yep, it fails, but the build succeeds, so no problem. > > > > It looks like these two files were NOT compiled in the ar based > > build. The two make.log files are attached. > > > > One more question. I tried to run the python based build > > the second time (after it was unsuccessful the first time) > > and then it deleted all the previous .o files it had created in > > ARCH/lib/libpetsc-obj. How can that be avoided? > > > > It does not do that unless you run 'clean'. > > > > Matt > > > > Chetan > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > mcs.anl.gov] On Behalf Of Matthew Knepley > Sent: Thursday, June 28, 2012 1:35 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani at gmail.com> > wrote: > > Hi devs, > > Is there a way to avoid running the archiver (ar) > every time a petsc directory is compiled? > > The reason I'm asking this is that when libpetsc.lib > becomes large enough (say 100MB+, esp. for debug build), > "ar Sqc" on cygwin takes very long in each directory. > Depending on OS caching, very long can be 10 seconds per > PETSc directory just to run ar. > > It creates a new temp file from scratch, puts stuff from > libpetsc.lib and new stuff in it and then renames the > temp file. As you can imagine, the compilation process is > fast in the beginning but crawls by the end. I've seen > the same IO sequence on Linux (with Sqc flags to ar), but > since cygwin IO is slower the effect is more clearly visible. > Building within a ramdisk leads to 25% faster compilation > but it was more of a curiosity. > > > > Build with Python. It never uses antique things like ar :) > > > > python2.7 ./config/builder2.py build > > > > Matt > > > > > Chetan > > > > > > > >
