> -----Original Message----- > From: Satish Balay > Sent: Thursday, June 28, 2012 7:39 PM > To: For users of the development version of PETSc > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > Hm - eventhough you added 'argparse.py' to python-2.6 on cygwin to get > builder.py working - perhaps Matt is using some other python-2.7 > feature - and thats causing this difference?
Don't know. For what it's worth, my Linux box has python 2.6.6 and I had to put argparse.py there as well. Chetan > > Satish > > > > > > > > > I don't mind extra debugging on cygwin/Windows before reporting issues > > > > since I know the petsc project has many other priorities. But a few useful > > > > debugging pointers will be serve both of us better. > > > > > > > > Chetan > > > > > > > > > > > > > > > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > > mcs.anl.gov] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 6:00 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 4:49 PM, Chetan Jhurani <chetan.jhurani at > > gmail.com> wrote: > > > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > > > > ./config/builder.py > > > > (The same happens (existing .o are deleted) if ./config/builder2.py > > > > build is used) > > > > > This is not what I wrote. You used builder.py, which is python 2.4 > > > compatible, but has to do > everything without options, > > > so yes it deletes all the objects and rebuilds everything. That is why I > > > did not suggest you use > it. > > > > > I said to use builder2.py, which does proper dependency tracking. > > > > > > > > That is what I used and I mentioned it above. I've tried both separately. > > > > > > > > Both give the same result that libpetsc.lib is not created. And > > > > whether I use builder.py or builder2.py, the existing objects > > > > files are deleted in the second build. I did not call for a cleanup. > > > > > > > > Look. It does not do that. Ever. In any universe. I don't know what you > > think you are doing, > > > > but it is definitely not what you wrote. Look at the code. It can't do that. > > > > > > > > matt > > > > > > > > To be sure, I tried the whole thing in a new ARCH with a fresh > > > > configure step, and this time builder.py was not used at all (by me) > > > > but it led to the same issues with builder2.py. > > > > > > > > I understand that this is not the most used platform, so let me > > > > know if I can be of help. And I forgot to mention that the > > > > python build was 4-5 times faster than "ar" based build and > > > > that's great... > > > > > > > > Chetan > > > > > > > > > > > > > > > > > > > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > > mcs.anl.gov] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 3:21 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <chetan.jhurani at > > gmail.com> wrote: > > > > Matt, > > > > > > > > I was surprised too on both counts. Couldn't find any libpetsc.lib after > > the python build > > > > and I did not run "clean" in between that could have deleted the .o files. > > > > > > > > Here's one more try below to show it. > > > > > > > > $ ls test_8/lib/ > > > > libpetsc-obj > > > > > > > > $ ls test_8/lib/libpetsc-obj/ | wc -l > > > > 560 > > > > > > > > $ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 > > ./config/builder.py > > > > (The same happens (existing .o are deleted) if ./config/builder2.py build > > is used) > > > > > > > > This is not what I wrote. You used builder.py, which is python 2.4 > > compatible, but has to do > everything without options, > > > > so yes it deletes all the objects and rebuilds everything. That is why I > > did not suggest you use it. > > > > > > > > I said to use builder2.py, which does proper dependency tracking. > > > > > > > > Matt > > > > > > > > > > > > Rebuilding Dependencies > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm > > > > Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll > > > > > > > > I stopped the build using control-c. It would have deleted all files first. > > > > > > > > Traceback (most recent call last): > > > > File "./config/builder.py", line 1534, in <module> > > > > PETScMaker().run() > > > > File "./config/builder.py", line 1526, in run > > > > if self.buildLibraries('libpetsc', self.rootDir): > > > > File "./config/builder.py", line 1331, in buildLibraries > > > > objects += self.buildDir(root, files, objDir) > > > > File "./config/builder.py", line 1251, in buildDir > > > > objects.extend(getattr(self, 'compile'+language)(sourceMap[language], > > objDir)) > > > > File "./config/builder.py", line 1025, in compileC > > > > return self.compile(self.configInfo.languages.clanguage, source, objDir) > > > > File "./config/builder.py", line 1012, in compile > > > > (output, error, status) = self.executeShellCommand(cmd, checkCommand = > > noCheckCommand, > log=self.log) > > > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", > > line 254, in > executeShellCommand > > > > (output, error, status) = runInShell(command, log, cwd) > > > > File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", > > line 243, in runInShell > > > > thread.join(timeout) > > > > File "/usr/lib/python2.6/threading.py", line 655, in join > > > > self.__block.wait(delay) > > > > File "/usr/lib/python2.6/threading.py", line 258, in wait > > > > _sleep(delay) > > > > KeyboardInterrupt > > > > > > > > $ ls test_8/lib/libpetsc-obj/ > > > > bag.o > > > > > > > > All the previous .o were deleted. Perhaps I'm not doing something right? > > > > > > > > Chetan > > > > > > > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > > mcs.anl.gov] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 2:46 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani at > > gmail.com> wrote: > > > > Thanks Matt, Satish. A few issues though. > > > > > > > > Like you said, the latest cygwin has python 2.6.8 which > > > > does not have argparse. When I place the following > > > > in path http://argparse.googlecode.com/svn/trunk/argparse.py > > > > I can run the python build with 2.6.8. > > > > > > > > However, the "ar" based build had worked fine and the python > > > > based build fails for these two files. > > > > > > > > src/snes/impls/MULTIB~1/multiblock.c > > > > src/snes/impls/vi/rsaug/virsaug.c > > > > > > > > Yep, it fails, but the build succeeds, so no problem. > > > > > > > > It looks like these two files were NOT compiled in the ar based > > > > build. The two make.log files are attached. > > > > > > > > One more question. I tried to run the python based build > > > > the second time (after it was unsuccessful the first time) > > > > and then it deleted all the previous .o files it had created in > > > > ARCH/lib/libpetsc-obj. How can that be avoided? > > > > > > > > It does not do that unless you run 'clean'. > > > > > > > > Matt > > > > > > > > Chetan > > > > > > > > > > > > From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at > > mcs.anl.gov] On Behalf Of Matthew > Knepley > > Sent: Thursday, June 28, 2012 1:35 PM > > To: For users of the development version of PETSc > > Subject: Re: [petsc-dev] slow "ar Scq" on cygwin > > > > > > > > On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani at > > gmail.com> wrote: > > > > Hi devs, > > > > Is there a way to avoid running the archiver (ar) > > every time a petsc directory is compiled? > > > > The reason I'm asking this is that when libpetsc.lib > > becomes large enough (say 100MB+, esp. for debug build), > > "ar Sqc" on cygwin takes very long in each directory. > > Depending on OS caching, very long can be 10 seconds per > > PETSc directory just to run ar. > > > > It creates a new temp file from scratch, puts stuff from > > libpetsc.lib and new stuff in it and then renames the > > temp file. As you can imagine, the compilation process is > > fast in the beginning but crawls by the end. I've seen > > the same IO sequence on Linux (with Sqc flags to ar), but > > since cygwin IO is slower the effect is more clearly visible. > > Building within a ramdisk leads to 25% faster compilation > > but it was more of a curiosity. > > > > > > > > Build with Python. It never uses antique things like ar :) > > > > > > > > python2.7 ./config/builder2.py build > > > > > > > > Matt > > > > > > > > > > Chetan > > > > > > > > > > > > > > > >
