Andrew: Thanks for your detail information. 1) We did the test to disable header file dependency, and find there is no obvious performance improvement.
2) Build tool has genc and genmake target. Have you tried them? 3) We are evaluating to enable multiple thread in GenFd phase. For example, each FV generation will take one thread. Then, FV image generation can be Parallel. 4) We have not fully tuning build performance. We will do it. 5) You mean prebuild/postbuild script? They are actually single thread. Last, EDK and EDK2 has the different build system and code base. They both impact build performance. I am not sure whether edk2 build system causes the big gap between EDK and EDK2 build performance. Thanks Liming From: af...@apple.com [mailto:af...@apple.com] Sent: Thursday, June 30, 2016 7:06 AM To: Kinney, Michael D <michael.d.kin...@intel.com> Cc: Justen, Jordan L <jordan.l.jus...@intel.com>; Shi, Steven <steven....@intel.com>; edk2-devel@lists.01.org; Gao, Liming <liming....@intel.com> Subject: Re: [edk2] [PATCH 2/7] BaseTools: Add missing Elf relocation type for LTO build > On Jun 29, 2016, at 3:06 PM, Kinney, Michael D wrote: > > Why can't we simply use the target.txt TOOL_CHAIN_CONF define > in target.txt as it works today? You could split tools_def.txt > into several versions, with the default one being the maintained > one and developers can opt into community or deprecated ones. > > I do want to make sure we are focused on the right problem here > > 1) # lines in tools_def.txt > 2) maintenance of tools_def.txt > 3) build performance Mike, If parsing the file 47 times was enough to make a function stand out in the Python performance log I would guess the parsing it 1 time is not that efficient. In general the built tools seems architected to "go slow", well to make it easier to debug the build system and performance was not important. 1) I saw regex was getting called 1,000,000 times in a Python performance log. I think most of it is scanning all the C code to build the dependencies for the makefile a) Likely make + compiler is better performance optimized for this, and is continuously getting improved. b) A build option to do this the normal way would be nice. c) I'm guessing a lot of the early ... time in the build is this step. 2) Parallel builds are done from Python threads vs make -j a) See #1 b) It would be nice to have a build phase that just generated the makefiles including a top level makefile was constructed to enable parallel build. Then the top level makefile could call the generated makefile. After that build can be called to construct the FDs. c) When we build DEBUG and RELEASE in parallel, or multiple platforms it seems like the Python is fighting the make -j. If it was all make -j then make could load balance. 3) The FV construction seems slow and it generates multiple text files per section per file and there is lots of calling out between Python and C tools. a) Maybe combining all the FD construction into a single C program would allow all the data to be in memory vs. on disk. 4) It does not seem that the Python code has really been performance tuned. A large file was getting parsed 47 time, regex getting called a 1,000,000 times etc. 5) Most of the build targets use scripts. a) A top level makefile, and moving some of the work to makefiles could help with parallelism. b) I've got this working on our proprietary tree. 1) Had to make Conf/ part of the Buid/ output. I made a small mod to the tool to copy the Conf over if the alternate Conf location passed to build did not exit. 2) I had to add .NOTPARALLEL: to some of the BaseTools GNUmakefiles since they break for a parallel build. 3) Then you port the build.sh (or build.bat) to be a makefile. Our old parallelized EDK builds took about 2m10s to do a DEBUG build for a target, and with the edk2 it is 3m52s. Thanks, Andrew Fish PS As we know from performance tuning the ROM the most important thing to do is profile, and not listen to the old grumpy guy :). _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
