> On Oct 11, 2018, at 10:43 AM, stephano <[email protected]>
> wrote:
>
> Thank you all for a great set of meetings!
>
> This is an overview of the topics discussed and the tasks that were assigned.
> Please feel free to send me any questions or comments.
>
...
> Improvements to the Build Process
> --------------------------------
> We would like to gather a list of concrete specific proposals. Nate mentioned
> that there will always be some specialized tooling because of the nature of
> BIOS. Shawn mentioned that we need to keep in mind that making development
> easier for current developers should be a priority over making the processes
> easier for newcomers. I will send out emails asking for specific feedback on
> some of these topics:
>
> -Which toolchains are being used, which are validated, and which are known to
> create reproducible builds.
> -When toolchains are known to be orphaned, should they be archived or simply
> removed.
> -Could we add some kconfig-like tool that allows introspection into what type
> of builds are available.
> -How can we better track the code quality of BaseTools and the current build
> system on the whole. Should we add a "requires documentation change" flag to
> BZ so that it will be easier to compile a list of required doc changes.
>
Stephano,
I'll help start the conversation on the build.
1) Making the process easier for newcomers.
a) Getting it to build the 1st time
I think we could aspire to a more Unix like flow:
git clone
./configure
make ovmf
make ovmf.run
The ./config step would automate resolving dependencies. Things like figuring
out what compiler is installed, what extra tools need to be installed etc. It
could be interactive?
make <target> would build all the build tools, and do what ever local config is
required to get the build working. Then build the requested target.
make <target>.run is more of an optional step. It could do things like launch
Ovmf with your new ROM you just constructed.
b) Debugging failures.
Better quality error messages.
Making the build.log from the build command a default setting(or have an
abstracted defined database that can be queried), and add documentation on how
to use it. Maybe make the output easier to consume?
c) Making changes.
That can be a phase 2.
2) Build performance, complexity, and maintainability.
I'm about to complain about the build system, but I was around when it was
first invented so I'm to blame as much as anyone....
We have way too much custom Python that is very complex, hard to maintain, and
slow. One thing we did better on the EDK was break up the build tools into
smaller chunks (maybe too small in some cases) and I always found I could
figure things out, but then again all the code was C. Another interesting data
point is we added parallel build, via makefiles, to the EDK. When we moved from
the EDK to the edk2 we noticed the builds slowed down a lot. I did some
profiling on the edk2 build and I noticed 1,000,000 calls to regex (part of
that long ..... sequence on a build of a complex platform). It turns out we
have custom Python code that is generating the makefile dependencies so it was
scanning all the C code. The other issue I see is build parallelism is
controlled from the Python code. It also seems the edk2 build system is not a
complete solution and a lot of full platform implementations have shell scripts
or batch files, that wrap the calls to build.
So I'd like to see a build system:
1) Start with a top level makefile in the root of the tree.
2) Replaces custom batch or shell scripts with makefiles to handle pre and post
build tasks.
3) Use a tool to construct a parser for the ekd2 build files (INF, DSC, DEC).
Use this data to construct the makefiles and build a database for other
build components to use.
Chris Roberts has messed around and built an Earley parser using the
EBNF in the build documents. There are some problematic issues in the grammar,
but some grammar streamlining might make it possible to use tooling to
construct a parser vs. writing one from scratch in Python.
4) Move to using the compiler to generate dependencies. gcc/clang support this
via -M*. I think it may be a little more complex for NMAKE but there is a
/showIncludes compiler flag.
5) Have the build tool terminate after parallel makefile generation
6) Top level makefile can invoke parallel build on generated makefiles.
7) Invoke build tool to generate FVs and FDs from makefile.
8) Invoke custom post processing steps in the makefile.
Thanks,
Andrew Fish
_______________________________________________
edk2-devel mailing list
[email protected]
https://lists.01.org/mailman/listinfo/edk2-devel
