----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://reviews.gem5.org/r/2247/#review5093 -----------------------------------------------------------
Ship it! Nice work! - Steve Reinhardt On May 7, 2014, 2:44 p.m., Andreas Hansson wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://reviews.gem5.org/r/2247/ > ----------------------------------------------------------- > > (Updated May 7, 2014, 2:44 p.m.) > > > Review request for Default. > > > Repository: gem5 > > > Description > ------- > > Changeset 10197:0bb745ea224b > --------------------------- > arch: teach ISA parser how to split code across files > > This patch encompasses several interrelated and interdependent changes > to the ISA generation step. The end goal is to reduce the size of the > generated compilation units for instruction execution and decoding so > that batch compilation can proceed with all CPUs active without > exhausting physical memory. > > The ISA parser (src/arch/isa_parser.py) has been improved so that it can > accept 'split [output_type];' directives at the top level of the grammar > and 'split(output_type)' python calls within 'exec {{ ... }}' blocks. > This has the effect of "splitting" the files into smaller compilation > units. I use air-quotes around "splitting" because the files themselves > are not split, but preprocessing directives are inserted to have the same > effect. > > Architecturally, the ISA parser has had some changes in how it works. > In general, it emits code sooner. It doesn't generate per-CPU files, > and instead defers to the C preprocessor to create the duplicate copies > for each CPU type. Likewise there are more files emitted and the C > preprocessor does more substitution that used to be done by the ISA parser. > > Finally, the build system (SCons) needs to be able to cope with a > dynamic list of source files coming out of the ISA parser. The changes > to the SCons{cript,truct} files support this. In broad strokes, the > targets requested on the command line are hidden from SCons until all > the build dependencies are determined, otherwise it would try, realize > it can't reach the goal, and terminate in failure. Since build steps > (i.e. running the ISA parser) must be taken to determine the file list, > several new build stages have been inserted at the very start of the > build. First, the build dependencies from the ISA parser will be emitted > to arch/$ISA/generated/inc.d, which is then read by a new SCons builder > to finalize the dependencies. (Once inc.d exists, the ISA parser will not > need to be run to complete this step.) Once the dependencies are known, > the 'Environments' are made by the makeEnv() function. This function used > to be called before the build began but now happens during the build. > It is easy to see that this step is quite slow; this is a known issue > and it's important to realize that it was already slow, but there was > no obvious cause to attribute it to since nothing was displayed to the > terminal. Since new steps that used to be performed serially are now in a > potentially-parallel build phase, the pathname handling in the SCons scripts > has been tightened up to deal with chdir() race conditions. In general, > pathnames are computed earlier and more likely to be stored, passed around, > and processed as absolute paths rather than relative paths. In the end, > some of these issues had to be fixed by inserting serializing dependencies > in the build. > > Minor note: > For the null ISA, we just provide a dummy inc.d so SCons is never > compelled to try to generate it. While it seems slightly wrong to have > anything in src/arch/*/generated (i.e. a non-generated 'generated' file), > it's by far the simplest solution. > > > Diffs > ----- > > SConstruct e40b35147270 > src/SConscript e40b35147270 > src/arch/SConscript e40b35147270 > src/arch/alpha/SConscript e40b35147270 > src/arch/alpha/isa/fp.isa e40b35147270 > src/arch/alpha/isa/main.isa e40b35147270 > src/arch/alpha/isa/mem.isa e40b35147270 > src/arch/alpha/isa/opcdec.isa e40b35147270 > src/arch/alpha/isa/unimp.isa e40b35147270 > src/arch/alpha/isa/unknown.isa e40b35147270 > src/arch/arm/SConscript e40b35147270 > src/arch/arm/isa/formats/breakpoint.isa e40b35147270 > src/arch/arm/isa/formats/unimp.isa e40b35147270 > src/arch/arm/isa/templates/basic.isa e40b35147270 > src/arch/arm/isa/templates/macromem.isa e40b35147270 > src/arch/arm/isa/templates/mem.isa e40b35147270 > src/arch/arm/isa/templates/mem64.isa e40b35147270 > src/arch/arm/isa/templates/neon.isa e40b35147270 > src/arch/arm/isa/templates/neon64.isa e40b35147270 > src/arch/arm/isa/templates/pred.isa e40b35147270 > src/arch/isa_parser.py e40b35147270 > src/arch/mips/SConscript e40b35147270 > src/arch/mips/isa/formats/basic.isa e40b35147270 > src/arch/mips/isa/formats/control.isa e40b35147270 > src/arch/mips/isa/formats/dsp.isa e40b35147270 > src/arch/mips/isa/formats/fp.isa e40b35147270 > src/arch/mips/isa/formats/int.isa e40b35147270 > src/arch/mips/isa/formats/mem.isa e40b35147270 > src/arch/mips/isa/formats/mt.isa e40b35147270 > src/arch/mips/isa/formats/noop.isa e40b35147270 > src/arch/mips/isa/formats/tlbop.isa e40b35147270 > src/arch/mips/isa/formats/trap.isa e40b35147270 > src/arch/mips/isa/formats/unimp.isa e40b35147270 > src/arch/mips/isa/formats/unknown.isa e40b35147270 > src/arch/power/SConscript e40b35147270 > src/arch/power/isa/formats/basic.isa e40b35147270 > src/arch/power/isa/formats/mem.isa e40b35147270 > src/arch/power/isa/formats/misc.isa e40b35147270 > src/arch/power/isa/formats/unimp.isa e40b35147270 > src/arch/power/isa/formats/unknown.isa e40b35147270 > src/arch/sparc/SConscript e40b35147270 > src/arch/sparc/isa/base.isa e40b35147270 > src/arch/sparc/isa/formats/basic.isa e40b35147270 > src/arch/sparc/isa/formats/branch.isa e40b35147270 > src/arch/sparc/isa/formats/integerop.isa e40b35147270 > src/arch/sparc/isa/formats/mem/swap.isa e40b35147270 > src/arch/sparc/isa/formats/mem/util.isa e40b35147270 > src/arch/sparc/isa/formats/nop.isa e40b35147270 > src/arch/sparc/isa/formats/priv.isa e40b35147270 > src/arch/sparc/isa/formats/trap.isa e40b35147270 > src/arch/sparc/isa/formats/unimp.isa e40b35147270 > src/arch/sparc/isa/formats/unknown.isa e40b35147270 > src/arch/x86/SConscript e40b35147270 > src/arch/x86/isa/formats/basic.isa e40b35147270 > src/arch/x86/isa/formats/cpuid.isa e40b35147270 > src/arch/x86/isa/formats/nop.isa e40b35147270 > src/arch/x86/isa/formats/syscall.isa e40b35147270 > src/arch/x86/isa/formats/unimp.isa e40b35147270 > src/arch/x86/isa/formats/unknown.isa e40b35147270 > src/arch/x86/isa/microops/debug.isa e40b35147270 > src/arch/x86/isa/microops/fpop.isa e40b35147270 > src/arch/x86/isa/microops/ldstop.isa e40b35147270 > src/arch/x86/isa/microops/limmop.isa e40b35147270 > src/arch/x86/isa/microops/mediaop.isa e40b35147270 > src/arch/x86/isa/microops/regop.isa e40b35147270 > src/arch/x86/isa/microops/seqop.isa e40b35147270 > src/arch/x86/isa/microops/specop.isa e40b35147270 > tests/SConscript e40b35147270 > > Diff: http://reviews.gem5.org/r/2247/diff/ > > > Testing > ------- > > > Thanks, > > Andreas Hansson > > _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
