[m5-dev] Introducing...InOrder AlphaFS!

2011-04-21 Thread Korey Sewell
For 1-4 wide, InOrder Cores running in ALPHA FS mode... Observe terminal
output:
M5 console: m5AlphaAccess @ 0xFD02
Got Configuration 623
memsize 800 pages 4000
First free page after ROM 0xFC018000
HWRPB 0xFC018000 l1pt 0xFC04 l2pt 0xFC042000
l3pt_rpb 0xFC044000 l3pt_kernel 0xFC048000 l2reserv
0xFC046000
kstart = 0xFC31, kend = 0xFC855898, kentry =
0xFC31, numCPUs = 0x1
CPU Clock at 2000 MHz IntrClockFrequency=1024
Booting with 1 processor(s)
...
Linux version 2.6.13 (h...@zed.eecs.umich.edu) (gcc version 3.4.3) #1 SMP
Sun Oct 8 ..
Brought up 1 CPUs
...
init started:  BusyBox v1.1.0 (2007.03.04-01:07+) multi-call binary
mounting filesystems...
EXT2-fs warning: checktime reached, running e2fsck is recommended
loading script...
*
**Two Words: Yes Sir!

*More Than Two Words: InOrder is booting Linux!!!

Getting to this point took about 30 patches, so I have a lot of
patch-cleaning work to do before I can place it in the repo.

Also, just because it boots doesn't mean I've tested it with any kind of
real workload suite (e.g. SPEC or Parsec).  I don't have the bandwidth to
complete stress test through all the benchmarks and test but if someone
tries and something goes wrong, please post to the mailing list and I'll do
my best to help you debug.

Either way, I think this is a good milestone and I'll be adding the patches
and the regression test for ALPHA_FS/10.linux_boot/InOrder-CPU within the
next week (or two) and then eventually updating more things in our
handy-dandy M5 status matrix:
http://m5sim.org/wiki/index.php/Status_Matrix

The next big thing I want to get in InOrder before the gem5 official merge
is complete is timing TLB translation and in the longer term microcode
support. I can't say exactly when those will be added (again, not enough
time for me to completely focus on this), but if anyone wants to try I have
a good idea of what needs to be done and can assist.

-- 
- Korey
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] Introducing...InOrder AlphaFS!

2011-04-21 Thread Gabe Black

Tada! Congratulations.

Gabe

On 4/21/2011 12:12 AM, Korey Sewell wrote:

For 1-4 wide, InOrder Cores running in ALPHA FS mode... Observe terminal
output:
M5 console: m5AlphaAccess @ 0xFD02
Got Configuration 623
memsize 800 pages 4000
First free page after ROM 0xFC018000
HWRPB 0xFC018000 l1pt 0xFC04 l2pt 0xFC042000
l3pt_rpb 0xFC044000 l3pt_kernel 0xFC048000 l2reserv
0xFC046000
kstart = 0xFC31, kend = 0xFC855898, kentry =
0xFC31, numCPUs = 0x1
CPU Clock at 2000 MHz IntrClockFrequency=1024
Booting with 1 processor(s)
...
Linux version 2.6.13 (h...@zed.eecs.umich.edu) (gcc version 3.4.3) #1 SMP
Sun Oct 8 ..
Brought up 1 CPUs
...
init started:  BusyBox v1.1.0 (2007.03.04-01:07+) multi-call binary
mounting filesystems...
EXT2-fs warning: checktime reached, running e2fsck is recommended
loading script...
*
**Two Words: Yes Sir!

*More Than Two Words: InOrder is booting Linux!!!

Getting to this point took about30 patches, so I have a lot of
patch-cleaning work to do before I can place it in the repo.

Also, just because it boots doesn't mean I've tested it with any kind of
real workload suite (e.g. SPEC or Parsec).  I don't have the bandwidth to
complete stress test through all the benchmarks and test but if someone
tries and something goes wrong, please post to the mailing list and I'll do
my best to help you debug.

Either way, I think this is a good milestone and I'll be adding the patches
and the regression test for ALPHA_FS/10.linux_boot/InOrder-CPU within the
next week (or two) and then eventually updating more things in our
handy-dandy M5 status matrix:
http://m5sim.org/wiki/index.php/Status_Matrix

The next big thing I want to get in InOrder before the gem5 official merge
is complete is timing TLB translation and in the longer term microcode
support. I can't say exactly when those will be added (again, not enough
time for me to completely focus on this), but if anyone wants to try I have
a good idea of what needs to be done and can assist.

   


___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


[m5-dev] Cron m5test@zizzer /z/m5/regression/do-regression quick

2011-04-21 Thread Cron Daemon
* build/ALPHA_SE/tests/opt/quick/60.rubytest/alpha/linux/rubytest-ruby 
passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/linux/o3-timing passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/linux/simple-timing passed.
* build/ALPHA_SE/tests/opt/quick/20.eio-short/alpha/eio/simple-timing 
passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/tru64/simple-timing passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/linux/simple-atomic passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/linux/inorder-timing passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/tru64/simple-atomic passed.
* build/ALPHA_SE/tests/opt/quick/30.eio-mp/alpha/eio/simple-timing-mp 
passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/tru64/simple-timing-ruby 
passed.
* build/ALPHA_SE/tests/opt/quick/20.eio-short/alpha/eio/simple-atomic 
passed.
* build/ALPHA_SE/tests/opt/quick/01.hello-2T-smt/alpha/linux/o3-timing 
passed.
* build/ALPHA_SE/tests/opt/quick/30.eio-mp/alpha/eio/simple-atomic-mp 
passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/tru64/o3-timing passed.
* build/ALPHA_SE/tests/opt/quick/00.hello/alpha/linux/simple-timing-ruby 
passed.
* build/ALPHA_SE/tests/opt/quick/50.memtest/alpha/linux/memtest-ruby passed.
* build/ALPHA_SE/tests/opt/quick/50.memtest/alpha/linux/memtest passed.
* 
build/ALPHA_SE_MOESI_hammer/tests/opt/quick/60.rubytest/alpha/linux/rubytest-ruby-MOESI_hammer
 passed.
* 
build/ALPHA_SE_MOESI_hammer/tests/opt/quick/00.hello/alpha/tru64/simple-timing-ruby-MOESI_hammer
 passed.
* 
build/ALPHA_SE_MOESI_hammer/tests/opt/quick/00.hello/alpha/linux/simple-timing-ruby-MOESI_hammer
 passed.
* 
build/ALPHA_SE_MESI_CMP_directory/tests/opt/quick/60.rubytest/alpha/linux/rubytest-ruby-MESI_CMP_directory
 passed.
* 
build/ALPHA_SE_MESI_CMP_directory/tests/opt/quick/00.hello/alpha/tru64/simple-timing-ruby-MESI_CMP_directory
 passed.
* 
build/ALPHA_SE_MESI_CMP_directory/tests/opt/quick/00.hello/alpha/linux/simple-timing-ruby-MESI_CMP_directory
 passed.
* 
build/ALPHA_SE_MOESI_hammer/tests/opt/quick/50.memtest/alpha/linux/memtest-ruby-MOESI_hammer
 passed.
* 
build/ALPHA_SE_MOESI_CMP_directory/tests/opt/quick/60.rubytest/alpha/linux/rubytest-ruby-MOESI_CMP_directory
 passed.
* 
build/ALPHA_SE_MOESI_CMP_directory/tests/opt/quick/00.hello/alpha/tru64/simple-timing-ruby-MOESI_CMP_directory
 passed.
* 
build/ALPHA_SE_MOESI_CMP_directory/tests/opt/quick/00.hello/alpha/linux/simple-timing-ruby-MOESI_CMP_directory
 passed.
* 
build/ALPHA_SE_MOESI_CMP_token/tests/opt/quick/60.rubytest/alpha/linux/rubytest-ruby-MOESI_CMP_token
 passed.
* 
build/ALPHA_SE_MOESI_CMP_token/tests/opt/quick/00.hello/alpha/tru64/simple-timing-ruby-MOESI_CMP_token
 passed.
* 
build/ALPHA_SE_MOESI_CMP_token/tests/opt/quick/00.hello/alpha/linux/simple-timing-ruby-MOESI_CMP_token
 passed.
* 
build/ALPHA_SE_MESI_CMP_directory/tests/opt/quick/50.memtest/alpha/linux/memtest-ruby-MESI_CMP_directory
 passed.
* 
build/ALPHA_SE_MOESI_CMP_token/tests/opt/quick/50.memtest/alpha/linux/memtest-ruby-MOESI_CMP_token
 passed.
* 
build/ALPHA_FS/tests/opt/quick/10.linux-boot/alpha/linux/tsunami-simple-atomic 
passed.
* 
build/ALPHA_FS/tests/opt/quick/10.linux-boot/alpha/linux/tsunami-simple-atomic-dual
 passed.
* 
build/ALPHA_SE_MOESI_CMP_directory/tests/opt/quick/50.memtest/alpha/linux/memtest-ruby-MOESI_CMP_directory
 passed.
* 
build/ALPHA_FS/tests/opt/quick/10.linux-boot/alpha/linux/tsunami-simple-timing-dual
 passed.
* 
build/ALPHA_FS/tests/opt/quick/10.linux-boot/alpha/linux/tsunami-simple-timing 
passed.
* 
build/ALPHA_FS/tests/opt/quick/80.netperf-stream/alpha/linux/twosys-tsunami-simple-atomic
 passed.
* build/MIPS_SE/tests/opt/quick/00.hello/mips/linux/simple-timing-ruby 
passed.
* build/MIPS_SE/tests/opt/quick/00.hello/mips/linux/simple-atomic passed.
* build/MIPS_SE/tests/opt/quick/00.hello/mips/linux/simple-timing passed.
* build/MIPS_SE/tests/opt/quick/00.hello/mips/linux/o3-timing passed.
* build/MIPS_SE/tests/opt/quick/00.hello/mips/linux/inorder-timing passed.
* build/POWER_SE/tests/opt/quick/00.hello/power/linux/simple-atomic passed.
* build/POWER_SE/tests/opt/quick/00.hello/power/linux/o3-timing passed.
* build/SPARC_SE/tests/opt/quick/00.hello/sparc/linux/simple-atomic passed.
* build/SPARC_SE/tests/opt/quick/00.hello/sparc/linux/simple-timing passed.
* build/SPARC_SE/tests/opt/quick/02.insttest/sparc/linux/simple-timing 
passed.
= Statistics differences =* 
build/SPARC_SE/tests/opt/quick/40.m5threads-test-atomic/sparc/linux/simple-atomic-mp
 passed.
* 
build/SPARC_SE/tests/opt/quick/40.m5threads-test-atomic/sparc/linux/simple-timing-mp
 passed.
* build/SPARC_SE/tests/opt/quick/02.insttest/sparc/linux/o3-timing passed.
* 
build/SPARC_SE/tests/opt/quick/40.m5threads-test-atomic/sparc/linux/o3-timing-mp
 passed.

Re: [m5-dev] Introducing...InOrder AlphaFS!

2011-04-21 Thread Steve Reinhardt
Way to go, Korey!

On Thu, Apr 21, 2011 at 1:11 AM, Gabe Black gbl...@eecs.umich.edu wrote:
 Tada! Congratulations.

 Gabe

 On 4/21/2011 12:12 AM, Korey Sewell wrote:

 For 1-4 wide, InOrder Cores running in ALPHA FS mode... Observe terminal
 output:
 M5 console: m5AlphaAccess @ 0xFD02
 Got Configuration 623
 memsize 800 pages 4000
 First free page after ROM 0xFC018000
 HWRPB 0xFC018000 l1pt 0xFC04 l2pt 0xFC042000
 l3pt_rpb 0xFC044000 l3pt_kernel 0xFC048000 l2reserv
 0xFC046000
 kstart = 0xFC31, kend = 0xFC855898, kentry =
 0xFC31, numCPUs = 0x1
 CPU Clock at 2000 MHz IntrClockFrequency=1024
 Booting with 1 processor(s)
 ...
 Linux version 2.6.13 (h...@zed.eecs.umich.edu) (gcc version 3.4.3) #1 SMP
 Sun Oct 8 ..
 Brought up 1 CPUs
 ...
 init started:  BusyBox v1.1.0 (2007.03.04-01:07+) multi-call binary
 mounting filesystems...
 EXT2-fs warning: checktime reached, running e2fsck is recommended
 loading script...
 *
 **Two Words: Yes Sir!

 *More Than Two Words: InOrder is booting Linux!!!

 Getting to this point took about30 patches, so I have a lot of
 patch-cleaning work to do before I can place it in the repo.

 Also, just because it boots doesn't mean I've tested it with any kind of
 real workload suite (e.g. SPEC or Parsec).  I don't have the bandwidth to
 complete stress test through all the benchmarks and test but if someone
 tries and something goes wrong, please post to the mailing list and I'll
 do
 my best to help you debug.

 Either way, I think this is a good milestone and I'll be adding the
 patches
 and the regression test for ALPHA_FS/10.linux_boot/InOrder-CPU within the
 next week (or two) and then eventually updating more things in our
 handy-dandy M5 status matrix:
 http://m5sim.org/wiki/index.php/Status_Matrix

 The next big thing I want to get in InOrder before the gem5 official
 merge
 is complete is timing TLB translation and in the longer term microcode
 support. I can't say exactly when those will be added (again, not enough
 time for me to completely focus on this), but if anyone wants to try I
 have
 a good idea of what needs to be done and can assist.



 ___
 m5-dev mailing list
 m5-dev@m5sim.org
 http://m5sim.org/mailman/listinfo/m5-dev

___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


[m5-dev] changeset in m5: python: fix another bug from changes to main.py

2011-04-21 Thread Nathan Binkert
changeset 914389024c33 in /z/repo/m5
details: http://repo.m5sim.org/m5?cmd=changeset;node=914389024c33
description:
python: fix another bug from changes to main.py

diffstat:

 configs/common/cpu2000.py |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r a9d06c894afe -r 914389024c33 configs/common/cpu2000.py
--- a/configs/common/cpu2000.py Wed Apr 20 18:45:03 2011 -0700
+++ b/configs/common/cpu2000.py Wed Apr 20 19:07:44 2011 -0700
@@ -155,7 +155,7 @@
 cwd = process_args.get('cwd')
 
 if not cwd:
-from m5.main import options
+from m5 import options
 cwd = options.outdir
 process_args['cwd'] = cwd
 if not isdir(cwd):
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] what scons can do

2011-04-21 Thread Gabriel Michael Black

Quoting nathan binkert n...@binkert.org:


Anyway, it seems like it would be useful to be able to have multiple
binaries that can be built by scons, specifically the utility stuff and
unit tests. That way we could avoid having a hodge podge of small build
systems which are either isolated or not in not quite the right ways. I
know some of Nate's recent changes suggested this was going to get
easier. Could you quickly summarize what that's all about, Nate?

What changes are you thinking of?  I'm not sure that anything I've
done makes this easier or harder.  I think it's just a matter of
creating SConscript files that the SConstruct file sources.


A mechanism where I can say binary foo needs file bar, and then when I  
tell it to build foo it build's it with bar, and bar doesn't get mixed  
in to other things. I think you're new widget that was like mercurial  
patch guards would help with that, right? You could have a guard for  
each extra target.





Also, I was thinking about how to handle the dependencies/generated
files/custom language issue a little while back, and what I kept coming
back to were schemes where scons would use a cache of dependency
information which it would regenerate if any of the input files which
determined outputs and/or dependencies changed. The problem is that
scons would need to run once and possibly regenerate its cache, and then
run again to actually run. Is this sort of multi-pass setup possible
somehow without major hacks?

I've looked into this in depth and I think that SCons just sucks for
this sort of thing.  I've seen a few different proposals for dealing
with this, but they all seem to suck.  Basically SCons builds the
dependence graph up front and then walks it.  It doesn't seem to be
able to build it on the fly.  (WAF is different in this regard.)  That
said, there are hacks out there to get around this, but I haven't
managed to get them to work and I'm not sure if they're fragile or
not.


That's unfortunate.




When you run for the first time, scons would see that foo.isa.dep
doesn't exist. During it's build phase, it would run foo.isa through the
system and see that it generated foo_exec.cc and bar_exec.cc and put
that into foo.isa.dep (as actual SConscript type code, or flat data,
or...). When scons ran the second time, it would read in foo.isa.dep and
extract the dependencies from it and build that into the graph. It
wouldn't construct foo.isa.dep again since all its inputs were the same,
and it would still capture all those dependencies. This time around, the
larger binary would see that it depended on foo_exec.cc and bar_exec.cc
and that those depend on foo.isa.dep (as a convenient aggregation point
of all *.isa files involved). If foo.isa changed later, foo.isa.dep
would be out of date and have to be regenerated, and then foo_exec.cc
and bar_exec.cc, and then the main binary.

Running SCons twice in the way you suggest would be FAR slower than
just running SLICC and the ISA parser twice the way we do.  Also,
notice that when SLICC is run twice, the modes are very different.
The first time it is run, it does parse the files, but only to figure
out what the dependencies are.  The second time it runs, it is run in
the mode to actually generate the files.  We parse all files twice
anyway to get the dependency information (scanners parse things like
.cc and .hh files to figure out dependencies with #includes, though
they basically just use regexes to do it).  I think you should do this
with the ISA parser.



I'm not sure how SLICC is structured, but it might be hard to get out  
file information without running the whole description, depending on  
how the multi-file output thing is implemented. I wrote an email about  
this a while ago, but basically it depends on if the output files are  
more like a list or a program. #include files are like a list where  
you can scan for them and know what they are without caring about any  
of the other text (or at least are sort of like that). The mulit-file  
output may be a lot more programmable and would be like determining  
what files a program is going to generate at run time. You could build  
in a dry run sort of mode that would just figure that out, but ISA  
descriptions tend to be pretty complicated and that would be a  
maintenance headache. It might still work out, though, since the  
descriptions tend to be relatively quick. That's compared to building  
the ginormous files they produce which can be S.L.O.W., especially if  
you start swapping.


Gabe
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] what scons can do

2011-04-21 Thread nathan binkert
 A mechanism where I can say binary foo needs file bar, and then when I tell
 it to build foo it build's it with bar, and bar doesn't get mixed in to
 other things. I think you're new widget that was like mercurial patch guards
 would help with that, right? You could have a guard for each extra target.
Ah, I see.  I personally wouldn't use any of that.  Don't use Source
or PySource or that sort of thing.  Just set up normal looking
SConscript files with their own targets and such.  The
Source/PySource/etc. stuff is really about building the m5 binary and
dealing with all of the variations that we have.

 I'm not sure how SLICC is structured, but it might be hard to get out file
 information without running the whole description, depending on how the
 multi-file output thing is implemented.
In SLICC, we parse the entire file using the ply grammar and generate
the AST in both phases.  In one phase, you walk the AST simply to
figure out which files will be generated.  In the other, you actually
do the generation.  I'm suggesting that you do that as well, the files
aren't that long and I bet that parsing the ISA desc files and
generating the AST takes a second or so.

  Nate
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] what scons can do

2011-04-21 Thread Gabriel Michael Black

Quoting nathan binkert n...@binkert.org:


A mechanism where I can say binary foo needs file bar, and then when I tell
it to build foo it build's it with bar, and bar doesn't get mixed in to
other things. I think you're new widget that was like mercurial patch guards
would help with that, right? You could have a guard for each extra target.

Ah, I see.  I personally wouldn't use any of that.  Don't use Source
or PySource or that sort of thing.  Just set up normal looking
SConscript files with their own targets and such.  The
Source/PySource/etc. stuff is really about building the m5 binary and
dealing with all of the variations that we have.


Ok.




I'm not sure how SLICC is structured, but it might be hard to get out file
information without running the whole description, depending on how the
multi-file output thing is implemented.

In SLICC, we parse the entire file using the ply grammar and generate
the AST in both phases.  In one phase, you walk the AST simply to
figure out which files will be generated.  In the other, you actually
do the generation.  I'm suggesting that you do that as well, the files
aren't that long and I bet that parsing the ISA desc files and
generating the AST takes a second or so.



That doesn't really fit with how the ISA files work. They get broken  
into an AST, but that gets consumed as it goes, and it has a lot of  
anonymous python in it that just gets executed somehow. I want to move  
more into the python, so the AST will be less and less useful. I'm not  
sure whether the new support will be with explicit language support or  
part of the embedded python. I would prefer the later, but if it's  
harder to use that way language support may be best. Ultimately I want  
to make the ISA stuff a part of regular python scripts instead of the  
python being embedded in the ISA stuff, so in the longer term it will  
probably be best to have that mechanism in regular python.


Gabe
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] what scons can do

2011-04-21 Thread nathan binkert
 That doesn't really fit with how the ISA files work. They get broken into an
 AST, but that gets consumed as it goes,
Does it have to be?

 and it has a lot of anonymous python
 in it that just gets executed somehow. I want to move more into the python,
 so the AST will be less and less useful.
Does the AST not contain enough information to know what files are
being generated?  The anonymous python itself creates files?  That
sounds crazy.

  Nate
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] what scons can do

2011-04-21 Thread Steve Reinhardt
On Thu, Apr 21, 2011 at 2:11 PM, nathan binkert n...@binkert.org wrote:

 Running SCons twice in the way you suggest would be FAR slower than
 just running SLICC and the ISA parser twice the way we do.

Are you sure?  Running SLICC every time just to get a (typically unchanging)
list of files is not exactly instantaneous, and the names of the SLICC
output files hardly ever change.  What about this approach:

1. store the list of output files
2. scons builds a dependence graph based on that stored list
3. when the SLICC files change and actually need to be re-parsed, we compare
the output file list generated as a side effect with the stored list
4. if those lists differ, scons blows itself away and starts over
5. if not, we can get by without parsing the SLICC files at all in the
common case where you're rebuilding the binary and none of the SLICC inputs
have changed

Yes, step 4 is ugly and expensive, but the key is that it should hardly ever
happen.  Make the common case fast, and all that.  If we commit the output
file lists to hg, it wouldn't even need to happen on a clean build.

Just brainstorming...

Steve
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] what scons can do

2011-04-21 Thread nathan binkert
 Are you sure?  Running SLICC every time just to get a (typically unchanging)
 list of files is not exactly instantaneous, and the names of the SLICC
 output files hardly ever change.  What about this approach:
It is pretty fast (and this includes starting the python interpreter)
0.48s user 0.03s system 99% cpu 0.514 total

 1. store the list of output files
 2. scons builds a dependence graph based on that stored list
 3. when the SLICC files change and actually need to be re-parsed, we compare
 the output file list generated as a side effect with the stored list
 4. if those lists differ, scons blows itself away and starts over
 5. if not, we can get by without parsing the SLICC files at all in the
 common case where you're rebuilding the binary and none of the SLICC inputs
 have changed

 Yes, step 4 is ugly and expensive, but the key is that it should hardly ever
 happen.  Make the common case fast, and all that.  If we commit the output
 file lists to hg, it wouldn't even need to happen on a clean build.

 Just brainstorming...

I think you're pre-optimizing.  How long does the ISA parser actually
take to parse the files?

  Nate
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev


Re: [m5-dev] what scons can do

2011-04-21 Thread Gabriel Michael Black

Quoting nathan binkert n...@binkert.org:


That doesn't really fit with how the ISA files work. They get broken into an
AST, but that gets consumed as it goes,

Does it have to be?


Making it not work that way would likely be very painful. The parser  
part is finicky (like they all are in any language) and we have lots  
and lots of very intricate code built on top of it in the form of the  
descriptions themselves.





and it has a lot of anonymous python
in it that just gets executed somehow. I want to move more into the python,
so the AST will be less and less useful.

Does the AST not contain enough information to know what files are
being generated?  The anonymous python itself creates files?  That
sounds crazy.


This may not be quite right, but off hand here is a summary of the isa  
parser's inputs and outputs. Going in, the parser starts with  
main.isa. There are ##includes (there are two #s on purpose) which  
bring in other .isa files. The parser reads in all those files by  
following the ##includes, stitches them together into one huge string,  
and then crunches through it all. As that's being processed, the  
description can read in other files that have, for instance, microcode  
in them. This already hits basically the problem we're talking about  
since the involved are determined by execution, not static landmarks  
like ##include. I solve this problem by manually listing all  
microcode files in the SConscript. It's a nasty hack, but it avoids  
not rebuilding when microcode changes which is even more annoying.


On the output side, the parser generates two files which implement the  
decoder, decoder.hh and decoder.cc. It also outputs one file for each  
CPU model involved that implements the exec (and related) functions.  
These are called something_something_exec.cc I think.


The problem is that for x86 for sure, but also now for ARM and likely  
for any other ISA with a lot of complexity and/or fidelity, those  
output files get to be very, very large. It's easy to run out or RAM,  
especially if scons tries to build more than one at a time or if  
you're on a smaller machine. Then the build grinds to a halt, as does  
everything else. Often the only solutions are to wait until it  
finishes or you die (whichever comes first) or rebooting the machine  
and trying again with more conservative settings.


What this mechanism would do would be to allow you to put different  
portions of the output into different files which would be compiled  
independently. Then scons compiling three things at once is equivalent  
to three normal files at once, not a million lines of code all at once.


To do that, you have to decide how to split things up so they still  
build. You could try hacking things up in an automated way, but that  
would likely either be overly restrictive, ineffective, incorrect, or  
all three. My plan is to expose the idea of different files to the ISA  
description author so that they can choose to put all the, say,  
floating point loads and stores together along with their utility  
functions. These may not all be defined in the same place or even in  
the same directory since there are ordering constraints in python as  
well as in the resulting C++. It might be that you define output files  
in a fixed place (def output floatMem, for instance) and then refer to  
them later when it comes time to put C++ someplace. That might make  
the most sense. It could also be that you have batches of similarly  
named output clusters (these will likely involve more than one file at  
a time, like a .cc and a .hh) and you'd want to generate them all  
programatically. I'm not sure exactly what it would look like to  
select an output file either. You might want to just put down markers  
that say, essentially, henceforth output goes in floatMem. Or pass  
an output cluster name into the outputting function (whatever that  
looks like).


It might work best in the near to mid term to put in static, ISA  
language defined declarations of output files which would be feasible  
to scan. In the mid to long term, though, I want to move away from  
having a custom language and move towards having the same machinery  
(extended and parameterized more) exposed as a module or something  
inside regular python scripts. Maybe something ala scons's SConscripts  
which are regular python that run in an armature, sort of.


I would be hesitant to make the ISA descriptions open and write to  
files themselves directly, but primarily because that would be  
cumbersome and error prone. I don't think we should design it out,  
though, unless it's just too evil to support.


Gabe
___
m5-dev mailing list
m5-dev@m5sim.org
http://m5sim.org/mailman/listinfo/m5-dev