Shoot, my bad on the breaking. I was so focused on making sure the script worked correctly that I didn't pay attention to the code changes.
Basically, the script in this changeset will take separate checkpoints and make them into a single multiprogrammed checkpoint, tested and used on ALPHA only. In order for the tlb to work correctly, each process needs a separate M5_pid that matches up with the fixed up asns in the checkpoint (which I guess asns don't exist for every TLB type), and the DTB misc reg also has to be set for things to work correctly. Even in non-alpha, page table entries match up with M5_pid in PageTable::allocate() so an appropriate distinction is required. I guess what I can do is make the script print out a fake m5_pid that matches appropriately the fixed up values and have unserialize read it in the process instead of in the page table. I'll try to get that done asap, I have some things I have to do at work today. For now, a quick fix would be to wrap the #if THE_ISA if around the asn statement as well. Lisa On Tue, Jan 19, 2010 at 12:34 AM, Gabe Black <[email protected]> wrote: > This change seems to have broken the build for everything except Alpha > (no regressions?). I don't like that #if THE_ISA. What is it doing, and > why can't it be done in some Alpha specific body of code? > > Gabe > > Hsu, Lisa wrote: > > > > This is the last one. > > > > Lisa > > > > -----Original Message----- > > From: [email protected] on behalf of [email protected] > > Sent: Mon 1/18/2010 2:40 PM > > To: Hsu, Lisa > > Subject: changeset in m5: util: make a generic checkpoint aggregator > > that... > > > > You are not allowed to post to this mailing list, and your message has > > been automatically rejected. If you think that your messages are > > being rejected in error, contact the mailing list owner at > > [email protected]. > > > > > > > > ------------------------------------------------------------------------ > > > > Subject: > > changeset in m5: util: make a generic checkpoint aggregator that... > > From: > > "Hsu, Lisa" <[email protected]> > > Date: > > Mon, 18 Jan 2010 14:40:15 -0800 > > To: > > <[email protected]> > > > > To: > > <[email protected]> > > > > > > changeset 5a0e3a283826 in /z/repo/m5 > > details: http://repo.m5sim.org/m5?cmd=changeset;node=5a0e3a283826 > > description: > > util: make a generic checkpoint aggregator that can aggregate > > different cpts into one multi-programmed cpt. Make minor changes to > > serialization/unserialization to get it to work properly. Note that > > checkpoints were made with a comment at the beginning with // - this > > must be changed to ## to work properly with the python config parser > > in the aggregator. > > > > diffstat: > > > > 3 files changed, 151 insertions(+), 2 deletions(-) > > src/mem/page_table.cc | 12 +++ > > src/sim/serialize.cc | 2 > > util/checkpoint-aggregator.py | 139 > > +++++++++++++++++++++++++++++++++++++++++ > > > > diffs (176 lines): > > > > diff -r 5aec45d0fc24 -r 5a0e3a283826 src/mem/page_table.cc > > --- a/src/mem/page_table.cc Tue Jan 12 10:53:02 2010 -0800 > > +++ b/src/mem/page_table.cc Mon Jan 18 14:30:31 2010 -0800 > > @@ -222,6 +222,16 @@ > > entry->unserialize(cp, csprintf("%s.Entry%d", > > process->name(), i)); > > pTable[vaddr] = *entry; > > ++i; > > - } > > + } > > + > > + process->M5_pid = pTable[vaddr].asn; > > + > > +#if THE_ISA == ALPHA_ISA > > + // The IPR_DTB_ASN misc reg must be set in Alpha for the tlb to work > > + // correctly > > + int id = process->contextIds[0]; > > + ThreadContext *tc = process->system->getThreadContext(id); > > + tc->setMiscRegNoEffect(IPR_DTB_ASN, process->M5_pid << 57); > > +#endif > > } > > > > diff -r 5aec45d0fc24 -r 5a0e3a283826 src/sim/serialize.cc > > --- a/src/sim/serialize.cc Tue Jan 12 10:53:02 2010 -0800 > > +++ b/src/sim/serialize.cc Mon Jan 18 14:30:31 2010 -0800 > > @@ -422,7 +422,7 @@ > > time_t t = time(NULL); > > if (!outstream.is_open()) > > fatal("Unable to open file %s for writing\n", cpt_file.c_str()); > > - outstream << "// checkpoint generated: " << ctime(&t); > > + outstream << "## checkpoint generated: " << ctime(&t); > > > > globals.serialize(outstream); > > SimObject::serializeAll(outstream); > > diff -r 5aec45d0fc24 -r 5a0e3a283826 util/checkpoint-aggregator.py > > --- /dev/null Thu Jan 01 00:00:00 1970 +0000 > > +++ b/util/checkpoint-aggregator.py Mon Jan 18 14:30:31 2010 -0800 > > @@ -0,0 +1,139 @@ > > +#! /usr/bin/env python2.6 > > + > > +from ConfigParser import ConfigParser > > +import gzip > > + > > +import sys, re, optparse, os > > + > > +class myCP(ConfigParser): > > + def __init__(self): > > + ConfigParser.__init__(self) > > + > > + def optionxform(self, optionstr): > > + return optionstr > > + > > +def aggregate(options, args): > > + merged = myCP() > > + page_ptr = 0 > > + > > + allfiles = os.listdir(os.getcwd()) > > + cpts = [] > > + for arg in args: > > + found = False > > + for f in allfiles: > > + if re.compile("cpt." + arg + ".\d+").search(f): > > + found = True > > + cpts.append(f) > > + break > > + if not found: > > + print "missing checkpoint: ", arg > > + sys.exit(1) > > + > > + dirname = "-".join([options.prefix, "cpt"]) > > + print dirname > > + agg_name = "-".join(args) > > + print agg_name > > + fullpath = os.path.join("..", dirname, "cpt." + agg_name + ".10000") > > + if not os.path.isdir(fullpath): > > + os.system("mkdir -p " + fullpath) > > + > > + myfile = open(fullpath + "/system.physmem.physmem", "wb+") > > + merged_mem = gzip.GzipFile(fileobj=myfile, mode="wb") > > + > > + max_curtick = 0 > > + when = 0 > > + for (i, arg) in enumerate(args): > > + config = myCP() > > + config.readfp(open(cpts[i] + "/m5.cpt")) > > + > > + for sec in config.sections(): > > + if re.compile("cpu").search(sec): > > + newsec = re.sub("cpu", "cpu" + str(i), sec) > > + merged.add_section(newsec) > > + > > + items = config.items(sec) > > + for item in items: > > + if item[0] == "ppn": > > + if config.getint(sec, "tag") != 0: > > + merged.set(newsec, item[0], int(item[1]) > > + page_ptr) > > + continue > > + elif item[0] == "asn": > > + tmp = > > re.compile("(.*).Entry(\d+)").search(sec).groups() > > + if config.has_option(tmp[0], "nlu"): > > + size = config.getint(tmp[0], "nlu") > > + if int(tmp[1]) < size: > > + merged.set(newsec, item[0], i) > > + continue > > + else: > > + merged.set(newsec, item[0], i) > > + continue > > + merged.set(newsec, item[0], item[1]) > > + elif sec == "system": > > + pass > > + elif sec == "Globals": > > + tick = config.getint(sec, "curTick") > > + if tick > max_curtick: > > + max_curtick = tick > > + when = config.getint("system.cpu.tickEvent", > "_when") > > + else: > > + if i == 0: > > + print sec > > + merged.add_section(sec) > > + for item in config.items(sec): > > + merged.set(sec, item[0], item[1]) > > + if item[0] == "curtick": > > + merged.optionxform(str("curTick")) > > + elif item[0] == "numevents": > > + merged.optionxform(str("numEvents")) > > + > > + page_ptr = page_ptr + int(config.get("system", "page_ptr")) > > + > > + ### memory stuff > > + f = open(cpts[i] + "/system.physmem.physmem", "rb") > > + gf = gzip.GzipFile(fileobj=f, mode="rb") > > + bytes = int(config.get("system", "page_ptr")) << 13 > > + print "bytes to be read: ", bytes > > + > > + bytesRead = gf.read(int(config.get("system", "page_ptr")) << 13) > > + merged_mem.write(bytesRead) > > + > > + gf.close() > > + f.close() > > + > > + merged.add_section("system") > > + merged.set("system", "page_ptr", page_ptr) > > + print "WARNING: " > > + print "Make sure the simulation using this checkpoint has at least " > > + if page_ptr > (1<<20): > > + print "8G ", > > + elif page_ptr > (1<<19): > > + print "4G ", > > + elif page_ptr > (1<<18): > > + print "2G ", > > + elif page_ptr > (1<<17): > > + print "1G ", > > + elif page_ptr > (1<<16): > > + print "512KB ", > > + else: > > + print "this is a small sim, you're probably fine", > > + print "of memory." > > + > > + merged.add_section("Globals") > > + merged.set("Globals", "curTick", max_curtick) > > + > > + for i in xrange(len(args)): > > + merged.set("system.cpu" + str(i) + ".tickEvent", "_when", when) > > + > > + merged.write(file(fullpath + "/m5.cpt", "wb")) > > + merged_mem.close() > > + myfile.close() > > + > > +if __name__ == "__main__": > > + > > + parser = optparse.OptionParser() > > + parser.add_option("--prefix", type="string", default="agg") > > + > > + (options, args) = parser.parse_args() > > + > > + aggregate(options, args) > > + > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > m5-dev mailing list > > [email protected] > > http://m5sim.org/mailman/listinfo/m5-dev > > > > _______________________________________________ > m5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/m5-dev > >
_______________________________________________ m5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/m5-dev
