Hello, I am trying to restore from checkpoint and run my benchmark immediately without entering the commands in the simulated system myself. I came across this thread and realized that hack_back_ckpt.rcs<http://grok.gem5.org/xref/gem5/configs/boot/hack_back_ckpt.rcS> is supposedly doing what I need. I was able to take checkpoints automatically using this script, but I'm not sure how to direct it to run my benchmark afterwards. Can anybody give me a clue on that?
Thanks. On Sat, Sep 29, 2012 at 2:24 PM, Jun Pang <pang...@cs.duke.edu> wrote: > Hi Joel, > > Thank you very much. I really appreciate your help. I need to think about > this. > > Best, > Jun > > > On Fri, Sep 28, 2012 at 8:22 PM, Joel Hestness <jthestn...@gmail.com>wrote: > >> Hi Jun, >> >> 1. Recompiled kernel x86_64-vmlinux-2.6.28.4.smp with SMP enabled and >>> CONFIG_NR_CPUS=512. >>> >> >> Ok. If my memory serves me correctly, you will need to also change the >> CONFIG_NR_CPUS range in the kconfigs for kernel 2.6.28.4. I don't recall >> the details on where to find these files, but the kconfigs specify which >> variables a config file can contain and the appropriate ranges within which >> the variables can be set. I believe the upper limit in the kconfigs for >> 2.6.28.4 was CONFIG_NR_CPUS=32. If this is the case the system may look >> like it is still booting more than 32 cores, but Linux will only show 32 of >> them as available after you've booted the system. >> >> >> >>> 2.Made a checkpoint right before "Loading script..." and I guess this is >>> similar to what /configs/boot/hack_back_ckpt.rcS does. I modified the file >>> system file: etc/init.d/rsC to create the checkpoint. >>> >> >> Cool, that works. The /configs/boot/hack_back_ckpt.rcS script will also >> work with any disk image and any ISA for which gem5 has checkpointing >> capability. In the future, it might be easier to use than modifying the >> disk image to collect checkpoints. >> >> >> 3.The interrupt devices do give me problems. >>> - Local and IO APIC have only 8 bits which means they will support >>> up to 256 cores, but the IO_APIC.id is set to be the numCPUs. This means I >>> cannot really use 256, because it is beyond 8 bits range. I tried to make >>> IO_APIC.id =numCPU-1 and boot 256 cores, but then I got error: "panic: >>> Legacy mode interrupts with error codes aren't implementde" when the >>> system boots the 256th core. However, it works correctly with 1 or 2 cores. >>> >> >> You're correct that the IO APIC reserves a bit in the ID mask so that >> only 7 bits are available (i.e. up to 128 cores). I'm not sure what it >> would take to remove this restriction, though I believe we investigated >> using the extra bit and decided it was going to be difficult to make it >> work. If you really need to use 256 cores, I would recommend starting by >> reading up on the Intel i8042, i82094aa, i8237, i8254, and i8259 devices, >> and also trying to identify the new devices that have replaced these (which >> may be capable of booting more than 128 cores in a single system). >> >> >> -My ideal GPU clock rate is 5GHz, but I will get an error with that >>> frequency:"MP-BIOS bug: 8254 timer not connected to IO-APIC". I have >>> searched around and it seems to be an known issue with some kernels. The >>> easiest way to fix it is to disable APIC which is not allowed in SMP I >>> guess. So I reduced the frequency to be 4GHz.... >>> >> >> I haven't run into this problem before, so I'm afraid I can't help here. >> >> >> Please let me know if what I have done so far makes sense or not. Any >>> suggestion will be greatly appreciated. I am still reading the code and >>> trying to understand the APIC part better. >>> >> >> Sounds good so far. Let me know if there's anything else I can help with, >> >> Joel >> >> >> >> >>> On Thu, Sep 27, 2012 at 3:14 PM, Joel Hestness <jthestn...@gmail.com>wrote: >>> >>>> Hi Jun, >>>> Can you let us know which kernel version you're working with or where >>>> you got the kernel that you're trying to boot? >>>> >>>> There are instructions here (http://gem5.org/Linux_kernel) for >>>> building Linux for x86 and gem5. Currently, there aren't any known ways to >>>> speed up Linux boot, since the kernel waits for interrupts to calibrate >>>> delay loops already. >>>> >>>> Note also that there are probably some limitations to building >>>> kernels to support more than 64 or 256 cores. Sometimes the kernel >>>> kconfigs set default ranges for things like core count that you cannot >>>> tweak just by changing the config files. Further, the interrupt devices >>>> currently in gem5 (./src/dev/x86/*) probably have a core count limitation, >>>> since they are implemented to function the same as devices used in current >>>> real hardware. You might want to check into these, because if you're >>>> running into a core count limitation, something may be hung up in the >>>> system you're booting with 255 cores. >>>> >>>> Overall, 3-5 hours to boot 64 cores doesn't sound unreasonable with >>>> the atomic or timing CPUs. The out-of-order core will take multiple times >>>> longer to boot Linux because of the added core detail, so we do not suggest >>>> that you try this, but instead you take checkpoints after Linux boot and >>>> restore from the checkpoint into the benchmarks you'd like to run. (Check >>>> out the runscript ./configs/boot/hack_back_ckpt.rcS, which makes this >>>> checkpointing easy) >>>> >>>> Hope this helps, >>>> Joel >>>> >>>> >>>> >>>> >>>> On Thu, Sep 27, 2012 at 12:42 PM, Jun Pang <pang...@cs.duke.edu> wrote: >>>> >>>>> It's been over one day already, and my 255 cores system is still >>>>> booting. Does anyone know the answer to speed up the booting? >>>>> >>>>> Thanks, >>>>> >>>>> Jun >>>>> >>>>> >>>>> On Tue, Sep 25, 2012 at 11:01 PM, Jun Pang <pang...@cs.duke.edu>wrote: >>>>> >>>>>> Hi guys, >>>>>> >>>>>> I want to boot more than 256 cores in X86 FS with ruby, however, it's >>>>>> slow. More than 3 hours for ~60 cores so far without ruby... >>>>>> >>>>>> I read this from X86 session of http://gem5.org/Architecture_Support: >>>>>> >>>>>> "patches are available for speeding up boot" >>>>>> >>>>>> I have searched around, but couldn't find any patches so far. Does >>>>>> this refer to checkpoint and fast-forwarding? or there are some real >>>>>> patches I haven't found yet. If so, I really appreciate it if someone >>>>>> could >>>>>> point me to the right place to download the patches. >>>>>> >>>>>> Or are there any other techniques I can make the boot faster? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jun >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> gem5-users mailing list >>>>> gem5-users@gem5.org >>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>> >>>> >>>> >>>> >>>> -- >>>> Joel Hestness >>>> PhD Student, Computer Architecture >>>> Dept. of Computer Science, University of Wisconsin - Madison >>>> http://www.cs.utexas.edu/~hestness >>>> >>>> _______________________________________________ >>>> gem5-users mailing list >>>> gem5-users@gem5.org >>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>> >>> >>> >>> _______________________________________________ >>> gem5-users mailing list >>> gem5-users@gem5.org >>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>> >> >> >> -- >> Joel Hestness >> PhD Student, Computer Architecture >> Dept. of Computer Science, University of Wisconsin - Madison >> http://www.cs.utexas.edu/~hestness >> >> _______________________________________________ >> gem5-users mailing list >> gem5-users@gem5.org >> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >> > > > _______________________________________________ > gem5-users mailing list > gem5-users@gem5.org > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >
_______________________________________________ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
