Hi Shehab, IIRC, there are some issues when using classic caches + x86 + multiple cores on full system mode. I suggest using Ruby (MESI_two_level or MOESI_hammer) for FS simulations.
Jason On Fri, Sep 6, 2019 at 11:24 AM Shehab Elsayed <shehaby...@gmail.com> wrote: > My latest experiments are with the classical memory system, but I remember > trying Ruby and it was not different. I am using kernel 4.8.13 and > ubuntu-16.04.1-server-amd64 disk image. I am using Pthreads for my Hello > World program. > > > On Fri, Sep 6, 2019 at 1:13 PM Pouya Fotouhi <pfoto...@ucdavis.edu> wrote: > >> Hi Shehab, >> >> Can you confirm a few details about the configuration you are using? Are >> you using classic caches or Ruby? What is the kernel version and disk image >> you are using? What is the implementation of your "multithreaded hello >> world" (are you using OMP)? >> >> Best, >> >> On Fri, Sep 6, 2019 at 8:58 AM Shehab Elsayed <shehaby...@gmail.com> >> wrote: >> >>> First of all, thanks for your replies, Ryan and Jason. >>> >>> I have already pulled the latest changes by Pouya and the problem still >>> persists. >>> >>> As for checkpointing, I was originally doing exactly what Jason >>> mentioned and ran into the same problem. I then switched to not >>> checkpointing just to avoid any problems that might be caused by >>> checkpointing (if any). My plan was to go back to checkpointing after >>> proving that it works without it. >>> >>> However, the problem doesn't seem to be related to KVM as linux boots >>> reliable every time. The problem happens after the benchmarks starts >>> execution and it seems to be happening only when running multiple cores >>> (>=4). My latest experiments with a single core and 8 threads for the >>> benchmark seem to be working fine. But once I increase the number of >>> simulated cores problems happen. >>> >>> Also, I have posted a link to the repo I am using to run my tests in a >>> previous message. I have also added 2 debug traces with the Exec flag for a >>> working and non-working examples. >>> >>> >>> On Fri, Sep 6, 2019 at 11:28 AM Jason Lowe-Power <ja...@lowepower.com> >>> wrote: >>> >>>> Hi Shehab, >>>> >>>> One quick note: There is *no way* to have deterministic behavior when >>>> running with KVM. Since you are using the hardware, the underlying host OS >>>> will influence the execution path of the workload. >>>> >>>> To try to narrow down the bug you're seeing, you can try to take a >>>> checkpoint after booting with KVM. Then, the execution from the checkpoint >>>> should be deterministic since it is 100% in gem5. >>>> >>>> BTW, I doubt you can run the KVM CPU in a VM since this would require >>>> your hardware and the VM to support nested virtualization. There *is* >>>> support for this in the Linux kernel, but I don't think it's been widely >>>> deployed outside of specific cloud environments. >>>> >>>> One other note: Pouya has pushed some changes which implement some x86 >>>> instructions that were causing issues for him. You can try with the current >>>> gem5 mainline to see if that helps. >>>> >>>> Cheers, >>>> Jason >>>> >>>> On Fri, Sep 6, 2019 at 8:22 AM Shehab Elsayed <shehaby...@gmail.com> >>>> wrote: >>>> >>>>> That's interesting. Are you using Full System as well? I don't think >>>>> FS behavior is supposed to be so dependent on the host environment! >>>>> >>>>> On Fri, Sep 6, 2019 at 11:16 AM Gambord, Ryan < >>>>> gambo...@oregonstate.edu> wrote: >>>>> >>>>>> I have found that gem5 behavior is sensitive to the execution >>>>>> environment. I now run gem5 inside an ubuntu vm on qemu and have had much >>>>>> more consistent results. I haven't tried running kvm gem5 inside a kvm >>>>>> qemu >>>>>> vm, so not sure how that works, but might be worth trying. >>>>>> >>>>>> Ryan >>>>>> >>>>>> >>>>>> On Fri, Sep 6, 2019, 08:07 Shehab Elsayed <shehaby...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I was wondering if anyone is running into the same problem or if >>>>>>> anyone has any suggestions on how to proceed with debugging this >>>>>>> problem. >>>>>>> >>>>>>> On Mon, Jul 29, 2019 at 4:57 PM Shehab Elsayed <shehaby...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Sorry for the spam. I just forgot to mention that the system >>>>>>>> configuration I am using is mainly from >>>>>>>> https://github.com/darchr/gem5/tree/jason/kvm-testing/configs/myconfigs. >>>>>>>> <https://github.com/darchr/gem5/tree/jason/kvm-testing/configs/myconfigs> >>>>>>>> >>>>>>>> >>>>>>>> Shehab Y. Elsayed, MSc. >>>>>>>> PhD Student >>>>>>>> The Edwards S. Rogers Sr. Dept. of Electrical and Computer >>>>>>>> Engineering >>>>>>>> University of Toronto >>>>>>>> E-mail: shehaby...@gmail.com >>>>>>>> <https://webmail.rice.edu/imp/message.php?mailbox=INBOX&index=11#> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jul 29, 2019 at 4:08 PM Shehab Elsayed < >>>>>>>> shehaby...@gmail.com> wrote: >>>>>>>> >>>>>>>>> I have set up a repo with gem5 that demonstrates the problem. The >>>>>>>>> repo includes the latest version of gem5 from gem5's github repo with >>>>>>>>> a few >>>>>>>>> patches applied to get KVM working together with the kernel binary >>>>>>>>> and disk >>>>>>>>> image I am using. You can get the repo at >>>>>>>>> https://github.com/ShehabElsayed/gem5_debug.git. >>>>>>>>> <https://github.com/ShehabElsayed/gem5_debug.git> >>>>>>>>> >>>>>>>>> These steps should reproduce the problem: >>>>>>>>> 1- scons build/X86/gem5.opt >>>>>>>>> 2- ./scripts/get_fs_stuff.sh >>>>>>>>> 3- ./scripts/run_fs.sh 8 >>>>>>>>> >>>>>>>>> I have also included sample m5term outputs for both a 2 thread run >>>>>>>>> (m5out_2t) and an 8 thread run (m5out_8t) >>>>>>>>> >>>>>>>>> Any help is really appreciated. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jul 23, 2019 at 11:01 AM Shehab Elsayed < >>>>>>>>> shehaby...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> When I enable the Exec debug flag I can see that it seems to be >>>>>>>>>> stuck in a spin lock (queued_spin_lock_slowpath) >>>>>>>>>> >>>>>>>>>> On Fri, Jul 19, 2019 at 5:28 PM Shehab Elsayed < >>>>>>>>>> shehaby...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hello All, >>>>>>>>>>> >>>>>>>>>>> I have a gem5 X86 full system set up that starts with KVM cores >>>>>>>>>>> and then switches to O3 cores once the benchmark reaches the region >>>>>>>>>>> of >>>>>>>>>>> interest. Right now I am testing with a simple multithreaded >>>>>>>>>>> hello world benchmark. Sometimes the benchmark completes >>>>>>>>>>> successfully while >>>>>>>>>>> others gem5 just seems to hang after starting the benchmark. I >>>>>>>>>>> believe it >>>>>>>>>>> is still executing some instructions but without making any >>>>>>>>>>> progress. The >>>>>>>>>>> chance of this behavior (indeterminism) happening increases as >>>>>>>>>>> the number of simulated cores or the number of threads created by >>>>>>>>>>> the >>>>>>>>>>> benchmark increases. >>>>>>>>>>> >>>>>>>>>>> Any ideas what might be the reason for this or how I can start >>>>>>>>>>> debugging this problem? >>>>>>>>>>> >>>>>>>>>>> Note: I have tried the patch in https://gem5-review.googlesource >>>>>>>>>>> .com/c/public/gem5/+/19568 but the problem persists. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>> gem5-users mailing list >>>>>>> gem5-users@gem5.org >>>>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>>> >>>>>> _______________________________________________ >>>>>> gem5-users mailing list >>>>>> gem5-users@gem5.org >>>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>> >>>>> _______________________________________________ >>>>> gem5-users mailing list >>>>> gem5-users@gem5.org >>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>> >>>> _______________________________________________ >>>> gem5-users mailing list >>>> gem5-users@gem5.org >>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>> >>> _______________________________________________ >>> gem5-users mailing list >>> gem5-users@gem5.org >>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >> >> >> >> -- >> Pouya Fotouhi >> PhD Candidate >> Department of Electrical and Computer Engineering >> University of California, Davis >> _______________________________________________ >> gem5-users mailing list >> gem5-users@gem5.org >> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users > > _______________________________________________ > gem5-users mailing list > gem5-users@gem5.org > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
_______________________________________________ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
