[gem5-users] Re: Using NIC in gem5 with x86 architecture
Hi Varun, for what it's worth, we've been using the Intel e1000/igb NIC in gem5 with regular Linux for a while without problems, I would start there unless you really want the other NIC. We have some working (albeit complicated) full system dist gem5 (for a slightly dated gem5 version) scripts with them here: https://github.com/simbricks/gem5/blob/main/configs/simbricks/dist.py https://github.com/simbricks/gem5/blob/main/util/dist/test/run_x86.sh In general the new gem5-library style configurations for x86 full system config seem to be much easier to understand and use, e.g. here: https://github.com/gem5/gem5/blob/stable/configs/example/gem5_library/x86-ubuntu-run.py Just creating multiple systems and adding the PCI NICs should be straight forward. Finally, if you plan to do larger simulations with many hosts etc. you might want to have a look at SimBricks https://simbricks.github.io/ which also includes some newer NICs that you can plug into gem5. Hope this helps, Antoine On Tue, May 16 18:12, Varun Chandrashekhar Gohil via gem5-users wrote: > Hello, > > The NIC module provided in gem5 works for the ARM architecture. However, I > want to use it to simulate a cluster of multiple x86 machines. > This email thread > (https://www.mail-archive.com/gem5-users@gem5.org/msg12680.html) explains how > to port the NIC module to x86. It requires using a specific configuration > file linux-2.6.28.4 configuration file (from this link: > http://www.m5sim.org/dist/current/x86/config-x86.tar.bz2) > > This link is invalid now and I could not find it in the gem5 resources. Could > someone please point me to the current link? > > Thank you, > Varun > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org -- Antoine Kaufmann signature.asc Description: PGP signature ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] cache-coherent numa config with classic memory system
Hi everyone, I am currently trying to build gem5 config with the classic memory/cache system that reasonbly resembles a typical multi-socket x86 numa server, but have so far not been successful. (I know Ruby has much more flexiblity here, but I would also like a config with classic components if at all possible). Conceptually I was expecting this should be straight forward: I create a coherent xbar as the main system bus. For each numa node I create a separate xbar where I connect the node's memory controller and local PCI devices, both PIO and DMA, along with the shared L3 cache. The bit I have not been able to figure out yet, is how to connect the per-numa-node xbars to the system xbar. Initially I was planning on using a pair of bridges (one per direction) to connect each numa xbar to the system bus. The problem here is that the bridges seem to be able to only advertise statically configured address ranges, fine for the memory controllers but bad for PCI bars, which are configured dynamically. Directly connecting both cpu and mem-side ports of the busses cross-wise clearly also does not work because of overlapping address ranges once updates propagate. (i.e. once numa node 0 receives the ranges from numa node 1 it will advertise to the system xbar as well and then there are two ports with overlapping ranges). I've also experimented with using the default port on the per-numa-node xbars to connect to the system bus, but also without success. So far I have not found a way to address this with existing classic memory system components (despite coherent_xbar.hh mentioning that it can be used to model things like qpi or hypertransport). Am I missing something obvious here? Alternatively I am thinking of implementing a custom "bi-directional" bridge component that will only forward address ranges in the direction where it sees them arrive first. I would also appreciate any other ideas. Thanks, Antoine -- Antoine Kaufmann signature.asc Description: PGP signature ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org
[gem5-users] Re: Modifying simulation structure after instantiation/checkpoint restore
Thanks for the quick response Sam! Yeah I saw that that also seems to be how the CPU swapping is implemented, and for switching out a few objects this seems like the easiest way to get this to work. But my use case is a bit messy and I want to make larger structural changes. So I am wondering if this is something that is doable in principle within the object/config model, or if I need to figure out some workaround. For a bit more context: we're trying to partition one larger multi-core simulation configuration into multiple smaller parts that we connect together (and synchronize) at the memory ports going across configurations. We have the interconnections and synchronization working for SE configs where we manually configure each partition, but I'm trying to figure out the simplest way of doing this for full-system, where the initialization in the workload etc. really needs the global view of all thread contexts etc. so I was hoping we could just create it and initialize it in one process, and then checkpoint, and then restore the checkpoint for each partition but delete the unneeded objects and add in the "proxy objects" connecting partitions. If that does not work the initialization gets a fair bit more complex, which I would like to avoid if possible. So if anyone has any pointers to indicate that making such changes to the simulation structure are generally possible or impossible that would be very useful before we get too deep down into the weeds. :-) Thanks Antoine On Tue, Apr 12 15:49, Thomas, Samuel via gem5-users wrote: > Hi Antoine, > > We had a similar use case in our work. Our solution was to include all of > the objects that we wanted to in the system and implement them so that they > forward all packets in the mode that you don't care about. For us, we > wanted to include an object that we didn't want to do anything during > fast-forward mode, so our implementation of its port functions that handle > atomic requests merely sends the packets along without doing any > processing. In theory, you could do the same with timing packets if you > don't want the object to be in the system during an atomic mode > fast-forward but not in the system for timing mode. > > I hope this helps! > > Best, > Sam > > On Tue, Apr 12, 2022 at 3:43 PM Antoine Kaufmann via gem5-users < > gem5-users@gem5.org> wrote: > > > Hi everyone, > > > > I have a quick conceptual question about gem5 configuration: are there > > restrictions about what can be modified in an already instantiated and > > initialized gem5 config, say after restoring from a checkpoint. I know > > there is > > some special cases such as replacing CPUs on restore. Is this something > > that > > gem5 supports more generally, e.g. swapping out, adding, > > connecting/disconnecting, or removing SimObjects in a simulation? Or is > > making > > changes to the simulation structure generally a problem? > > > > The use-case I'm looking at is similar to the CPU swapping, but wih other > > objects as well, basically boot system with a fast config, and then swap > > out > > other things too for more detailed versions. (setting up logical state for > > the > > replacement simobjects is something I have in hand, so I'm only wondering > > about > > whether gem5 will be unhappy if I attempt to make such structural changes > > to a > > simulation). > > > > Thanks in advance, > > Antoine > > > > -- > > Antoine Kaufmann > > > > ___ > > gem5-users mailing list -- gem5-users@gem5.org > > To unsubscribe send an email to gem5-users-le...@gem5.org > > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s > ___ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-le...@gem5.org > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s -- Antoine Kaufmann signature.asc Description: PGP signature ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Modifying simulation structure after instantiation/checkpoint restore
Hi everyone, I have a quick conceptual question about gem5 configuration: are there restrictions about what can be modified in an already instantiated and initialized gem5 config, say after restoring from a checkpoint. I know there is some special cases such as replacing CPUs on restore. Is this something that gem5 supports more generally, e.g. swapping out, adding, connecting/disconnecting, or removing SimObjects in a simulation? Or is making changes to the simulation structure generally a problem? The use-case I'm looking at is similar to the CPU swapping, but wih other objects as well, basically boot system with a fast config, and then swap out other things too for more detailed versions. (setting up logical state for the replacement simobjects is something I have in hand, so I'm only wondering about whether gem5 will be unhappy if I attempt to make such structural changes to a simulation). Thanks in advance, Antoine -- Antoine Kaufmann signature.asc Description: PGP signature ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Re: Working X86 multi-core full system config with caches?
Hey Bobby, Thank you very much for your prompt reply. At least your example below "just worked", wow a very new and different gem5 experience. Really loooking forward to the new infrastructure. This was what I hoping for, now I have a known working configuration and can start tracking down the differences to our config and hopefully get that to work. I will report back if I do not succeed. :-) Thanks again, Antoine On Thu, Oct 28 15:06, Bobby Bruce wrote: > Hey Antoine, > > First of all, multi-core TimingSimple with classic caches will not work, > you have to use Ruby or run a single-core simulation. > > I think we have something in development which can help you out. You can > run a full-system X86 Ubuntu simulation with the Ruby MESI Two Level > protocol with the following: > > ``` > git clone https://gem5.googlesource.com/public/gem5 > cd gem5 > git checkout develop > scons build/X86_MESI_Two_Level/gem5.opt -j`nproc` > ./build/X86_MESI_Two_Level/gem5.opt > configs/example/gem5_library/x86-ubuntu-run.py > ``` > > If you have KVM on your system this will boot Ubuntu with KVM then run a > very simple script using the Timing Simple CPU. If you don't have KVM then > you can alter the simulation at line 94 in the > `configs/example/gem5_library/x86-ubuntu-run.py` file, and just run the > whole boot with the Timing Simple CPU. > > The `x86-ubuntu-run.py` script builds upon our gem5 Python Library, which > is still under development, but you can look into it (under > `src/python/gem5`) to see how it all fits together and works. The > `x86-ubuntu-run.py` script itself should be relatively self-explanatory, > but please do not hesitate to get back to me if there's something you don't > understand. > > Getting a full-system setup working is _tough_ if building a simulation > from scratch which is part of the reason we're moving more towards > providing a library for such things. > > Kind regards, > Bobby > -- > Dr. Bobby R. Bruce > Room 3050, > Kemper Hall, UC Davis > Davis, > CA, 95616 > > web: https://www.bobbybruce.net > > > On Thu, Oct 28, 2021 at 10:55 AM Antoine Kaufmann via gem5-users < > gem5-users@gem5.org> wrote: > > > Hi Everyone, > > > > Do we have any known-working configurations for x86 multi-core full system > > simulations with caches? > > > > We have successfully been using single core configs for more than a year > > now, > > but our attempts at anything multi-core have so far not been successful, > > with > > problems ranging from gem5 crashing to Linux getting stuck at boot. > > > > We have been using the TimingSimple CPU and tried both classic and ruby > > with > > various protocols. > > > > Does anyone have pointers to working configurations? Or is this something > > that > > is known to be fundamentally broken? > > > > Thanks in advance, > > Antoine > > > > -- > > Antoine Kaufmann > > > > ___ > > gem5-users mailing list -- gem5-users@gem5.org > > To unsubscribe send an email to gem5-users-le...@gem5.org > > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s -- Antoine Kaufmann signature.asc Description: PGP signature ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] Working X86 multi-core full system config with caches?
Hi Everyone, Do we have any known-working configurations for x86 multi-core full system simulations with caches? We have successfully been using single core configs for more than a year now, but our attempts at anything multi-core have so far not been successful, with problems ranging from gem5 crashing to Linux getting stuck at boot. We have been using the TimingSimple CPU and tried both classic and ruby with various protocols. Does anyone have pointers to working configurations? Or is this something that is known to be fundamentally broken? Thanks in advance, Antoine -- Antoine Kaufmann signature.asc Description: PGP signature ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
[gem5-users] PCI operation ordering
Hi, We have been using gem5 with the classic memory system for a while for full system X86 simulations integrating a number of different more modern PCIe device models. Now we are trying to get things to work with Ruby as well for multi-core X86 simulations. Currently we are tracking down some driver-related bugs that only show up with Ruby, and led us to the more fundamental question of how to model PCIe semantics, in particular with regards to ordering, correctly. My understanding is that PCIe guarantees strong ordering of operations (at least within a traffic class). And modren devices do take advantage of this: e.g. with descriptor queues where a posted write for the data buffer is immediately followed by a posted write for the descriptor, and then an MSI. Now currently our bridge for our device simulations in gem5 is implemented as a PCI device and just calls dmaRead/dmaWrite operations asynchronously without sequencing them synchronously. This at least seems to work for the classic memory system, but with ruby (MESI_Two_Level atm) we are seeing problems (still need to dig a bit deeper to confirm that this is 100% due to ordering). Is there a proper specification of what the model is for these memory operations? If the current dma ordering is too weak for PCIe semantics (looking at device models like the e1000, I see that DMAs are manually ordered by the device), any suggestions for how to go about ordering operations correctly without fully sequentializing everything? Thanks in advance, Antoine -- Antoine Kaufmann signature.asc Description: PGP signature ___ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
