Hi Mike,
The I/O devices in gem5 are cache coherent. So if you leave them
attached to the iobus, their reads/writes will snoop the caches. If you
want it closer you can move it up the hierarchy, but yes, you'll need a
cache to participate in the coherence protocol for you. By default if
your device is below the L2 cache it will not allocate data in any of
the higher level caches (it's possible for a dirty block in the IO cache
to migrate to another cacehe if a read happens before it's written
back), but by default data will end up in the I/O cache and be written
back to memory. If you want some other behavior you'll need to hack the
cache models or attach the nic like I showed below where data would
allocate into the L2.
Ali
On Tue, 30 Aug 2011 08:55:56 -0600, Michael Levenhagen
<[email protected]> wrote:
This does help. It looks like my configuration is invalid. I'm trying
to model a cache coherent NIC so I attached a DMADevice directly to
the Bus that the CPU caches and PhysicalMemory are connected to.
If I understand I need to place a Cache between the NICs DMADevice
and Bus.
Mike
On Aug 29, 2011, at 6:21 PM, Ali Saidi wrote:
Hi Michael,
The short answer is that it should work. M5 (gem5) was created out
of a desire to do multi-system simulation of TCP/IP and one of the
experiments we did early on was cache placement of DMA data from the
NIC. By default DMADevice::dmaRead() and DMADevice::dmaWrite() should
issue standard cacheable requests. The read/write requests that the
device is doing are no different than the read/write requests that a
CPU is doing. All DMA devices do this in a sense as the IOCache is
just a small cache that makes DMA operations coherent with the rest of
memory (otherwise the devices wouldn't work because they couldn't read
dirty data (e.g a descriptor ring) out of the caches). There are
several possibilities. (1) There might be a bug in the code you're
using, a year is a pretty long time with our code base. (2) You might
have created some topology that isn't supported. (3) If your cache
models don't support the is_top_level flag there could definitely be
as issue with the cache trying
to
hand ownership to the device (which it shouldn't) in the case of
full-block reads to dirty data. A topology like the following should
work with the current code base:
CPU NIC
| | |
I D IOC2/Bridge2
| | |
-------------------------
|
L2
|
mem ---------------
|
IOC1/Bridge1--------------
| |
Other Devices...
You can see the bridge/cache pairs in our configuration files. The
ports are setup to only allow accesses in one direction. The I/O cache
participates in the coherences protocol and converts the read/write
transaction into the appropriate coherency operations and the bridge
provides memory mapped I/O access to the peripheral.
Hope that helps,
Ali
On Aug 29, 2011, at 4:01 PM, Michael Levenhagen wrote:
Do the cache models in M5 support Cache Injection?
I've created a simulation where I have a memory mapped NIC that
uses a DmaDevice to read and write physical memory (note the memory
is not restricted to UNCACHEABLE). I've followed WriteReq packets
from the DmaDevice to the Cache and it appears that the data in these
packets is not injected into the cache even if the cache block is
present. I've hacked handleSnoop() so it injects write request into
the cache if the cache block is present. This hack has allowed me to
run MPI apps such as osu_bw and osu_latency but I'm running into a
problem, with another app, that looks like a memory consistency
issue. I'd consider my use of the DmaDevice to be pretty standard so
I'm stumped why I'm having a problem unless using it to access
CACHEABLE memory is not allowed.
Note that I'm using a snapshot of M5 from last fall.
Michael Levenhagen
Sandia National Labs.
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
