Hi Tim, I agree that memory controller stats are quite confusing. The problem is that there has been significant Ruby memory system refactoring over the last ~18 mos., and the stats have not been cleaned up in a clear way. Here are some notes that may be helpful - though note they too are subject to change in future revisions.
There are a few factors that make memory stats confusing: (1) there are separate stats for the actual simulated memory controller components (buffers, arbiters, etc.) vs. the underlying memories they access (i.e. the data itself, but not actual components in the simulated system), (2) some Ruby protocols do functional accesses to memories to achieve data correctness, which mucks with the stats, and (3) depending on which revision of gem5 you're using, the default simulated memory controllers might be different. First, memories in gem5 report 2 separate sets of stats. The first set are stats that report the simulated hardware activity for the memory controller itself, while the other set of stats reports accesses to the underlying data in the memories. In recent versions of gem5 (since roughly 10524:fff17530cef6?), these sets of stats are both reported with the same prefix as the component name (something like "system.mem_ctrls0"). The stats for the underlying memory include bytes_read, bytes_written, num_reads, num_writes, num_line_reads, num_line_writes, num_line_others, bw_read, bw_write, bw_total (you can find specifics on how these are collected in src/mem/abstract_mem.cc). Note that these are stats about the actual data accesses rather than the activity of the memory controllers themselves. Ruby can also make these underlying memory access stats more confusing... As a simplification, some Ruby protocols require the use of a "backing store" underlying memory, which is used to hold the current version of cached data (rather than making sure the correct data is held in the Ruby caches). In order to achieve data correctness, the Ruby Sequencers accesses the data from the backing store at the end of a memory access, which contributes to the underlying memory access stats I noted in the last paragraph. In cases like this, these underlying access stats become nearly meaningless, because they don't just represent actual access to the off-chip memory store, but rather they include all cache accesses also. The protocols that require this backing store are the GPU protocols, including those from AMD that have been recently added to gem5, and those in gem5-gpu (using the backing store simplifies modeling of atomic memory accesses). The standard CPU protocols in gem5 do NOT use the backing store (e.g. MI_example, MESI_Two_Level, MESI_Three_Level, MOESI_hammer, etc.). Now that we have info about the underlying memory access out of the way, you're probably wondering how you can find the simulated memory controller activity (e.g. how many read requests actually go off-chip through the memory controller?). Depending on which memory controller you are modeling, there may be different statistics reported for the actual activity of the simulated controllers. If you are using a gem5 revision before 10524:fff17530cef6 (Nov. 2014), there was only one memory controller that worked with Ruby, the RubyMemoryControl. It reports the actual count of off-chip memory accesses, both reads (memRead) and writes (memWrite), as well as other stats about the performance of memory accesses through the controller (e.g. stallsPerReq). For more specifics on the stats reported by the RubyMemoryControl, check out the regStats() function in src/mem/ruby/profiler/MemCntrlProfiler.cc. Since gem5 revision 10524:fff17530cef6, Ruby has been able to use any of the DRAMCtrl-derived memory controllers (e.g. DDR3_1600_x64, DDR4_2400_x64, GDDR5_4000_x64). Since these controllers are modeled with C++ code different from the RubyMemoryControl, they report different but similar stats. Again, this includes number of reads (readReqs) and writes (writeReqs), and other performance stats like stall counts. For a clearer picture on which stats these controllers report, check out the regStats() function in src/mem/dram_ctrl.cc. Hope this helps, Joel On Wed, Feb 24, 2016 at 3:52 PM, Timothy Chong <[email protected]> wrote: > I also came across this post: > https://groups.google.com/forum/#!msg/gem5-gpu-dev/zBUXxv65H_k/n7Q5pGtJ4VUJ > > which says that "system.mem_ctrls, bytes_read, bytes_written, num_reads, > num_writes, etc.. These actually measure the total memory accesses from > each of the different requesters ». And they claimed that > system.ruby.dir_cntrl0.memBuffer is the number to look at. I’m quite > confused... > > Cheers, > Tim > > Le Feb 24, 2016 à 12:12 PM, Timothy Chong <[email protected]> a écrit : > > Hi Andreas, > > Thank you for the response. I believe ruby has it’s own memory hierarchy, > which overrides the original memory controller. I quickly looked at > bytesReadDRAM, but it has a value of 0, which I assume is because the > traffic’s directed somewhere else. But I’m still confused by the difference > between mem_ctrls.num_read and system.ruby_cntrl0.memBuffer.memReads. The > latter is the numberReads that go into the directory and then to the memory > controller (as I can tell after I dug deep into the code). I don’t really > know what mem_ctrls.num_read includes. All memory traffic to memory > controller? > > Would mem_ctrls.num_reads * cache line size be the same as the > bytesReadDRAM, since shouldn’t all the « mem_ctrls.num_reads » be going > into the memory controller? > > Thanks, > Tim > > Le Feb 24, 2016 à 3:04 AM, Andreas Hansson <[email protected]> a > écrit : > > Hi Timothy, > > I do not use Ruby myself, so I do not dare say what the memBuffer stats > represent. The num_read and num_write stats for the memory controller are > just that, how many read requests and how many write requests have > accessed the memory. They are in fact from AbstractMemory. If you run with > the proper DRAMCtrl I would instead suggest to look at the actual DRAM > accesses (bytesReadDRAM and similar). > > Also remember that there may be many channels of DRAM, so you have to look > at the stats across all the mem_ctrl instances. > > Andreas > > On 24/02/2016, 04:21, "gem5-users on behalf of Timothy Chong" > <[email protected] on behalf of [email protected]> wrote: > > Hello there, > > I’ve been running FS with ruby. I would like to see the statistics of > memory access to the memory controller. > > I was a little confused by the different mem read numbers. > > Specifically, there’s the mem_ctrls.num_read, which is broken down to > inst, data, dtb.walker, and itbwalker. It says mem_ctrls, so I sort of > assumed that it is for the memory controller access. > > However, when I looked more into the statistics, I also found > system.ruby_cntrl0.memBuffer.memReads, which are MUCH lower than the > numbers for mem_ctrls.num_read. > > Which one is the read/write access into he memory controller? And what > exactly is mem_ctrls.num_read and the broken down statistics? > > Thank you, > Timothy Chong > _______________________________________________ > gem5-users mailing list > [email protected] > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users > > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or store or copy the > information in any medium. Thank you. > _______________________________________________ > gem5-users mailing list > [email protected] > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users > > > > > _______________________________________________ > gem5-users mailing list > [email protected] > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users > -- Joel Hestness PhD Candidate, Computer Architecture Dept. of Computer Science, University of Wisconsin - Madison http://pages.cs.wisc.edu/~hestness/
_______________________________________________ gem5-users mailing list [email protected] http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
