Hi All, Looking at the details of the provided inclusive MOESI protocol found in components/FastCache/InclusiveMOESI.hpp, I think I have found a couple of minor bugs and there's some possibly strange behaviour with downgrade and return snoop requests.
Others may use it as a baseline for experiments, so I'd like to ask the list subscriber's opinions about my observations below. I'm assuming a general case where a core has 2 (or more) levels of dedicated caches and I am considering the core is at the "top". 1. Functionality There is no StoreAccess (store from the core) in the action declarations. This means the model is suitable only for L2 and lower caches. Moreover, the first action, reading a Modified block, would not work for an L1 as the Dirty bit is lost to the cache above, which isn’t there, so we won't know that the block is dirty anymore. Am I right about this? 2. Bugs in handling NonAllocatingWrites DECLARE_REQUEST_ACTION( kModified, kNAWAccess, SendNone, NoAllocate, E_State, UpdateLRU, FillDirty, HitNAWModified ); DECLARE_REQUEST_ACTION( kOwned, kNAWAccess, SendUpgrade, NoAllocate, E_State, UpdateLRU, FillWritable, MissNAWOwned ); DECLARE_REQUEST_ACTION( kShared, kNAWAccess, SendUpgrade, NoAllocate, E_State, UpdateLRU, FillWritable, MissNAWShared ); This is a non-allocating access, so the next state should stay/change to Modified as the block remains at this cache level. Also the responses should be NAW-acks not Fills, though this shouldn't affect correctness. 3. Interesting/strange behaviour. DECLARE_SNOOP_ACTION( kOwned, MM::ReturnReq, MM::ReturnReplyDirty, NoSnoop, S_State, HitReturnReqOwned, false, false ); DECLARE_SNOOP_ACTION( kOwned, MM::Invalidate, MM::InvalidateAck, DoSnoop, I_State, HitInvalidateOwned, false, true ); Interesting difference for the response to snoop commands for a block in Owned state. Invalidate, responds with just an Ack and ReturnReq responds with Dirty. The Dirty response seems strange; there’s another cache with a copy in the system. There’s also an issue with forwarding the snoop up or not. See next comment. DECLARE_SNOOP_ACTION( kModified, MM::Downgrade, MM::DownUpdateAck, DoSnoop, O_State, HitDowngradeModified, ......... ); DECLARE_SNOOP_ACTION( kExclusive, MM::Downgrade, SnoopDepend, DoSnoop, DependState, HitDowngradeExclusive, ....... ); Two issues: - For Modified, why pass Downgrade snoop message up? Any copies above a Modified block can be read-only. There’s nothing to downgrade and, actually, looking at the action for a Shared (i.e. read-only) block for a downgrade request, it gives poison! - Exclusive seems ok but consider this case: A block is Modified in L1 and Exclusive in L2. When a Downgrade request is received in L2, it is forwarded to L1. L1 changes state to Owned and responds with DownUpdateAck. L2 also changes to Owned and passes on the same response. So we are left with 2 levels having Owned as their state. That is OK in general, except for handling ReturnReq (see previous comment) where snooping is not forwarded up. DECLARE_SNOOP_ACTION( kExclusive, MM::ReturnReq, MM::ReturnReply, DoSnoop, DependState,HitReturnReqExclusive, ........ ); Two issues: - Dirty status is not passed down when above this Exclusive block there is a Modified one. The point here is that if we have a single cache with a modified block, we get the Dirty status in snoop response, but if we have a hierarchy of caches with the top one holding a modified block (and the others exclusive), we don’t get the dirty status. Does this matter? - In the method code for this action, the snoop message sent up the hierarchy is Downgrade. Depending on the snoop reply the next state could be Owned (reply is UpdateAck) or Shared (reply is Ack). There is a subtle difference of this and the plain Downgrade snoop request: The top (originally) modified cache changes to Owned for downgrade as do all the ones below it which were exclusive. For a return request, the top changes to Shared and the ones below to Owned. Does it matter? Thanks, Aris PS: Since this is my first post, I'm also sending regards all to the people I met at EPFL during my visit there in November 2009. I hope you are well. -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
