Hi,
I have had to put this problem on the back burner for a little bit while I
finish up something else, but revisiting it, I think I have better solution
in mind than what I suggested earlier.
As previously stated, the RubySystem::functionalRead() member checks the
access permissions for all the cache controllers in the system, and then
for controllers that have the block in either READ_ONLY/READ_WRITE state,
it retrieves a copy of the data.
Now in the specific case of Writebacks (such as in the one I am
encountered, as well as the one encountered in the previous link), the
cache block itself is deallocated during Replacement Event, but the data is
copied into the TBE (MSHR-like) data structure as it awaits WB completion.
Now even though the access permissions are switched to BUSY state, the
getDataBlock() function call invoked from RubySystem::functionalRead()
would actually return the correct data from the TBE: (getDataBlock()
function defined in the MOESI-L1 protocol.
DataBlock getDataBlock(Address addr), return_by_ref="yes" {
Entry cache_entry := getCacheEntry(addr);
if(is_valid(cache_entry)) {
return cache_entry.DataBlk;
}
TBE tbe := TBEs[addr];
if(is_valid(tbe)) {
return tbe.DataBlk;
}
error("Data block missing!");
}
My suggestion/solution would be to further divide the BUSY AccessPermission
State into BUSY + BUSY_WB, such that Blocks in BUSY_WB state would be
allowed to provide data for functional accesses only. Other cases when a
block is in a BUSY state are when its waiting to receive a valid copy of
the data from another cache, so simply allowing accessing to BUSY blocks in
all cases won't solve this problem, but sub-dividing BUSY into BUSY/BUSY_WB
would solve these race conditions between Timing/Functional Accesses.
The tricky thing with this is that it seems adding/changing access
permissions has/could have effect on other things in the simulator, which I
am not sure would cause but is indicated in this comment in
RubySlicc_Exports.sm:
// These permissions have multiple uses. They coordinate locking and
// synchronization primitives, as well as enable functional accesses.
// One should not need to add any additional permission values and it is
very
// risky to do so.
Could anyone comment on this suggestion, either for or against, or things
that would definitely need to modified (e.g. what code sections
specifically wrt locking primitives) to support this solution? Doing a grep
through the ruby code, I only see one check for BUSY Access Permission
check, and that is in the same functionalRead() function.
Malek
On Thu, Aug 1, 2013 at 12:52 PM, Malek Musleh <[email protected]>wrote:
> Hi Andreas,
>
> It was a poor choice of wording. It's more like A functional access is
> scheduled after the block was moved to MI State but before the block could
> move to a final stable state. THE DPRINTF statements show that there are
> multiple distinct functional accesses to the same address. If only the
> functional access was scheduled before / after the block finished moving to
> a stable state.
>
> Is it possible to reschedule functional accesses (of course its "possible"
> but does it make sense to), if there is only one valid copy of the block
> and its in a busy state? I don't know if that would make sense in the
> classic memory model, but for Ruby, when the level of detail of the
> coherence protocol is much higher, and the modeling of new
> states/permissions.
>
>
> On Thu, Aug 1, 2013 at 12:09 PM, Andreas Hansson
> <[email protected]>wrote:
>
>> Hi Malek,
>>
>> I find this statement puzzling: "before the functional access
>> completes". A functional access is purely a chain of function calls that
>> complete in 0 time. How can anything happen before it completes?
>>
>> Thanks,
>>
>> Andreas
>>
>> From: Malek Musleh <[email protected]>
>> Reply-To: gem5 users mailing list <[email protected]>
>> Date: Thursday, 1 August 2013 16:38
>> To: gem5 users mailing list <[email protected]>
>> Subject: [gem5-users] Functional Accesses Failing in Ruby
>>
>> Hi,
>>
>> I am running into a problem with one of my simulations where a
>> functional read accss is failing. It is failing in this case because as the
>> RubySystem is handling the functional accesses, it checks all the
>> controllers in the system to determine which controllers have the block
>> with the necesary permissions (RW/RO) and which have unavailable (I/BUSY).
>>
>> For this specific case, the block is only valid in one cache, and is at
>> the tme in 'M' state which has RW permission. Then, a L1_Replacement event
>> is triggered in between the LOAD on M, and before the functional access
>> completes, such that when the ruby system checks, it only finds one copy of
>> the block, that copy being in MI (BUSY) state as it is being written back.
>>
>> Searching the list, it looks like this is a general (although rare)
>> problem that can occur due to functional accesses coexisting with timing
>> accesses:
>>
>> http://www.mail-archive.com/[email protected]/msg02126.html
>>
>> In solving that problem, the access permission of the WB state was
>> changed to allow read permissions, but I am hesitant about doing that for
>> the MI state in the MOESI_CMP_directory protocol, as it might have other
>> negative effects.
>>
>> I am not using the latest latest dev repo (about a couple of months
>> old), and I have made changes to the code, not any of which affects this
>> behavior, it just exposed the problem.
>>
>> I have noticed a few others on the list have experienced similar
>> problems, but no solution/response was ever posted.
>>
>> Any body have a better/cleaner idea than to change the access
>> permissions of the MI State?
>>
>> Below is a snippet of the protocol trace/RubySystem flag:
>>
>> 2499502566500 0 L1Cache Load M>M
>> [0x7510000, line 0x7510000]
>> 2499527076000 0 L1Cache L1_Replacement M>MI
>> [0x7510000, line 0x7510000]
>> 2499527078000 0 L2Cache L1_PUTX ILX>ILXW
>> [0x7510000, line 0x7510000]
>> 2499527082000 0 L1Cache Writeback_Ack_Data MI>I
>> [0x7510000, line 0x7510000]
>> 2499527084000 0 L2Cache L1_WBDIRTYDATA ILXW>M
>> [0x7510000, line 0x7510000]
>> 2499530130500 0 Seq Begin >
>> [0x7510000, line 0x7510000] LD
>> 2499530132000 0 L1Cache Load I>IS
>> [0x7510000, line 0x7510000]
>> 2499530134000 0 L2Cache L1_GETS M>OO
>> [0x7510000, line 0x7510000]
>> 2499530136000 0 Seq Done >
>> [0x7510000, line 0x7510000] 11 cycles
>> 2499530136000 0 L1Cache Exclusive_Data IS>M_W
>> [0x7510000, line 0x7510000]
>> 2499530138000 0 L2Cache Exclusive_Unblock OO>ILX
>> [0x7510000, line 0x7510000]
>> 2499530161000 0 L1Cache Use_Timeout M_W>M
>> [0x7510000, line 0x7510000]
>> 2499531935500 0 Seq Begin >
>> [0x7510000, line 0x7510000] LD
>> 2499531937000 0 Seq Done >
>> [0x7510000, line 0x7510000] 3 cycles
>> 2499531937000 0 L1Cache Load M>M
>> [0x7510000, line 0x7510000]
>> 2499663708000: system.ruby: Functional Read request for [0x7510000, line
>> 0x7510000]
>> 2499663888000: system.ruby: Functional Read request for [0x7510000, line
>> 0x7510000]
>> 2499664653000 0 L1Cache L1_Replacement M>MI
>> [0x7510000, line 0x7510000]
>> 2499664661500: system.ruby: Functional Read request for [0x7510000, line
>> 0x7510000]
>> fatal: Ruby functional read failed for address 0x7510000
>>
>>
>> Malek
>>
>> -- IMPORTANT NOTICE: The contents of this email and any attachments are
>> confidential and may also be privileged. If you are not the intended
>> recipient, please notify the sender immediately and do not disclose the
>> contents to any other person, use it for any purpose, or store or copy the
>> information in any medium. Thank you.
>>
>> _______________________________________________
>> gem5-users mailing list
>> [email protected]
>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
>>
>
>
_______________________________________________
gem5-users mailing list
[email protected]
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
