Hi everybody,
I was upgrading to gem5 version 20, and running some x86 applications
with the MESI_Three_Level protocol I found something weird.
I don't know if this is a bug or not, so I ended thinking that the
mail list will be the best place to comment on this.
I was executing some Splash-3 benchmarks on the FullSystem mode of
gem5 running Linux kernel 5.4.48 and Ubuntu 20.04.1. When using the O3
CPU and the ruby MESI_Three_Level protocol, it ended outputting the
FIFO ordering panic from MessageBuffer for bufferFromL1.
MessageBuffer::enqueue
if (m_strict_fifo) {
if (arrival_time < m_last_arrival_time) {
panic("FIFO ordering violated: %s name: %s current time: %d "
"delta: %d arrival_time: %d last arrival_time: %d\n",
*this, name(), current_time, delta, arrival_time,
m_last_arrival_time);
}
}
Then I checked the config for MESI_Three_Level, and the queue has FIFO
ordering enforced:
l0_cntrl.bufferFromL1 = MessageBuffer(ordered = True)
I was thinking about what could be happening. Then I remember x86
atomics, they execute delayHead (lately gdb confirmed this).
void
delayHead(Tick current_time, Tick delta)
{
MsgPtr m = m_prio_heap.front();
std::pop_heap(m_prio_heap.begin(), m_prio_heap.end(),
std::greater<MsgPtr>());
m_prio_heap.pop_back();
enqueue(m, current_time, delta);
}
But that call to delayHead arrive from L0Controller_wakeup, so,
checking the protocol, L0 has to handle the block_on functionality
from L1 cache messages:
// Messages for this L0 cache from the L1 cache
in_port(messgeBuffer_in, CoherenceMsg, bufferFromL1, rank = 1) {
if (messgeBuffer_in.isReady(clockEdge())) {
peek(messgeBuffer_in, CoherenceMsg, block_on="addr") {
And that block generates the delayHead call:
if "block_on" in self.pairs:
address_field = self.pairs['block_on']
code('''
if (m_is_blocking &&
(m_block_map.count(in_msg_ptr->m_$address_field) == 1) &&
(m_block_map[in_msg_ptr->m_$address_field] != &$qcode)) {
$qcode.delayHead(clockEdge(), cyclesToTicks(Cycles(1)));
continue;
}
''')
The only set of instructions that can trigger this are x86 atomic
instructions using ldstl and stul that blocks and unblocks cache
addresses in the m_block_map using AbstractController::blockOnQueue
and AbstractController::unblock.
With all set, here are my questions:
Does it make sense to force FIFO ordering in this queue where messages
can be delayed?
When delaying the head because of atomic instructions. Should all the
messages be delayed and not only the head?
Thanks a lot for your attention.
Best Regards,
Eduardo
--
Eduardo José Gómez Hernández
Department of Computer Engineering and Technology
Computer Science Faculty
University of Murcia, Spain
eduardojose.go...@um.es
_______________________________________________
gem5-dev mailing list -- gem5-dev@gem5.org
To unsubscribe send an email to gem5-dev-le...@gem5.org
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
