Hi Eduardo, I remember having similar issue with MESI_THREE_LEVEL protocol before. I tried to fix it by delaying all messages (not just the header) to avoid violating the FIFO semantics, but I think I got deadlock problems as the L0 may expect some other messages that are also delayed (I can't remember the exact case). Then I just set ordered to false and it works for me : )
Regarding to your questions. I am also not sure if FIFO ordering should be forced. It would be great if someone more experienced could help clarify it. *王 钲 荣* Zhengrong Wang Computer Science Department University of California, Los Angeles California, USA 90024 Work Email: [email protected] Mobile :+1 310-447-4568 Eduardo José Gómez Hernández via gem5-dev <[email protected]> 于2020年8月20日周四 上午7:18写道： > Hi everybody, > > I was upgrading to gem5 version 20, and running some x86 applications > with the MESI_Three_Level protocol I found something weird. > > I don't know if this is a bug or not, so I ended thinking that the > mail list will be the best place to comment on this. > > I was executing some Splash-3 benchmarks on the FullSystem mode of > gem5 running Linux kernel 5.4.48 and Ubuntu 20.04.1. When using the O3 > CPU and the ruby MESI_Three_Level protocol, it ended outputting the > FIFO ordering panic from MessageBuffer for bufferFromL1. > > MessageBuffer::enqueue > > if (m_strict_fifo) { > if (arrival_time < m_last_arrival_time) { > panic("FIFO ordering violated: %s name: %s current time: %d " > "delta: %d arrival_time: %d last arrival_time: %d\n", > *this, name(), current_time, delta, arrival_time, > m_last_arrival_time); > } > } > > Then I checked the config for MESI_Three_Level, and the queue has FIFO > ordering enforced: > > l0_cntrl.bufferFromL1 = MessageBuffer(ordered = True) > > I was thinking about what could be happening. Then I remember x86 > atomics, they execute delayHead (lately gdb confirmed this). > > void > delayHead(Tick current_time, Tick delta) > { > MsgPtr m = m_prio_heap.front(); > std::pop_heap(m_prio_heap.begin(), m_prio_heap.end(), > std::greater<MsgPtr>()); > m_prio_heap.pop_back(); > enqueue(m, current_time, delta); > } > > But that call to delayHead arrive from L0Controller_wakeup, so, > checking the protocol, L0 has to handle the block_on functionality > from L1 cache messages: > > > // Messages for this L0 cache from the L1 cache > in_port(messgeBuffer_in, CoherenceMsg, bufferFromL1, rank = 1) { > if (messgeBuffer_in.isReady(clockEdge())) { > peek(messgeBuffer_in, CoherenceMsg, block_on="addr") { > > And that block generates the delayHead call: > > if "block_on" in self.pairs: > address_field = self.pairs['block_on'] > code(''' > if (m_is_blocking && > (m_block_map.count(in_msg_ptr->m_$address_field) == 1) && > (m_block_map[in_msg_ptr->m_$address_field] != &$qcode)) { > $qcode.delayHead(clockEdge(), cyclesToTicks(Cycles(1))); > continue; > } > ''') > > The only set of instructions that can trigger this are x86 atomic > instructions using ldstl and stul that blocks and unblocks cache > addresses in the m_block_map using AbstractController::blockOnQueue > and AbstractController::unblock. > > With all set, here are my questions: > > Does it make sense to force FIFO ordering in this queue where messages > can be delayed? > When delaying the head because of atomic instructions. Should all the > messages be delayed and not only the head? > > > Thanks a lot for your attention. > > Best Regards, > Eduardo > > -- > Eduardo José Gómez Hernández > Department of Computer Engineering and Technology > Computer Science Faculty > University of Murcia, Spain > [email protected] > _______________________________________________ > gem5-dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
_______________________________________________ gem5-dev mailing list -- [email protected] To unsubscribe send an email to [email protected] %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
