I can see that there is a read memory barrier in the ixgbe_clean_rx_irq()
functions with the comment "This memory barrier is needed to keep us from
reading any other fields out of the rx_desc until we know the RXD_STAT_DD
bit is set" which is exactly my problem, but that should emit an lfence
which if I remember correctly did not happen to fix the problem for me.
I'll take a look when I get home. I can make a simple test to try to
determine if the fence is actually fixing the problem, or if it is just
adding a delay which makes the problem seem to disappear. Since I have a
very lightweight and lockless RX implementation, if it is a hardware bug it
is unlikely to show up in a 'proper' driver given they just happen to do
enough logic between the DD check and reading packet contents.
My code looks as such:
.poll:
-B
On Wed, Apr 22, 2015 at 5:04 PM, Rose, Gregory V <gregory.v.r...@intel.com>
wrote:
>
>
> > -----Original Message-----
> > From: Brandon Falk [mailto:bf...@gamozolabs.com]
> > Sent: Wednesday, April 22, 2015 1:10 PM
> > To: e1000-devel@lists.sourceforge.net
> > Subject: [E1000-devel] [non-linux] Cache Coherency Problems on X540?
> >
> > I'm writing a custom driver for a custom OS and thus this does not relate
> > to the Linux driver.
> >
> > I have implemented a driver which polls for the DD and EOP flags to get
> > set (using legacy receive descriptors) and I have noticed a problem where
> > these flags both can get set very slightly before the packet contents
> have
> > actually been DMAed to the host (or at least appear to be).
> >
> > I'm under the understanding that the X540 device should respect caching
> > and thus I should not have to be flushing caches for the packet contents.
> > The only thing I could think of would be that I need to add some memory
> > barriers as maybe the CPU is speculatively loading the packet contents
> > before it is reading the flags, and thus the contents are older than the
> > flags?
>
> The Intel Linux driver for the X540 does implement memory barriers and I'm
> pretty sure it's to prevent this sort of problem. You may want to examine
> that code (understanding at the same time that GPL will prevent you from
> using it directly).
>
> - Greg
>
> ----------
> Greg Rose
> FreeBSD/NFV PAE
> Network Division
> Intel Corporation
> Desk - 503-712-5048
>
> Any man who afflicts the human race with ideas must be prepared to see
> them misunderstood.
>
> - H. L. Mencken
>
>
>
> >
> > I just want to verify that this isn't a hardware problem with the X540
> and
> > that I will need to do a workaround on my end. Ideally the workaround
> > actually fixes the problem, rather than a small delay/sleep that makes it
> > seem to go away in most cases.
> >
> > Anyone have any ideas? Adding an clflush, mfence, sfence fixes the
> > problem, but an lfence does not. I'm under the suspicion that it's not
> the
> > fences that actually are doing anything, rather the delay of these rather
> > long instructions happens to just be a long enough sleep. I just want to
> > make sure that I'm fixing the problem correctly and not just hoping the
> > sleep delays long enough for the DMA to occur, as that is quite kludgey.
> >
> > It is a 4 physical processor system, AMD Fam 15h.
> >
> > -B
>
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit
http://communities.intel.com/community/wired
