Myles Watson wrote: > >> -----Original Message----- >> From: [EMAIL PROTECTED] [mailto:linuxbios- >> [EMAIL PROTECTED] On Behalf Of Marc Jones >> Sent: Thursday, June 14, 2007 2:39 PM >> To: Roman Kononov >> Cc: [EMAIL PROTECTED]; [email protected] >> Subject: Re: [LinuxBIOS] Opteron caching of device memory >> >> Roman Kononov wrote: >> >>> On 06/13/2007 09:39 AM, Myles Watson wrote: >>> >>> >>>> I'm using LinuxBIOS on my Tyan s2892. I have a device that maps a lot >>>> of the memory space, but I'm struggling trying to get the Opteron to >>>> read and write to my device in larger blocks. I have set the variable >>>> MTRRs in the device driver to writeback (witnessed by /proc/mtrr), but >>>> >> I >> >>>> still get 64-bit accesses instead of 64-byte (cache line). >>>> >>>> >>> Some time ago I officially asked an AMD's rep about this. Someone from >>> >> AMD >> >>> unofficially said that MemIO can be only Write-Combined. And I gave up >>> >> with >> >>> caching. >>> >>> Below is what I asked. Maybe you can spot what I missed. Please let me >>> >> know >> >>> if you succeed. >>> >>> Roman >>> -------------------------------------------------------------------- >>> I have a difficulty related to cacheability of Memory-Mapped I/O memory >>> >> in >> >>> Opteron. Can you please assist me? >>> >>> I have an Opteron-based system, where the Opteron and an FPGA are >>> >> connected >> >>> using a HyperTransport link. As a target of HT traffic, the FPGA >>> >> responses >> >>> to certain range of physical addresses. I am trying to configure the CPU >>> >> and >> >>> the North Bridge to treat the FPGA as Writeback memory. I cannot do >>> >> this. >> >>> I've been able to make the FPGA Uncacheable, Write-Combining, Write- >>> >> Protect >> >>> and Writethrough. When I try to make it Writeback it behaves as if it >>> >> were >> >>> Writethrough. >>> >>> Is it possible to make the FPGA Writeback? If yes, what should I change? >>> >>> I have observability of the HT traffic from the FPGA point of view, so I >>> know what is written and what is requested by the CPU. >>> >>> Here is how I program the Opteron: >>> >>> The processor is running in the Long Mode with paging enabled. >>> >>> A pair of NB PCI Function 1, Memory-Mapped I/O Base and Limit registers >>> >> is >> >>> programmed with the FPGA physical address range, proper link and node >>> >> IDs, >> >>> posted. >>> >>> A pair of the variable MTRR Base and Mask (MSR 0x200-0x20f) registers is >>> programmed with the FPGA physical address range, valid bit and the >>> >> desired >> >>> caching method. Another pair of MTRR registers describes the DRAM, all >>> others are disabled. >>> >>> The PAT register (MSR 0x277) is 0x0606060606060606ull. >>> >>> The Page Table Entry for the virtual address mapped to the FPGA has PAT, >>> >> PCD >> >>> and PWT bits cleared. >>> >>> Bit CD in the CR0 register is cleared. >>> >>> >>> >>> >> I am not an expert but these are my thoughts.... >> >> Now that I think about it, it makes sense that you can't set writeback >> to noncoherent (nonsystem) memory space. What if another device wants to >> write to that memory. There is no way for the cache to snoop that it was >> written. You would need a coherent HT link to to your FPGA to get the >> cache snoop messaging. >> >> Marc >> > > I think that the FPGA should be able to set the coherent bit and cause > snooping, even if it is not on the coherent bus, because it targets coherent > memory. > > An example is DMA: > 1. The processor writes data to DRAM (some of it may/will still be cached) > 2. The processor writes to the DMA controller (say a SATA controller) > 3. The DMA transfers the data to the disk drive > > If there's no snooping, the DMA controller reads stale data from the DRAM. > > Feel free to point out shortcomings in my example. I need to learn more > about it. > > Myles > > > I am sorry, I don't have the expertise to answer your question. I recommend that you try the Hyper Transport Consortium and if you still can't get an answer let me know. I will try to find the AMD Torrenza guys to get us an answer.
http://www.hypertransport.org/ http://enterprise.amd.com/us-en/AMD-Business/Technology-Home/Torrenza.aspx Marc -- Marc Jones Senior Software Engineer (970) 226-9684 Office mailto:[EMAIL PROTECTED] http://www.amd.com/embeddedprocessors -- linuxbios mailing list [email protected] http://www.linuxbios.org/mailman/listinfo/linuxbios
