Re: [casper] DRAM (DDR3) ROACH-2
Hi David, Sounds very interesting. Thanks for the info. No one is using a soft core cpu as there is already the PPC on the ROACH boards. The DDR and QDR have interfaces to the PPC over the epb/opb bus. This runs at 66MHz and is not idea to use as part of your data pipeline as it is generally to slow. The best way is to packetize the data and sent it over either 10Gig or 1Gig ethernet to a server for storage or further software/GPU processing. Hope this helps. Regards Wesley Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 7:35 PM, David Saroff dsar...@nrao.edu wrote: Wesley, Yes, I am interested in both the QDRSRAM and the DDR3RAM on the roach2. My current project is an FX correlator for 38 small antennae at the prime focus of the Green Bank Telescope, for 21cm HI work. With limited bandwidth MHz an FX correlator -might- fit in one v6. I need a lot of memory for accumulators, and if the v6 can't keep up with the data flow, for buffering out the results at partial duty cycle. Are people instantiating the soft microprocessor to get at those memories, or is there some state machine implementation that could be wired into the data flow more directly? David Comments below. On Wed, Jul 24, 2013 at 12:07 PM, Matthew Bridges matthewbridge...@gmail.com wrote: Hi JP and Wes, I am very interested in this work, but from a RHINO perspective. I am looking into how I should go about adding the RHINOs DRAM into my Architecture. I have a few questions, if you have time to answer. Did you use the Memory Interface Generator? And did it work for you? Yes, use the MIG. Both R1 and now R2 yellow blocks are based around MIG Are there other coregens used in your block? E.g FIFOs? Yes, the outputted data goes though an asynchronous fifo. How many ports did you implement? I was thinking of putting 2 into my Architecture, 1 for the Wishbone, 1 for direct processor access. What interface do you use to talk to the block? Is it OPB? Both the QDR and DDR use a single port which maps the memory address space to OPB bus so that the memory can be read from the PPC. Thanks, Matthew On Wed, Jul 24, 2013 at 11:48 AM, Wesley New wes...@ska.ac.za wrote: This is great work JP. Out of interest is anyone else planning to use the DDR3 on ROACH2? Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 8:24 AM, Juan-Pierre Jansen van Rensburg jvrensburg...@gmail.com wrote: Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg -- Software Defined Radio Group Department of Electrical Engineering University of Cape Town Private Bag X3, Rondebosch 7701 Tell: +27 (0)21 650 4659 Mob: +27 (0)84 413 2642
Re: [casper] DRAM (DDR3) ROACH-2
The mlib_devel library with the ddr3 support is available from github.com/juan-pierre/mlib_devel - fetch the ddr3_devel branch (not merged to master). I have successfully tested designs with the DDR3 clock rate set to 200 MHz (400 MHz DDR) and 250 MHz (500 MHz DDR) - just a word of caution, the MIG documentation mentions that if the memory is clocked above 480 MHz (DDR) then the IDELAYCTRL reference frequency must be set to 300 MHz and not 200 MHz - currently only 200 MHz is used. On Wed, Jul 24, 2013 at 2:53 PM, Primiani, Rurik rprimi...@cfa.harvard.eduwrote: Hi JP, We recently (last week) discovered we desperately need the DDR3 on ROACH2 since our work-around for getting the visibility data out of our system won't work in the long run (due to 10 GbE switch limiations... it's a long story). So just this week I re-started work on a DDR3 yellow block I had left behind a while ago. I am using the ddr3_controller and ddr3_clk modules from github.com/ska-sa/roach2_test_gateware which I believe is known to work. Originally I used the wishbone bus that was used for testing the board with a bridge but the only OPB-to-WB bridge I found online had the memory addresses hard-coded into a netlist. Instead I took the opb_dram_sniffer (which contains arbitration between CPU and FPGA) and the async_dram pcores from ROACH-1 and am in the process of interfacing them to the above modules. I have not yet determined how to translate the old MIG application interface to the new one but came up with some hack that seems to compile. I haven't tested anything in hardware yet! I'm very happy that it seems I won't have to since JP's done all the work :) Also I'm completely fine with abandoning the very little work I've done to help JP on the CPU-interface side. If the current yellow block isn't online somewhere presently could I get a copy to use in testing the DDR3 on our ROACH2's? Thanks and good work! Rurik On Wed, Jul 24, 2013 at 7:02 AM, Jonathan Weintroub jweintr...@cfa.harvard.edu wrote: Hi JP, Wes, We need the DDR3 rather urgently (time scale of 1 to 2 months), and Rurik Primiani has recently been working on it. Laura Vertatschitsch will be joining our group in September and we had her in mind to help with this too though recently concluded her start date isn't soon enough. Of course these efforts should be coordinated, and your note, JP, is certainly apropos and well timed for us. I'll leave it to Rurik to describe what he has been doing, he talked a bit about bus interfaces, so perhaps the PPC comes into it. Cheers, Jonathan On Jul 24, 2013, at 5:48 AM, Wesley New wes...@ska.ac.za wrote: This is great work JP. Out of interest is anyone else planning to use the DDR3 on ROACH2? Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 8:24 AM, Juan-Pierre Jansen van Rensburg jvrensburg...@gmail.com wrote: Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg
[casper] DRAM (DDR3) ROACH-2
Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg
Re: [casper] DRAM (DDR3) ROACH-2
This is great work JP. Out of interest is anyone else planning to use the DDR3 on ROACH2? Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 8:24 AM, Juan-Pierre Jansen van Rensburg jvrensburg...@gmail.com wrote: Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg
Re: [casper] DRAM (DDR3) ROACH-2
Hi JP and Wes, I am very interested in this work, but from a RHINO perspective. I am looking into how I should go about adding the RHINOs DRAM into my Architecture. I have a few questions, if you have time to answer. Did you use the Memory Interface Generator? And did it work for you? Are there other coregens used in your block? E.g FIFOs? How many ports did you implement? I was thinking of putting 2 into my Architecture, 1 for the Wishbone, 1 for direct processor access. What interface do you use to talk to the block? Is it OPB? Thanks, Matthew On Wed, Jul 24, 2013 at 11:48 AM, Wesley New wes...@ska.ac.za wrote: This is great work JP. Out of interest is anyone else planning to use the DDR3 on ROACH2? Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 8:24 AM, Juan-Pierre Jansen van Rensburg jvrensburg...@gmail.com wrote: Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg -- Software Defined Radio Group Department of Electrical Engineering University of Cape Town Private Bag X3, Rondebosch 7701 Tell: +27 (0)21 650 4659 Mob: +27 (0)84 413 2642
Re: [casper] DRAM (DDR3) ROACH-2
Comments below. On Wed, Jul 24, 2013 at 12:07 PM, Matthew Bridges matthewbridge...@gmail.com wrote: Hi JP and Wes, I am very interested in this work, but from a RHINO perspective. I am looking into how I should go about adding the RHINOs DRAM into my Architecture. I have a few questions, if you have time to answer. Did you use the Memory Interface Generator? And did it work for you? Yes, use the MIG. Both R1 and now R2 yellow blocks are based around MIG Are there other coregens used in your block? E.g FIFOs? Yes, the outputted data goes though an asynchronous fifo. How many ports did you implement? I was thinking of putting 2 into my Architecture, 1 for the Wishbone, 1 for direct processor access. What interface do you use to talk to the block? Is it OPB? Both the QDR and DDR use a single port which maps the memory address space to OPB bus so that the memory can be read from the PPC. Thanks, Matthew On Wed, Jul 24, 2013 at 11:48 AM, Wesley New wes...@ska.ac.za wrote: This is great work JP. Out of interest is anyone else planning to use the DDR3 on ROACH2? Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 8:24 AM, Juan-Pierre Jansen van Rensburg jvrensburg...@gmail.com wrote: Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg -- Software Defined Radio Group Department of Electrical Engineering University of Cape Town Private Bag X3, Rondebosch 7701 Tell: +27 (0)21 650 4659 Mob: +27 (0)84 413 2642
Re: [casper] DRAM (DDR3) ROACH-2
Hi JP, Wes, We need the DDR3 rather urgently (time scale of 1 to 2 months), and Rurik Primiani has recently been working on it. Laura Vertatschitsch will be joining our group in September and we had her in mind to help with this too though recently concluded her start date isn't soon enough. Of course these efforts should be coordinated, and your note, JP, is certainly apropos and well timed for us. I'll leave it to Rurik to describe what he has been doing, he talked a bit about bus interfaces, so perhaps the PPC comes into it. Cheers, Jonathan On Jul 24, 2013, at 5:48 AM, Wesley New wes...@ska.ac.za wrote: This is great work JP. Out of interest is anyone else planning to use the DDR3 on ROACH2? Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 8:24 AM, Juan-Pierre Jansen van Rensburg jvrensburg...@gmail.com wrote: Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg
Re: [casper] DRAM (DDR3) ROACH-2
Hi JP, We recently (last week) discovered we desperately need the DDR3 on ROACH2 since our work-around for getting the visibility data out of our system won't work in the long run (due to 10 GbE switch limiations... it's a long story). So just this week I re-started work on a DDR3 yellow block I had left behind a while ago. I am using the ddr3_controller and ddr3_clk modules from github.com/ska-sa/roach2_test_gateware which I believe is known to work. Originally I used the wishbone bus that was used for testing the board with a bridge but the only OPB-to-WB bridge I found online had the memory addresses hard-coded into a netlist. Instead I took the opb_dram_sniffer (which contains arbitration between CPU and FPGA) and the async_dram pcores from ROACH-1 and am in the process of interfacing them to the above modules. I have not yet determined how to translate the old MIG application interface to the new one but came up with some hack that seems to compile. I haven't tested anything in hardware yet! I'm very happy that it seems I won't have to since JP's done all the work :) Also I'm completely fine with abandoning the very little work I've done to help JP on the CPU-interface side. If the current yellow block isn't online somewhere presently could I get a copy to use in testing the DDR3 on our ROACH2's? Thanks and good work! Rurik On Wed, Jul 24, 2013 at 7:02 AM, Jonathan Weintroub jweintr...@cfa.harvard.edu wrote: Hi JP, Wes, We need the DDR3 rather urgently (time scale of 1 to 2 months), and Rurik Primiani has recently been working on it. Laura Vertatschitsch will be joining our group in September and we had her in mind to help with this too though recently concluded her start date isn't soon enough. Of course these efforts should be coordinated, and your note, JP, is certainly apropos and well timed for us. I'll leave it to Rurik to describe what he has been doing, he talked a bit about bus interfaces, so perhaps the PPC comes into it. Cheers, Jonathan On Jul 24, 2013, at 5:48 AM, Wesley New wes...@ska.ac.za wrote: This is great work JP. Out of interest is anyone else planning to use the DDR3 on ROACH2? Wesley New South African SKA Project +2721 506 7365 www.ska.ac.za On Wed, Jul 24, 2013 at 8:24 AM, Juan-Pierre Jansen van Rensburg jvrensburg...@gmail.com wrote: Hi all I have been working on a yellow block for the DDR3 of the ROACH-2. As far as I know this yellow block does not yet exist? The same DRAM yellow block is used and interfacing the memory remains the same (as for the ROACH-1). The DRAM also uses an asynchronous fifo to allow long write bursts. I have tested the memory (thoroughly) using standard memory test patterns, and the memory passes reliably (I have yet to see a failure). I have not yet implemented a CPU interface to the DDR3, but this will hopefully be done soon. I have a couple more things that I would like to check/test, and if this is done I'll ask one of the SKA-SA guys to push this onto their CASPER mlib git repo. I thought this is information worth sharing so that multiple people don't end up working on the same thing... Hopefully this is not already the case! Thanks, JP van Rensburg
