[casper] QDR Help / Timing Diagram
Hi All, I've abandoned my attempt to use the DRAM for now and moved on to QDR. I'm somewhat confused by the wiki at https://casper.berkeley.edu/wiki/Qdr . The "issuing commands section" says that one type of command cannot be issued in two consecutive cycles. The "bursting" section says that data for both read and write is presented for two cycles, and that data_in and be must be set for both of those cycles. Does this mean that a "write event" lasts for two clock cycles? If so, can data_in change from the first clock cycle to the second, storing a total of 144 bits over two cycles? Thanks, --Adam Adam Schoenwald - Electrical Engineer -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To post to this group, send email to casper@lists.berkeley.edu.
Re: [casper] QDR Help / Timing Diagram
On Wed, Jul 5, 2017, 6:56 AM Schoenwald, Adam J. (GSFC-5640) < adam.schoenw...@nasa.gov> wrote: > Hi All, > > I’ve abandoned my attempt to use the DRAM for now and moved on to QDR. > > I’m somewhat confused by the wiki at https://casper.berkeley.edu/wiki/Qdr > . > > > > The “issuing commands section” says that one type of command cannot be > issued in two consecutive cycles. > > The “bursting” section says that data for both read and write is presented > for two cycles, and that data_in and be must be set for both of those > cycles. > > > > Does this mean that a “write event” lasts for two clock cycles? > > If so, can data_in change from the first clock cycle to the second, > storing a total of 144 bits over two cycles? > Correct. I believe there is a qdr transpose block in the library which may be a useful reference design. Cheers Jack > > > Thanks, > > --Adam > > > > > > Adam Schoenwald - Electrical Engineer > > > > > > -- > You received this message because you are subscribed to the Google Groups " > casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To post to this group, send email to casper@lists.berkeley.edu. > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To post to this group, send email to casper@lists.berkeley.edu.
Re: [casper] QDR Help / Timing Diagram
There's also a testbench for our ROACH2 VACC that's currently in service on MeerKAT, which may help you: https://github.com/ska-sa/cbf_fpga_testbenches/blob/master/test_qdr_vacc5.slx Note that there are calibration problems with QDR on ROACH2, so you'll likely encounter bit errors. If this matters to you, I highly recommend you leverage the ECC bits... Jason Manley CBF Manager SKA-SA Cell: +27 82 662 7726 Work: +27 21 506 7300 On 05 Jul 2017, at 16:49, Jack Hickish wrote: > > > On Wed, Jul 5, 2017, 6:56 AM Schoenwald, Adam J. (GSFC-5640) > wrote: > Hi All, > > I’ve abandoned my attempt to use the DRAM for now and moved on to QDR. > > I’m somewhat confused by the wiki at https://casper.berkeley.edu/wiki/Qdr . > > > > The “issuing commands section” says that one type of command cannot be issued > in two consecutive cycles. > > The “bursting” section says that data for both read and write is presented > for two cycles, and that data_in and be must be set for both of those cycles. > > > > Does this mean that a “write event” lasts for two clock cycles? > > If so, can data_in change from the first clock cycle to the second, storing a > total of 144 bits over two cycles? > > > Correct. I believe there is a qdr transpose block in the library which may be > a useful reference design. > > Cheers > Jack > > > > Thanks, > > --Adam > > > > > > Adam Schoenwald - Electrical Engineer > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To post to this group, send email to casper@lists.berkeley.edu. > > -- > You received this message because you are subscribed to the Google Groups > "casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To post to this group, send email to casper@lists.berkeley.edu. -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To post to this group, send email to casper@lists.berkeley.edu.
RE: [casper] QDR Help / Timing Diagram
Thanks Jack, Jason, I think I have the QDR Timing down now, at least in simulation. Do I need to do anything special to use the ECC bits? If I want to calibrate QDR, does that mean I must include the CPU interface? Is this related to how I set the be flag ? Thanks, --Adam Adam Schoenwald - Electrical Engineer Office: 301.286.4175 | Cell & Text: 631-241-0003 -Original Message- From: Jason Manley [mailto:jman...@ska.ac.za] Sent: Wednesday, July 5, 2017 10:55 AM To: Jack Hickish Cc: Schoenwald, Adam J. (GSFC-5640) ; Casper Lists Subject: Re: [casper] QDR Help / Timing Diagram There's also a testbench for our ROACH2 VACC that's currently in service on MeerKAT, which may help you: https://github.com/ska-sa/cbf_fpga_testbenches/blob/master/test_qdr_vacc5.slx Note that there are calibration problems with QDR on ROACH2, so you'll likely encounter bit errors. If this matters to you, I highly recommend you leverage the ECC bits... Jason Manley CBF Manager SKA-SA Cell: +27 82 662 7726 Work: +27 21 506 7300 On 05 Jul 2017, at 16:49, Jack Hickish wrote: > > > On Wed, Jul 5, 2017, 6:56 AM Schoenwald, Adam J. (GSFC-5640) > wrote: > Hi All, > > I've abandoned my attempt to use the DRAM for now and moved on to QDR. > > I'm somewhat confused by the wiki at https://casper.berkeley.edu/wiki/Qdr . > > > > The "issuing commands section" says that one type of command cannot be issued > in two consecutive cycles. > > The "bursting" section says that data for both read and write is presented > for two cycles, and that data_in and be must be set for both of those cycles. > > > > Does this mean that a "write event" lasts for two clock cycles? > > If so, can data_in change from the first clock cycle to the second, storing a > total of 144 bits over two cycles? > > > Correct. I believe there is a qdr transpose block in the library which may be > a useful reference design. > > Cheers > Jack > > > > Thanks, > > --Adam > > > > > > Adam Schoenwald - Electrical Engineer > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To post to this group, send email to casper@lists.berkeley.edu. > > -- > You received this message because you are subscribed to the Google Groups > "casper@lists.berkeley.edu" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to casper+unsubscr...@lists.berkeley.edu. > To post to this group, send email to casper@lists.berkeley.edu. -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To post to this group, send email to casper@lists.berkeley.edu.
Re: [casper] QDR Help / Timing Diagram
On Wed, 5 Jul 2017 at 11:01 Schoenwald, Adam J. (GSFC-5640) < adam.schoenw...@nasa.gov> wrote: > Thanks Jack, Jason, > I think I have the QDR Timing down now, at least in simulation. > > Do I need to do anything special to use the ECC bits? If I want to > calibrate QDR, does that mean I must include the CPU interface? Is this > related to how I set the be flag ? > The CPU interface is needed, as you surmise. Depending on what version of mlib_devel you're using, the CPU interface yellow block option parameter should be overridden by the toolflow in the case of ROACH2, but there's no harm in manually enabling the interface.. The ECC bits are made available to simulink (i.e., the data bus is a multiple of 9 bits, not 8 bits) but (someone can correct me if I'm wrong) you have to implement the correction code yourself, the controller doesn't do anything smart with the extra bits, it just makes them available for user storage. Cheers Jack PS -- please come to the CASPER workshop! > > Thanks, > --Adam > > > Adam Schoenwald - Electrical Engineer > Office: 301.286.4175 | Cell & Text: 631-241-0003 > > > > -Original Message- > From: Jason Manley [mailto:jman...@ska.ac.za] > Sent: Wednesday, July 5, 2017 10:55 AM > To: Jack Hickish > Cc: Schoenwald, Adam J. (GSFC-5640) ; Casper > Lists > Subject: Re: [casper] QDR Help / Timing Diagram > > There's also a testbench for our ROACH2 VACC that's currently in service > on MeerKAT, which may help you: > https://github.com/ska-sa/cbf_fpga_testbenches/blob/master/test_qdr_vacc5.slx > > Note that there are calibration problems with QDR on ROACH2, so you'll > likely encounter bit errors. If this matters to you, I highly recommend you > leverage the ECC bits... > > Jason Manley > CBF Manager > SKA-SA > > Cell: +27 82 662 7726 <+27%2082%20662%207726> > Work: +27 21 506 7300 <+27%2021%20506%207300> > > On 05 Jul 2017, at 16:49, Jack Hickish wrote: > > > > > > > On Wed, Jul 5, 2017, 6:56 AM Schoenwald, Adam J. (GSFC-5640) < > adam.schoenw...@nasa.gov> wrote: > > Hi All, > > > > I've abandoned my attempt to use the DRAM for now and moved on to QDR. > > > > I'm somewhat confused by the wiki at > https://casper.berkeley.edu/wiki/Qdr . > > > > > > > > The "issuing commands section" says that one type of command cannot be > issued in two consecutive cycles. > > > > The "bursting" section says that data for both read and write is > presented for two cycles, and that data_in and be must be set for both of > those cycles. > > > > > > > > Does this mean that a "write event" lasts for two clock cycles? > > > > If so, can data_in change from the first clock cycle to the second, > storing a total of 144 bits over two cycles? > > > > > > Correct. I believe there is a qdr transpose block in the library which > may be a useful reference design. > > > > Cheers > > Jack > > > > > > > > Thanks, > > > > --Adam > > > > > > > > > > > > Adam Schoenwald - Electrical Engineer > > > > > > > > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "casper@lists.berkeley.edu" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to casper+unsubscr...@lists.berkeley.edu. > > To post to this group, send email to casper@lists.berkeley.edu. > > > > -- > > You received this message because you are subscribed to the Google > Groups "casper@lists.berkeley.edu" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to casper+unsubscr...@lists.berkeley.edu. > > To post to this group, send email to casper@lists.berkeley.edu. > > -- You received this message because you are subscribed to the Google Groups "casper@lists.berkeley.edu" group. To unsubscribe from this group and stop receiving emails from it, send an email to casper+unsubscr...@lists.berkeley.edu. To post to this group, send email to casper@lists.berkeley.edu.
Re: [casper] Configuring 40GbE Yellow Block in JASPER
Hello, I am still having issues receiving data over 40 GbE. I currently have a direct connection between the transmitting and receiving SKARABs, both on the leftmost 40 GbE port of Mezzanine slot 3 . Using tcpdump on a server with a 40GbE interface, I was able to confirm that the SKARAB which transmits data is successfully doing so. Also, both SKARABs can successfully be pinged over 40GbE. Despite this, I am still unable to receive data on the other SKARAB, even when Promiscuous Mode is enabled. I did observe that the orange and green LEDs are turned on for both of the SKARAB's 40GbE ports. The green light blinks on the SKARAB that is transmitting data and the orange light blinks on the SKARAB that is receiving data, therefore it appears they are communicating with each other. Any other suggestions on how to debug the reception data over 40 GbE would be much appreciated, as I am still unsure what could be causing the issue. Thanks, Mark On Fri, Jun 30, 2017 at 3:57 AM, Paul Prozesky wrote: > Morning Mark > > Please have a look here for SKARAB 40gbe TX and RX models with demo > software. > > https://github.com/ska-sa/mkat_fpga/tree/devel/source/skarab_dev/tut2 > > Cheers > Paul > > > On 30 June 2017 at 07:46, Jason Manley wrote: > >> Hi Mark >> >> The current SKARAB 40G yellowblock is a bit of a hack. >> >> 1) It is currently not parameterised, and is hard-coded for the first >> port on Mezzanine slot 3. >> >> 2) The tx_valid and rx_valid lines are 4-bits wide, not 1 bit. This >> will be rectified soon, but in the meanwhile, just feed the value "3" to >> tx_valid to send packets. >> >> 3) Note that there was a recent change to the TX and RX 64-bit word >> ordering (was previously incorrectly swapped). Make you built your >> bitstreams with a recent git checkout. >> >> It's on our todo list to get it properly parameterised and to fix the >> 4-bit weirdness, but it's not clear when we'll have that done. In the >> meanwhile, the 40G does work, and we are successfully using this first port >> actively at SKA-SA. >> >> Some things you should try to help debug: >> >> 1) The microblaze will attempt to DHCP on the 40G interface. Did it >> obtain a lease? Check your DHCP server logs. >> >> 2) Check (using casperfpga software) that the cores are actually >> configured like you think. There was a version of microblaze that overwrote >> things. And if it's DHCP'ing (this is how we use the ports), then the IP >> will have changed from what you expect. >> myfpgaobject.gbes['my_forty_gbe_core'].print_core_details() >> (or g.get_core_details()). >> >> 3) What does tcpdump/wireshark show is coming out of the TX board? >> >> 4) Did you select different MAC addresses for the "hardcoded" >> yellowblocks on the TX and RX side? Else a switch might not forward the >> packets. >> >> 5) Can you ping both 40G interfaces from a computer? >> >> Jason Manley >> CBF Manager >> SKA-SA >> >> Cell: +27 82 662 7726 <+27%2082%20662%207726> >> Work: +27 21 506 7300 <+27%2021%20506%207300> >> >> On 29 Jun 2017, at 20:37, Peryer, Mark A. >> wrote: >> >> > Hello, >> > >> > I am trying to send data from one SKARAB to another SKARAB over 40GbE. >> I have created two separate JASPER files, one for receiving and one for >> transmitting. Each design has one forty_gbe yellow block and are configured >> similar to Tutorial 2 on the casper website. In the JASPER file used for >> transmitting, I have placed a snapshot block on the tx_data signal for the >> forty_gbe block and am able to read the correct data from the snapshot >> block using casperfpga. While it appears that the correct data is being >> transmitted, nothing is received by the second SKARAB. In the JASPER file >> used for receiving data, I have a snapshot block on the rx_data signal, >> which should trigger once a valid frame is received. However, when I run >> fpga01.snapshots.snapshot2.print_snap(50) it just hangs, indicating >> nothing has been received. >> > >> > I did notice that when I right click on the forty_gbe yellow block and >> go to Mask >> Edit Mask >> Parameters&Dialog, there is a menu that allows >> for the card slot location of the 40GbE card to be selected. Our SKARAB >> units have the 40GbE card populated on Mezzanine Slot 3, however only Card >> Slot 0 and 1 are available to be selected. Should this option be set to >> slot 3 in order to properly use the 40GbE ports? >> > >> > I should also mention that I have configured the tx_dest_ip block input >> to be 192.168.5.20 and the tx_dest_port to be 1, which are the default >> values for the forty_gbe block. >> > >> > If there is anything else I should look at in order to properly >> configure the forty_gbe yellow blocks please advise. >> > >> > Thanks, >> > >> > Mark >> > >> > >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "casper@lists.berkeley.edu" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email t
Re: [casper] Configuring 40GbE Yellow Block in JASPER
Hello, I have solved my issue of not being able to receive data. Instead of directly connecting the SKARABs, I routed them through a 40 GbE switch where the DHCP server can assign both of the SKARABs an IP address. I am still not sure why directly connecting them does not work, however that appears to be what was causing the issue all along. Thanks for your help, Mark On Wed, Jul 5, 2017 at 4:20 PM, Peryer, Mark A. wrote: > Hello, > > I am still having issues receiving data over 40 GbE. I currently have a > direct connection between the transmitting and receiving SKARABs, both on > the leftmost 40 GbE port of Mezzanine slot 3 . Using tcpdump on a server > with a 40GbE interface, I was able to confirm that the SKARAB which > transmits data is successfully doing so. Also, both SKARABs can > successfully be pinged over 40GbE. Despite this, I am still unable to > receive data on the other SKARAB, even when Promiscuous Mode is enabled. I > did observe that the orange and green LEDs are turned on for both of the > SKARAB's 40GbE ports. The green light blinks on the SKARAB that is > transmitting data and the orange light blinks on the SKARAB that is > receiving data, therefore it appears they are communicating with each > other. > > Any other suggestions on how to debug the reception data over 40 GbE would > be much appreciated, as I am still unsure what could be causing the issue. > > Thanks, > > Mark > > On Fri, Jun 30, 2017 at 3:57 AM, Paul Prozesky > wrote: > >> Morning Mark >> >> Please have a look here for SKARAB 40gbe TX and RX models with demo >> software. >> >> https://github.com/ska-sa/mkat_fpga/tree/devel/source/skarab_dev/tut2 >> >> Cheers >> Paul >> >> >> On 30 June 2017 at 07:46, Jason Manley wrote: >> >>> Hi Mark >>> >>> The current SKARAB 40G yellowblock is a bit of a hack. >>> >>> 1) It is currently not parameterised, and is hard-coded for the first >>> port on Mezzanine slot 3. >>> >>> 2) The tx_valid and rx_valid lines are 4-bits wide, not 1 bit. This >>> will be rectified soon, but in the meanwhile, just feed the value "3" to >>> tx_valid to send packets. >>> >>> 3) Note that there was a recent change to the TX and RX 64-bit word >>> ordering (was previously incorrectly swapped). Make you built your >>> bitstreams with a recent git checkout. >>> >>> It's on our todo list to get it properly parameterised and to fix the >>> 4-bit weirdness, but it's not clear when we'll have that done. In the >>> meanwhile, the 40G does work, and we are successfully using this first port >>> actively at SKA-SA. >>> >>> Some things you should try to help debug: >>> >>> 1) The microblaze will attempt to DHCP on the 40G interface. Did it >>> obtain a lease? Check your DHCP server logs. >>> >>> 2) Check (using casperfpga software) that the cores are actually >>> configured like you think. There was a version of microblaze that overwrote >>> things. And if it's DHCP'ing (this is how we use the ports), then the IP >>> will have changed from what you expect. >>> myfpgaobject.gbes['my_forty_gbe_core'].print_core_details() >>> (or g.get_core_details()). >>> >>> 3) What does tcpdump/wireshark show is coming out of the TX board? >>> >>> 4) Did you select different MAC addresses for the "hardcoded" >>> yellowblocks on the TX and RX side? Else a switch might not forward the >>> packets. >>> >>> 5) Can you ping both 40G interfaces from a computer? >>> >>> Jason Manley >>> CBF Manager >>> SKA-SA >>> >>> Cell: +27 82 662 7726 <+27%2082%20662%207726> >>> Work: +27 21 506 7300 <+27%2021%20506%207300> >>> >>> On 29 Jun 2017, at 20:37, Peryer, Mark A. >>> wrote: >>> >>> > Hello, >>> > >>> > I am trying to send data from one SKARAB to another SKARAB over 40GbE. >>> I have created two separate JASPER files, one for receiving and one for >>> transmitting. Each design has one forty_gbe yellow block and are configured >>> similar to Tutorial 2 on the casper website. In the JASPER file used for >>> transmitting, I have placed a snapshot block on the tx_data signal for the >>> forty_gbe block and am able to read the correct data from the snapshot >>> block using casperfpga. While it appears that the correct data is being >>> transmitted, nothing is received by the second SKARAB. In the JASPER file >>> used for receiving data, I have a snapshot block on the rx_data signal, >>> which should trigger once a valid frame is received. However, when I run >>> fpga01.snapshots.snapshot2.print_snap(50) it just hangs, indicating >>> nothing has been received. >>> > >>> > I did notice that when I right click on the forty_gbe yellow block and >>> go to Mask >> Edit Mask >> Parameters&Dialog, there is a menu that allows >>> for the card slot location of the 40GbE card to be selected. Our SKARAB >>> units have the 40GbE card populated on Mezzanine Slot 3, however only Card >>> Slot 0 and 1 are available to be selected. Should this option be set to >>> slot 3 in order to properly use the 40GbE ports? >>> > >>> > I should also m
