Casperites, Thank you so much for all of the information and the speedy replies. This is such a kind community to work with. We will get this guy running yet!
--Laura On Fri, Mar 16, 2012 at 4:08 PM, Louis Dartez <[email protected]> wrote: > Hello Laura, > > At UT Brownsville we use > this<http://www.colfaxdirect.com/store/pc/viewPrd.asp?idproduct=958>card on a > machine running CentOS 5. Since CentOS is basically RedHat or > Fedora, I don't think you'd have a problem with compatibility. > > > -Louis P. Dartez > > On Fri, Mar 16, 2012 at 5:54 PM, Laura Vertatschitsch > <[email protected]>wrote: > >> Any specific recommendations for a 10Ge NIC? Our sys admin is warning us >> about the compatibility with Red Hat (I think we are running red hat 6 on >> the computer we would like to stream to.) He had suggested that >> this<http://www.cdw.com/shop/products/Myricom-Myri-10G-network-adapter/2034587.aspx>may >> work, but I would be grateful for input from others that are currently >> using a RedHat machine with a 10GbE NIC. >> >> --Laura >> >> >> On Thu, Mar 15, 2012 at 9:45 AM, Dan Werthimer <[email protected]>wrote: >> >>> >>> hi andy and others interested: >>> >>> i suggest we create a section on the casper wiki >>> that documents the various open source programs that >>> people have developed for receiving and processing10Gbit data >>> on computers and GPU's. >>> >>> there's PSRDADA and GUPPI/VEGAS/HASHPIPE: >>> PSRDADA was developed by matthew bailes' group in australia. >>> GUPPI was developed at NRAO, mostly by paul demorest, for pulsar >>> instrumentation. >>> VEGAS and HASHPIPE are berkeley derivatives of GUPPI for spectrometers >>> and correlators. >>> these opensource codes typically have one thread that moves data from a >>> NIC to a circular buffer, >>> another thread the moves the data from a circular buffer to a GPU or CPU >>> based compute process. >>> another thread the moves the output from GPU/CPU to a circular output >>> buffer, >>> and another thread that write the output buffers to disk. >>> >>> optionally, there's a fifth thread that gathers meta data from a >>> telescope >>> (eg: pointing and frequency information) and puts this info into the >>> circular buffers. >>> and there are optional several processes that perform monitor and >>> control to make sure >>> everything is working, display data, etc. >>> >>> >>> best wishes, >>> >>> dan >>> >>> >>> >>> On Thu, Mar 15, 2012 at 6:24 AM, Andrew Lutomirski <[email protected]> wrote: >>> >>>> >>>> On Mar 14, 2012 3:26 PM, "Dan Werthimer" <[email protected]> wrote: >>>> > >>>> > >>>> > >>>> > hi laura, >>>> > >>>> > i second john's remarks: >>>> > >>>> > an inexpensive 10Gbit nic card in your computer >>>> > would make your data streaming task relatively easy, >>>> > as there are tutorials and several instrument designs >>>> > that stream data from a roach over 10Gbe into a computer. >>>> > >>>> > there's also a lot of good open source software you can use >>>> > to receive 10gBE data on a computer and send it to a disk drive.... >>>> > >>>> >>>> I have a program to do exactly this on the roach. MIT should allow me >>>> to open-source it any day now. >>>> >>>> --Andy >>>> >>>> > best wishes, >>>> > >>>> > dan >>>> > >>>> > >>>> > On Wed, Mar 14, 2012 at 3:18 PM, John Ford <[email protected]> wrote: >>>> >> >>>> >> > Hey Andrew, >>>> >> > >>>> >> > Thanks for the reply - we are looking forward to the ROACH 2 in the >>>> >> > future. >>>> >> > >>>> >> > I haven't been able to open all of the tutorials, but I have seen >>>> a few. >>>> >> > So I apologize if I may have missed something obvious, and I >>>> would even >>>> >> > really appreciate an answer that says "hey you, go read about >>>> this." I >>>> >> > see >>>> >> > that data can be saved using a snap block, and I could save a >>>> concatenated >>>> >> > set of samples as a single 64 bit integer, and I can save a >>>> maximum of >>>> >> > 2^16 >>>> >> > of these. Is there a better way of using the roach memory to >>>> guarantee >>>> >> > that I can read and save all of the data continuously? I'm >>>> wondering if >>>> >> > there is perhaps a way of constantly reading from a snap register >>>> fast >>>> >> > enough to insure that we have captured the data continuously and >>>> saved it >>>> >> > to our external computer. >>>> >> >>>> >> Hi Laura. In addition to what Dan just said, I want to point out >>>> that the >>>> >> 10m/100m/1g bps port (on at least many ROACH I's) is not reliable at >>>> 1 >>>> >> gbps. >>>> >> >>>> >> So you would be streaming 10 MB/s (80 mb/s plus framing) over a 100 >>>> mb >>>> >> link. Pretty risky... >>>> >> >>>> >> The 10 gbe solution is better in so many ways. If your computer is >>>> within >>>> >> a few meters of the ROACH, I highly recommend you think about buying >>>> a 10 >>>> >> gb card for the computer. It'll only cost ~$500.00 plus a $100.00 >>>> cable, >>>> >> and will save you a lot of sleepless nights, I think! >>>> >> >>>> >> John >>>> >> >>>> >> > >>>> >> > --Laura >>>> >> > >>>> >> > >>>> >> > On Mon, Mar 12, 2012 at 11:59 PM, Andrew Martens <[email protected]> >>>> wrote: >>>> >> > >>>> >> >> Hi Laura >>>> >> >> >>>> >> >> The ROACH does not have a direct 1Ge connection to the FPGA, any >>>> data >>>> >> >> must exit via the PPC if it is not going through the 10Ge links. >>>> >> >> >>>> >> >> The ROACH2 however, has a 1Ge link directly to the FPGA. The >>>> yellow >>>> >> >> block has been completed and has been designed to act the same as >>>> the >>>> >> >> 10Ge links (basically the same interface, can be accessed from >>>> the PPC >>>> >> >> if needed, ARP table managed from PPC etc) except that the data >>>> rate is >>>> >> >> lower (8 bit data paths instead of 64 bit). >>>> >> >> >>>> >> >> We should be finalising these yellow blocks and making an >>>> announcement >>>> >> >> soon so that people who are interested can start designing for >>>> ROACH2. >>>> >> >> >>>> >> >> Regards >>>> >> >> Andrew >>>> >> >> >>>> >> >> On Mon, 2012-03-12 at 17:22 -0700, Laura Vertatschitsch wrote: >>>> >> >> > Hey Casperites, >>>> >> >> > >>>> >> >> > I see a lot of data about reliable streaming using the 10GbE >>>> ports and >>>> >> >> > a lovely simulink block to boot. Is there an analogous method >>>> for >>>> >> >> > streaming data out over the 1Gbps ethernet? >>>> >> >> > >>>> >> >> > Not sure if someone has written some python control scripts to >>>> >> >> > accomplish this - I may have just missed it on the wiki. We >>>> would >>>> >> >> > love to stream 10MHz time-domain data off our board. >>>> >> >> > >>>> >> >> > --Laura >>>> >> >> >>>> >> >> >>>> >> >> >>>> >> > >>>> >> >>>> >> >>>> >> >>>> > >>>> >>> >>> >> > > > -- > -- Louis P. Dartez > (956) 372-5812 > Arecibo Remote Command Center Scholar > Center for Advanced Radio Astronomy > University of Texas at Brownsville > >
