Hello Laura,
At UT Brownsville we use
this<http://www.colfaxdirect.com/store/pc/viewPrd.asp?idproduct=958>card
on a machine running CentOS 5. Since CentOS is basically RedHat or
Fedora, I don't think you'd have a problem with compatibility.
-Louis P. Dartez
On Fri, Mar 16, 2012 at 5:54 PM, Laura Vertatschitsch <[email protected]>wrote:
> Any specific recommendations for a 10Ge NIC? Our sys admin is warning us
> about the compatibility with Red Hat (I think we are running red hat 6 on
> the computer we would like to stream to.) He had suggested that
> this<http://www.cdw.com/shop/products/Myricom-Myri-10G-network-adapter/2034587.aspx>may
> work, but I would be grateful for input from others that are currently
> using a RedHat machine with a 10GbE NIC.
>
> --Laura
>
>
> On Thu, Mar 15, 2012 at 9:45 AM, Dan Werthimer <[email protected]>wrote:
>
>>
>> hi andy and others interested:
>>
>> i suggest we create a section on the casper wiki
>> that documents the various open source programs that
>> people have developed for receiving and processing10Gbit data
>> on computers and GPU's.
>>
>> there's PSRDADA and GUPPI/VEGAS/HASHPIPE:
>> PSRDADA was developed by matthew bailes' group in australia.
>> GUPPI was developed at NRAO, mostly by paul demorest, for pulsar
>> instrumentation.
>> VEGAS and HASHPIPE are berkeley derivatives of GUPPI for spectrometers
>> and correlators.
>> these opensource codes typically have one thread that moves data from a
>> NIC to a circular buffer,
>> another thread the moves the data from a circular buffer to a GPU or CPU
>> based compute process.
>> another thread the moves the output from GPU/CPU to a circular output
>> buffer,
>> and another thread that write the output buffers to disk.
>>
>> optionally, there's a fifth thread that gathers meta data from a
>> telescope
>> (eg: pointing and frequency information) and puts this info into the
>> circular buffers.
>> and there are optional several processes that perform monitor and control
>> to make sure
>> everything is working, display data, etc.
>>
>>
>> best wishes,
>>
>> dan
>>
>>
>>
>> On Thu, Mar 15, 2012 at 6:24 AM, Andrew Lutomirski <[email protected]> wrote:
>>
>>>
>>> On Mar 14, 2012 3:26 PM, "Dan Werthimer" <[email protected]> wrote:
>>> >
>>> >
>>> >
>>> > hi laura,
>>> >
>>> > i second john's remarks:
>>> >
>>> > an inexpensive 10Gbit nic card in your computer
>>> > would make your data streaming task relatively easy,
>>> > as there are tutorials and several instrument designs
>>> > that stream data from a roach over 10Gbe into a computer.
>>> >
>>> > there's also a lot of good open source software you can use
>>> > to receive 10gBE data on a computer and send it to a disk drive....
>>> >
>>>
>>> I have a program to do exactly this on the roach. MIT should allow me
>>> to open-source it any day now.
>>>
>>> --Andy
>>>
>>> > best wishes,
>>> >
>>> > dan
>>> >
>>> >
>>> > On Wed, Mar 14, 2012 at 3:18 PM, John Ford <[email protected]> wrote:
>>> >>
>>> >> > Hey Andrew,
>>> >> >
>>> >> > Thanks for the reply - we are looking forward to the ROACH 2 in the
>>> >> > future.
>>> >> >
>>> >> > I haven't been able to open all of the tutorials, but I have seen a
>>> few.
>>> >> > So I apologize if I may have missed something obvious, and I would
>>> even
>>> >> > really appreciate an answer that says "hey you, go read about
>>> this." I
>>> >> > see
>>> >> > that data can be saved using a snap block, and I could save a
>>> concatenated
>>> >> > set of samples as a single 64 bit integer, and I can save a maximum
>>> of
>>> >> > 2^16
>>> >> > of these. Is there a better way of using the roach memory to
>>> guarantee
>>> >> > that I can read and save all of the data continuously? I'm
>>> wondering if
>>> >> > there is perhaps a way of constantly reading from a snap register
>>> fast
>>> >> > enough to insure that we have captured the data continuously and
>>> saved it
>>> >> > to our external computer.
>>> >>
>>> >> Hi Laura. In addition to what Dan just said, I want to point out
>>> that the
>>> >> 10m/100m/1g bps port (on at least many ROACH I's) is not reliable at 1
>>> >> gbps.
>>> >>
>>> >> So you would be streaming 10 MB/s (80 mb/s plus framing) over a 100 mb
>>> >> link. Pretty risky...
>>> >>
>>> >> The 10 gbe solution is better in so many ways. If your computer is
>>> within
>>> >> a few meters of the ROACH, I highly recommend you think about buying
>>> a 10
>>> >> gb card for the computer. It'll only cost ~$500.00 plus a $100.00
>>> cable,
>>> >> and will save you a lot of sleepless nights, I think!
>>> >>
>>> >> John
>>> >>
>>> >> >
>>> >> > --Laura
>>> >> >
>>> >> >
>>> >> > On Mon, Mar 12, 2012 at 11:59 PM, Andrew Martens <[email protected]>
>>> wrote:
>>> >> >
>>> >> >> Hi Laura
>>> >> >>
>>> >> >> The ROACH does not have a direct 1Ge connection to the FPGA, any
>>> data
>>> >> >> must exit via the PPC if it is not going through the 10Ge links.
>>> >> >>
>>> >> >> The ROACH2 however, has a 1Ge link directly to the FPGA. The yellow
>>> >> >> block has been completed and has been designed to act the same as
>>> the
>>> >> >> 10Ge links (basically the same interface, can be accessed from
>>> the PPC
>>> >> >> if needed, ARP table managed from PPC etc) except that the data
>>> rate is
>>> >> >> lower (8 bit data paths instead of 64 bit).
>>> >> >>
>>> >> >> We should be finalising these yellow blocks and making an
>>> announcement
>>> >> >> soon so that people who are interested can start designing for
>>> ROACH2.
>>> >> >>
>>> >> >> Regards
>>> >> >> Andrew
>>> >> >>
>>> >> >> On Mon, 2012-03-12 at 17:22 -0700, Laura Vertatschitsch wrote:
>>> >> >> > Hey Casperites,
>>> >> >> >
>>> >> >> > I see a lot of data about reliable streaming using the 10GbE
>>> ports and
>>> >> >> > a lovely simulink block to boot. Is there an analogous method
>>> for
>>> >> >> > streaming data out over the 1Gbps ethernet?
>>> >> >> >
>>> >> >> > Not sure if someone has written some python control scripts to
>>> >> >> > accomplish this - I may have just missed it on the wiki. We
>>> would
>>> >> >> > love to stream 10MHz time-domain data off our board.
>>> >> >> >
>>> >> >> > --Laura
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >
>>>
>>
>>
>
--
-- Louis P. Dartez
(956) 372-5812
Arecibo Remote Command Center Scholar
Center for Advanced Radio Astronomy
University of Texas at Brownsville
