On Wed, 6 May 2015 at 07:35 Johan Burger <[email protected]> wrote: > > Hi there, > > Do you maybe have any idea of requirement specifications for the HERA's RF > phase stability and time (?) - this might determine what technology could > be appropriate. >
Hi Johan, Thanks for your response. We're sampling at 500 MHz, so we'd like to have a stability of "few" degrees, preferably over timescales of many hours but perhaps more reasonably on a calibration cadence of O(10 minutes) PPS is not such a big deal, and synchronization to a couple of ADC clock cycles is probably fine. We're investigating simple-ish ways to calibrate these out with signal injection. > > We at SKA Africa have after some iteration come up, with a precision RF > distribution system for many antennas. The type of laser and integrated > modulator have been proven in the field on large arrays (not just > MeerKAT). The RF can be directly transmitted (in our case up to 2-3 GHz > limited by our synthesizer - the precise frequency is 1.712GHz). 500MHz RF > over fibre can be done by this as well. There is conditioning of the RF > taking place on MeerKAT at the receiving end. As Jason said, not any or all > modules really do the job properly - we converged on a solution after > testing, that implicitly included modules evaluated from KAT-7 days, and > more recent modules from other manufacturers. > > "Low precision" timing ~100ns can indeed be done using PTP. If PPS is > required instead of an Ethernet package a special conversion board (PCIe) > is necessary. This is really enough for fringe finding - used in MeerKAT > S-band for example. That digitiser is mounted in an RFI shielded pedestal > of the antenna though. We supply the high precision PPS using our custom > system as described below. > > For our L-band digitisers mounted on the outside we had to come up with > special low power, low cost, high accuracy solution - this is being > implemented by Renier and Etienne and others here at SKA Africa (so a joint > effort by our time and frequency and digitiser team). The reason is that > White Rabbit is not compatible with 10Gbe links used on this system. > Furthermore Ethernet is actually quite noisy as per MeerKAT measurements, > and White Rabbit and PTP uses that (and with highish power consumption and > largish board size), and is not preferable in a high purity clock signal > and PPS module. We found that measurement based PPS system will meet our > requirements though, for stabilized links and provides us with accurate > absolute time references at antennas, using analog methodologies. This for > example being important in pulsar science. > > I am not sure what level of RFI shielding you would be able to mount > around modules, but as said RFI from Ethernet has certainly been found to > be an RFI culprit, and cannot be therefore be used in MeerKAT close to > sensitive modules - and needs to separately shielded. This therefore means > that if PPS is generated from White Rabbit/PTP there is still some > uncertain propagation paths left (important at least for MeerKAT) up to the > point of digitization where a timing edge is inserted. We are using > seperate fibres for PPS and RF, to further limit self-RFI and as it was > found that requirements could only be met in this way. > This is a good point, and something we'll make sure to keep in mind... Thanks again, Jack > > Regards > > Johan Burger > > > > > > For MeerKAT high precision timing a special PPS solution is used. There > are seperate PPS transmitters and > On Tue, May 5, 2015 at 7:56 PM, Michael Inggs <[email protected]> wrote: > >> Hi >> >> We have a complete White Rabbit setup, including 5km fibre test reels. It >> is driven by a Meinberg Plenum 1 clock. We are quite happy to chat with >> anyone on this topic of phase coherence. As you all know, the hardware and >> firmware is all Open Source, so as Jason mentions, easy to incorporate. I >> think I saw that the new CASPER card has provision for WR? Members of the >> team have specialised GPSDO systems developed in the lab that seem to give >> below 10 ns of jitter. >> >> Someone has tested it over 900km and beyond in Norway and sub ps jitter >> is achieved. >> >> One drawback is it needs a 1Ge fibre link, which I guess means, these >> days, its own fibre. We spoke to CERN about developing 10Ge and above, but >> it looks complex due to the many channels embedded in these streams. There >> has been some good work on sending it over microwave, but I guess that is a >> no-no for astronomy reserves. >> >> Regards >> >> >> >> On 5 May 2015 at 19:27, Jason Manley <[email protected]> wrote: >> >>> White rabbit is the version of IEEE1588 that locks bit clocks. Normal >>> IEEE1588 doesn't do this. For this reason, (last I looked), there was no >>> 10G implementation. 1G speeds is the highest White Rabbit implementation. >>> The HW support for "normal" IEEE1588 is used to timestamp the packets >>> without software in the loop (where the CPU will introduce jitter). >>> >>> But you don't necessarily need any special HW for IEEE1588... it can all >>> be done in a normal FPGA using the standard Xilinx 10G IP core. There are >>> FPGA cores available for the IEEE1588 part, so you don't have to implement >>> it yourself. We considered this for MeerKAT at one stage, but in the end we >>> couldn't achieve the required performance. It might be quite workable for >>> HERA, though. Anyway, just a thought. It'd save you buying special HW and >>> running additional fibres, if it meets your performance targets. >>> >>> Jason Manley >>> CBF Manager >>> SKA-SA >>> >>> Cell: +27 82 662 7726 >>> Work: +27 21 506 7300 >>> >>> On 05 May 2015, at 18:49, Dan Werthimer <[email protected]> wrote: >>> >>> > >>> > >>> > hi dave, >>> > >>> > i also think distributing clock and 1 PPS is simpler than IEEE1588. >>> > >>> > some of the IEEE1588 and white rabbit experts are here at berkeley. >>> > see for example: >>> > http://chess.eecs.berkeley.edu/pubs/881/dreams.pdf >>> > >>> > my limited understanding is that 1588 phase locks the bit clocks of >>> > routers and switches together. 1588 routers and switches have SMA >>> > connectors on them so you can use external maser/rubidium/GPS >>> references. >>> > >>> > you can achieve spectacular accuracy and stabilitity with white rabbit >>> > if you employ really good oscillators at each node, >>> > i think white rabbit can acheive 70 picosecond RMS time transfer. >>> > >>> > best, >>> > >>> > dan >>> > >>> > >>> > >>> > >>> > >>> > On Tue, May 5, 2015 at 8:52 AM, David MacMahon < >>> [email protected]> wrote: >>> > Hi, Jason, >>> > >>> > I have a great deal of curiosity about IEEE-1588, but I've always >>> wondered about the precision/stability that's attainable. Compared with >>> multiple sample clocks, correlating signals sampled with one common clock >>> seems far more forgiving vis a vis clock frequency/phase stability. If you >>> or John could point me to any information about this, please do! >>> > >>> > Thanks, >>> > Dave >>> > >>> > On May 4, 2015, at 11:44 PM, Jason Manley wrote: >>> > >>> > > On the far end of the concept spectrum, have you considered >>> distributing time over your existing ethernet network with IEEE-1588, and >>> using this to discipline local ovenised 10MHz oscs at each antenna? >>> > > >>> > > I'm cc'ing Johan Burger, who heads up our Timing and Frequency >>> Reference subsystem, who might be able to offer some additional insight. I >>> know they've tried a few different lasers and detectors, with varying >>> levels of success. >>> > > >>> > > Jason Manley >>> > > CBF Manager >>> > > SKA-SA >>> > > >>> > > Cell: +27 82 662 7726 >>> > > Work: +27 21 506 7300 >>> > > >>> > > On 05 May 2015, at 5:18, Bob Stricklin <[email protected]> wrote: >>> > > >>> > >> Hi Jack and John, >>> > >> >>> > >> I wanted to add an input here….. >>> > >> >>> > >> I am working on a 10 MHz GPS slaved reference for my personal use. >>> I am working with a Analog Devices AD9548 Evaluation board (~$250) , GPS >>> with 1 PPS, and a ovenized 10 MHz osc. I also plan to distribute this clock >>> and have considered the Avago fiber product line. One of the older >>> generation Avago fiber parts should work fine for <$25 per channel. With >>> careful control of lengths and delays it should be possible to maintain >>> good phasing between channels. The analog devices chip is <$50 so a custom >>> solution should be <$500/reference but with considerable development time. >>> > >> >>> > >> Bob Stricklin >>> > >> >>> > >> >>> > >>> On May 4, 2015, at 10:02 PM, Jack Hickish <[email protected]> >>> wrote: >>> > >>> >>> > >>> Hi John, >>> > >>> >>> > >>> Thanks for the info. I'll add Litelink to my list of suppliers to >>> investigate. >>> > >>> We have no particular urge to multiplex the signals on to the >>> fiber unless there's a particularly neat/cheap solution to do that. There's >>> no great appetite to go custom. We've got about ~30 nodes, and my first >>> stab at getting an off-the-shelf solution turned up at a few k$ / node, not >>> including any cleanup electronics. >>> > >>> >>> > >>> Thanks again, >>> > >>> >>> > >>> Jack >>> > >>> >>> > >>> On Mon, 4 May 2015 at 19:25 John Ford <[email protected]> wrote: >>> > >>>> Hi CASPERites, >>> > >>>> >>> > >>>> For HERA, we're looking at distributing timing signals (PPS & >>> 10Mhz ref or >>> > >>>> 500 MHz clock) over O(100m) fibers to various digitization nodes. >>> > >>>> I figure some folks in CASPERland have experience with this kind >>> of >>> > >>>> system? >>> > >>>> Did you use custom RF-over-fiber kit, or off-the-shelf PPS/10MHz >>> > >>>> solutions? >>> > >>>> Any words of wisdom/caution to share? >>> > >>>> >>> > >>>> Any responses much appreciated! >>> > >>> >>> > >>> We have several different schemes for the different signals. Are >>> you >>> > >>> planning for one fiber per signal per node? or one fiber with the >>> signals >>> > >>> multiplexed on them? >>> > >>> >>> > >>> If the signals are one per signal, you can use some off-the-shelf >>> > >>> solutions, but they are kind of pricey, and if you have a lot of >>> nodes to >>> > >>> supply, it might be worth working on something custom. We have >>> used Math >>> > >>> Associates stuff for this kind of work. Math Associates is now >>> litelink, >>> > >>> and they tout the affordability of their stuff, so maybe it's >>> > >>> reasonable... >>> > >>> >>> > >>> >>> > >>> On the 10 MHz, we send the 10 MHz reference over fiber, and at the >>> far end >>> > >>> use a crystal oscillator locked to the reference to clean up the >>> noise >>> > >>> from the fiber electronics. This is essential for interferometry, >>> but >>> > >>> maybe not for single-dish use. >>> > >>> >>> > >>> John >>> > >>> >>> > >>>> >>> > >>>> Jack >>> > >>>> >>> > >>> >>> > >>> >>> > >> >>> > > >>> > > >>> > >>> > >>> > >>> >>> >>> >> >> >> -- >> Michael Inggs >> Department of Electrical Engineering, University of Cape Town, Private >> Bag, Rondebosch 7701, South Africa. Tel: +27 21 650 2799 Fax: +27 21 650 >> 3465 Skype: mikings >> "Ex Africa semper aliquid novi" >> >> >
