Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
hi randy, the sync outputs from the adc yellow block are a copy of the signal that is injected into the adc's sync input SMA connector. (in your case, the 1 PPS signal).all four syncs are identical. to know which adc sample is taken on the 1 second tick, one needs to calibrate by looking at a known source on the sky, or by coupling the 1 PPS signal into the analog input.this offset can change everytime the boards are powered up (there is a divide by four counter in the adc). best wishes, dan On 3/4/2010 11:06 AM, Randy Mccullough wrote: Given the following scenario... A high speed sampler comprised of... 1 iBOB with logic fabric running at 200MHz, derived from ADC0 1 ADC2x1000-8 operating in its interleaved mode with an 800MHz sampling clock 1 1PPS Site Timing Reference applied to the ADC's SYNC IN Is it possible, using the four SYNC outputs of the ADC block, to ascertain which of the 8 samples presented during a logic clock cycle was most closely aligned with an in-coming 1PPS signal? Randy
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
Hi, Randy, On Mar 4, 2010, at 11:06 , Randy Mccullough wrote: Is it possible, using the four SYNC outputs of the ADC block, to ascertain which of the 8 samples presented during a logic clock cycle was most closely aligned with an in-coming 1PPS signal? Even if the ADC sampling is synchronous with the 1 PPS signal, I think there is a 0/90/180/270 degree phase ambiguity between the Fs/4 ADC output clock and the 1 PPS signal, so I don't think you can be guaranteed that a given sync output corresponds to a given ADC output from power cycle to power cycle. While the board is running however, our experience at the ATA has shown that things are stable in whichever one of the four possible states it happened to come up in. At the ATA, an internal 1 PPS is generated from the FPGA clock (which is a divided down version of the ADC sample clock). This internal 1 PPS is sync'ed to the external 1 PPS at power on (or via software rearm). The fixed delays are calibrated in the system (including the ADC sample delay ambiguity) whenever power is cycled (or the ibobs otherwise reinitialized), but that turns out to be fairly infrequently so it's not too burdensome. Hope this helps, Dave
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
i don't think there's a reason, except perhaps decorative. henry - is there a reason for four sync's? dan On 3/4/2010 11:52 AM, Paul Demorest wrote: Hi Dan, I have to ask.. if all four syncs are the same, why are there four of them? ;) -Paul On Thu, 4 Mar 2010, Dan Werthimer wrote: hi randy, the sync outputs from the adc yellow block are a copy of the signal that is injected into the adc's sync input SMA connector. (in your case, the 1 PPS signal). all four syncs are identical. to know which adc sample is taken on the 1 second tick, one needs to calibrate by looking at a known source on the sky, or by coupling the 1 PPS signal into the analog input. this offset can change everytime the boards are powered up (there is a divide by four counter in the adc). best wishes, dan On 3/4/2010 11:06 AM, Randy Mccullough wrote: Given the following scenario... A high speed sampler comprised of... 1 iBOB with logic fabric running at 200MHz, derived from ADC0 1 ADC2x1000-8 operating in its interleaved mode with an 800MHz sampling clock 1 1PPS Site Timing Reference applied to the ADC's SYNC IN Is it possible, using the four SYNC outputs of the ADC block, to ascertain which of the 8 samples presented during a logic clock cycle was most closely aligned with an in-coming 1PPS signal? Randy
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
On Thu, 4 Mar 2010, David MacMahon wrote: On Mar 4, 2010, at 11:52 , Paul Demorest wrote: I have to ask.. if all four syncs are the same, why are there four of them? ;) Just to clarify, they represent the same signal sampled at (i.e. registered using) four different phases (0/90/180/270) of the FPGA clock. Or did you already know that and were just needling Dan? :-) No, I really was curious why there would be four copies of the same thing! To summarize your explanation Dave, the four syncs really do represent the input sync sampled on each of 4 ADC clocks. But there is also some uncertainty after each power-up that means sync0 doesn't necessarily match up with data0, etc. Is that right? -Paul
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
Hi, Paul, On Mar 4, 2010, at 12:46 , Paul Demorest wrote: Or did you already know that and were just needling Dan? :-) No, I really was curious why there would be four copies of the same thing! I notice that you didn't deny that you were needling Dan. :-) To summarize your explanation Dave, the four syncs really do represent the input sync sampled on each of 4 ADC clocks. Well, to be precise, the four syncs represent the input sync sampled at four times the FPGA clock rate with all the jitter of the FPGA's DCM. But there is also some uncertainty after each power-up that means sync0 doesn't necessarily match up with data0, etc. This is exactly my understanding. Furthermore, the sync signal will typically be longer than one ADC sample cycle, so one needs to be careful about detecting the edge of the sync signal. This can get a little tricky with 4 parallel inputs. The ATA DDC just uses sync0 to synchronize the internal 1 PPS with the external 1 PPS. Any slop/ imprecision there gets taken out in calibration. Is that right? It's right in that it captures my explanation, but you'll notice I didn't claim my explanation is correct (though I think it is). :-) Dave
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
Hi, Jason, On Mar 4, 2010, at 13:18 , Jason Manley wrote: And this mostly does work, however, it breaks when using two ADCs on one board, because the sample clock for the second ADC's 1PPS is derived from the first ADC, not from that second ADC. I think the ibob startup software expends to a non-trivial amount of effort to ensure that the two ADC output clocks are in phase, so sampling ADC1's sync using a clock derived from ADC0's output clock shouldn't be much worse than using a clock derived from ADC1's clock. Where it does break down, I think, is across multiple ibobs. Another small detail is that the sync signal passes straight from the input sma connector to the FPGA (possibly being buffered in between), but I don't know whether it is delayed any in the FPGA (i.e. in the yellow block) to compensate for the pipeline delay of the ADC. Dave
[casper] Where is sshd?
Hi, Excitement with our very first ROACH/CASPER experience today when we fired up our new roach board. However, all getting-started tutorials I find indicate sshd should be running but I can't find it anywhere. Port 22 doesn't answer, no sshd running, no results for busybox find -name *ssh*. Do I need a different linux image? Current: Linux version 2.6.25-svn2338 (d...@lappy) (gcc version 4.0.0 (DENX ELDK 4.0 4.0. 0)) #111 Tue Oct 6 14:24:10 SAST 2009 Thanks, Steve
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
hi jason, dave and paul, regarding syncing up one or two adc's with 1 PPS: at boot up, the iADC (also called ADC2x1000-8), yellow block software/gateware aligns two adc's that are plugged into roach or ibob. the code does this by sampling the relative phase of the two clocks that emerge from each adc (these clocks are generated internally in each adc, by dividing the sample clock by four); the code resets one of the adc's until the two adc clocks are lined up in phase. although it might be possible to crudely recover the phase of the 1PPS signal by looking at all four sync pulses, in practice, as you guys point out, it's better to calibrate this delay, best by looking at a known source on the sky, or possibly digitizing the 1 PPS by coupling it into the analog input and then finding it's edge in software using lots of samples. this is should be done everytime the system is powered up. dan On 3/4/2010 1:18 PM, Jason Manley wrote: The four sync lines are supposed to represent the ADC sample where the sync was input (as Paul suggests). And this mostly does work, however, it breaks when using two ADCs on one board, because the sample clock for the second ADC's 1PPS is derived from the first ADC, not from that second ADC. This is then a problem for correlator applications where you have two ADCs in one IBOB where you can have an unknown phase on that second ADC. In reality, most people just OR the four sync outputs together, resulting in an unknown phase difference which gets calibrated-out as Dave suggests. Jason On 04 Mar 2010, at 12:46, Paul Demorest wrote: On Thu, 4 Mar 2010, David MacMahon wrote: On Mar 4, 2010, at 11:52 , Paul Demorest wrote: I have to ask.. if all four syncs are the same, why are there four of them? ;) Just to clarify, they represent the same signal sampled at (i.e. registered using) four different phases (0/90/180/270) of the FPGA clock. Or did you already know that and were just needling Dan? :-) No, I really was curious why there would be four copies of the same thing! To summarize your explanation Dave, the four syncs really do represent the input sync sampled on each of 4 ADC clocks. But there is also some uncertainty after each power-up that means sync0 doesn't necessarily match up with data0, etc. Is that right? -Paul
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
Dan and everyone, Thanks for the info, this discussion has been helpful. To put this all into context, our application is a single-dish pulsar backend, not a correlator. This means that absolute time (ideally at the few ns level) is important, which is why we're talking about all this in the first place. It also means we can't calibrate these delays astronomically. Probably injecting a calibration signal with known phase is the way to go. -Paul On Thu, 4 Mar 2010, Dan Werthimer wrote: hi jason, dave and paul, regarding syncing up one or two adc's with 1 PPS: at boot up, the iADC (also called ADC2x1000-8), yellow block software/gateware aligns two adc's that are plugged into roach or ibob. the code does this by sampling the relative phase of the two clocks that emerge from each adc (these clocks are generated internally in each adc, by dividing the sample clock by four); the code resets one of the adc's until the two adc clocks are lined up in phase. although it might be possible to crudely recover the phase of the 1PPS signal by looking at all four sync pulses, in practice, as you guys point out, it's better to calibrate this delay, best by looking at a known source on the sky, or possibly digitizing the 1 PPS by coupling it into the analog input and then finding it's edge in software using lots of samples. this is should be done everytime the system is powered up. dan On 3/4/2010 1:18 PM, Jason Manley wrote: The four sync lines are supposed to represent the ADC sample where the sync was input (as Paul suggests). And this mostly does work, however, it breaks when using two ADCs on one board, because the sample clock for the second ADC's 1PPS is derived from the first ADC, not from that second ADC. This is then a problem for correlator applications where you have two ADCs in one IBOB where you can have an unknown phase on that second ADC. In reality, most people just OR the four sync outputs together, resulting in an unknown phase difference which gets calibrated-out as Dave suggests. Jason On 04 Mar 2010, at 12:46, Paul Demorest wrote: On Thu, 4 Mar 2010, David MacMahon wrote: On Mar 4, 2010, at 11:52 , Paul Demorest wrote: I have to ask.. if all four syncs are the same, why are there four of them? ;) Just to clarify, they represent the same signal sampled at (i.e. registered using) four different phases (0/90/180/270) of the FPGA clock. Or did you already know that and were just needling Dan? :-) No, I really was curious why there would be four copies of the same thing! To summarize your explanation Dave, the four syncs really do represent the input sync sampled on each of 4 ADC clocks. But there is also some uncertainty after each power-up that means sync0 doesn't necessarily match up with data0, etc. Is that right? -Paul
Re: [casper] Where is sshd?
Hi Steve, On Debian/Ubuntu 'sshd' is at /usr/sbin/sshd, but I would use the script /etc/init.d/ssh, which should be started at boot. If not you can try restarting it yourself: /etc/init.d/ssh stop | start | restart Mark On Thu, Mar 4, 2010 at 1:48 PM, Steve Maher stephen.f.ma...@nasa.govwrote: Hi, Excitement with our very first ROACH/CASPER experience today when we fired up our new roach board. However, all getting-started tutorials I find indicate sshd should be running but I can't find it anywhere. Port 22 doesn't answer, no sshd running, no results for busybox find -name *ssh*. Do I need a different linux image? Current: Linux version 2.6.25-svn2338 (d...@lappy) (gcc version 4.0.0 (DENX ELDK 4.0 4.0. 0)) #111 Tue Oct 6 14:24:10 SAST 2009 Thanks, Steve
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
Hi, Jason, On Mar 4, 2010, at 14:32 , Jason Manley wrote: On 04 Mar 2010, at 13:49, Dan Werthimer wrote: the code resets one of the adc's until the two adc clocks are lined up in phase. For future reference, this code gives up trying to sync after a while. I can demonstrate systems where it doesn't succeed before the timeout and then gives up. It occurs at PAPER-GB8 regularly. We've seen this problem at the ATA, too. In our case it turned out to be due to differences in the ADC clock cables, but not physical length so much as cable materials (presumably resulting is propagation delay or phase differences). It turns out that the phase alignment test in the ibob software imposes a fairly stringent requirement on the phase alignment of the two ADC input clocks. Here's an excerpt from a 2.5 year old message about this (from the ATA where Fs is 838.8608 MHz)... On Jul 18, 2007, at 0:09 , David MacMahon wrote: Here are the details on the ADC clock phase-up process (Fs is sampling frequency, i.e. 100*2^23 Hz == 838.8608 MHz)... The adc initialization software measures the relative phase of the two incoming clocks (Fs/4). This is a rather complicated process that I would rather not explain the inner working of here. Apparently this measurement is made with +/- 5% precision. If the measured relative phase is greater/less than +/- 20 degrees (at the FPGA clock frequency, Fs/4), then the clocks are deemed to be out of phase, the ADCs are reset, and the measurement process starts again. If the two clocks are never deemed to be in phase, i.e. within +/- 21 degrees (20 degrees plus 5% == 21 degrees) after 50 attempts, the software finally gives up. 20 degrees at Fs/4 is 80 degrees at Fs, so if the two ADC clocks are more than 80 degrees (at Fs) apart (give or take a few degrees), then I think they will never phase up. 80 degrees at Fs is only 265 ps! The problem at the ATA was tracked down by Billy Barrott... On Jul 18, 2007, at 14:03 , William Barott wrote: I have traced the problem to the cables themselves. Three different stocks of cables were used in the construction of the RFCB 800 MHz clock feed lines: The bulk is LMR-100A from the same spool. We also have RG-174U and High-Quality RG-174A/U. The RG-174A/U (1 found in system) measures about 35 degrees out of phase with the LMR-100A. The RG-174/U (4 found in system) measures about 110 degrees out of phase with the LMR-100A. It was these RG-174/U cables that were associated with wildly-off phases and non-syncing iBobs in both chassis. In another message, Billy reported that all these cables were length matched to within 1 cm. I believe (but am not 100% sure) that the problem was using mismatched cable types for an ibob's two ADC clocks rather than one cable type being unusable. Back to the present... On Mar 4, 2010, at 14:32 , Jason Manley wrote: I think Paul's on the right track: since you cannot calibrate off the sky, injecting a reference for the calibration is their most reliable option. Agreed. Dave
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
Hi, Dan, On Mar 4, 2010, at 15:15 , Dan Werthimer wrote: i don't think there will be a power up divide by four ambiguity if you only have one adc board I think there will be a divide by four ambiguity even for one ADC board. Assuming the ADC sample clock is synchronous to 1 PPS, there is no way to tell (without somehow calibrating) which of the four ADC CLK OUT signals the FPGA will get as its input clock. For this input... 1 PPS 0... ADC CLK IN 010101010101010101010... You will get one of these outputs... ADC CLK OUT 1... ADC CLK OUT 11100... ADC CLK OUT 0... ADC CLK OUT 00011... ...but how can you tell which one you've got? Dave
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
Thanks, Matt, Just to be clear, if a different observatory used RG-174U only then could this same problem be caused by some mistakenly installed Times Microwave LMR-100A? I wouldn't want the take away from this thread to be that one kind of cable is evil and another kind of cable is blessed (unless that is in fact true). Thanks again, Dave On Mar 4, 2010, at 15:52 , Matt Dexter wrote: yes it was 2 cable types. we use Times Microwave LMR-100A only. The problem Billy found and fixed was due to some mistakenly installed RG-174U. (see the same email discussion of 2007jul18) On Thu, 4 Mar 2010, David MacMahon wrote: Hi, Jason, On Mar 4, 2010, at 14:32 , Jason Manley wrote: On 04 Mar 2010, at 13:49, Dan Werthimer wrote: the code resets one of the adc's until the two adc clocks are lined up in phase. For future reference, this code gives up trying to sync after a while. I can demonstrate systems where it doesn't succeed before the timeout and then gives up. It occurs at PAPER-GB8 regularly. We've seen this problem at the ATA, too. In our case it turned out to be due to differences in the ADC clock cables, but not physical length so much as cable materials (presumably resulting is propagation delay or phase differences). It turns out that the phase alignment test in the ibob software imposes a fairly stringent requirement on the phase alignment of the two ADC input clocks. Here's an excerpt from a 2.5 year old message about this (from the ATA where Fs is 838.8608 MHz)... On Jul 18, 2007, at 0:09 , David MacMahon wrote: Here are the details on the ADC clock phase-up process (Fs is sampling frequency, i.e. 100*2^23 Hz == 838.8608 MHz)... The adc initialization software measures the relative phase of the two incoming clocks (Fs/4). This is a rather complicated process that I would rather not explain the inner working of here. Apparently this measurement is made with +/- 5% precision. If the measured relative phase is greater/less than +/- 20 degrees (at the FPGA clock frequency, Fs/4), then the clocks are deemed to be out of phase, the ADCs are reset, and the measurement process starts again. If the two clocks are never deemed to be in phase, i.e. within +/- 21 degrees (20 degrees plus 5% == 21 degrees) after 50 attempts, the software finally gives up. 20 degrees at Fs/4 is 80 degrees at Fs, so if the two ADC clocks are more than 80 degrees (at Fs) apart (give or take a few degrees), then I think they will never phase up. 80 degrees at Fs is only 265 ps! The problem at the ATA was tracked down by Billy Barrott... On Jul 18, 2007, at 14:03 , William Barott wrote: I have traced the problem to the cables themselves. Three different stocks of cables were used in the construction of the RFCB 800 MHz clock feed lines: The bulk is LMR-100A from the same spool. We also have RG-174U and High-Quality RG-174A/U. The RG-174A/U (1 found in system) measures about 35 degrees out of phase with the LMR-100A. The RG-174/U (4 found in system) measures about 110 degrees out of phase with the LMR-100A. It was these RG-174/U cables that were associated with wildly-off phases and non-syncing iBobs in both chassis. In another message, Billy reported that all these cables were length matched to within 1 cm. I believe (but am not 100% sure) that the problem was using mismatched cable types for an ibob's two ADC clocks rather than one cable type being unusable. Back to the present... On Mar 4, 2010, at 14:32 , Jason Manley wrote: I think Paul's on the right track: since you cannot calibrate off the sky, injecting a reference for the calibration is their most reliable option. Agreed. Dave
Re: [casper] Correlating iADC SYNC Outputs to Individual Samples...
This works clock source - N way splitter - cable A - iADC input A - cable B - iADC input B Cables A and B must have the same electrical delay for the clock signal to be distributed. This is pretty easy to do if use same cable type and physical length and routing (bends and such) for each of the different ADC clock inputs. RG-174U can be used (when complete set) but it is pretty low end cable. Times Microwave LMR-100A is pretty nice cable. better by ~ 12dB/100ft @ 1 GHz better shielding 100% vas 88% and spec 90dB vs not specified. etc. Not blessed; just advised. On Thu, 4 Mar 2010, David MacMahon wrote: Thanks, Matt, Just to be clear, if a different observatory used RG-174U only then could this same problem be caused by some mistakenly installed Times Microwave LMR-100A? I wouldn't want the take away from this thread to be that one kind of cable is evil and another kind of cable is blessed (unless that is in fact true). Thanks again, Dave On Mar 4, 2010, at 15:52 , Matt Dexter wrote: yes it was 2 cable types. we use Times Microwave LMR-100A only. The problem Billy found and fixed was due to some mistakenly installed RG-174U. (see the same email discussion of 2007jul18) On Thu, 4 Mar 2010, David MacMahon wrote: Hi, Jason, On Mar 4, 2010, at 14:32 , Jason Manley wrote: On 04 Mar 2010, at 13:49, Dan Werthimer wrote: the code resets one of the adc's until the two adc clocks are lined up in phase. For future reference, this code gives up trying to sync after a while. I can demonstrate systems where it doesn't succeed before the timeout and then gives up. It occurs at PAPER-GB8 regularly. We've seen this problem at the ATA, too. In our case it turned out to be due to differences in the ADC clock cables, but not physical length so much as cable materials (presumably resulting is propagation delay or phase differences). It turns out that the phase alignment test in the ibob software imposes a fairly stringent requirement on the phase alignment of the two ADC input clocks. Here's an excerpt from a 2.5 year old message about this (from the ATA where Fs is 838.8608 MHz)... On Jul 18, 2007, at 0:09 , David MacMahon wrote: Here are the details on the ADC clock phase-up process (Fs is sampling frequency, i.e. 100*2^23 Hz == 838.8608 MHz)... The adc initialization software measures the relative phase of the two incoming clocks (Fs/4). This is a rather complicated process that I would rather not explain the inner working of here. Apparently this measurement is made with +/- 5% precision. If the measured relative phase is greater/less than +/- 20 degrees (at the FPGA clock frequency, Fs/4), then the clocks are deemed to be out of phase, the ADCs are reset, and the measurement process starts again. If the two clocks are never deemed to be in phase, i.e. within +/- 21 degrees (20 degrees plus 5% == 21 degrees) after 50 attempts, the software finally gives up. 20 degrees at Fs/4 is 80 degrees at Fs, so if the two ADC clocks are more than 80 degrees (at Fs) apart (give or take a few degrees), then I think they will never phase up. 80 degrees at Fs is only 265 ps! The problem at the ATA was tracked down by Billy Barrott... On Jul 18, 2007, at 14:03 , William Barott wrote: I have traced the problem to the cables themselves. Three different stocks of cables were used in the construction of the RFCB 800 MHz clock feed lines: The bulk is LMR-100A from the same spool. We also have RG-174U and High-Quality RG-174A/U. The RG-174A/U (1 found in system) measures about 35 degrees out of phase with the LMR-100A. The RG-174/U (4 found in system) measures about 110 degrees out of phase with the LMR-100A. It was these RG-174/U cables that were associated with wildly-off phases and non-syncing iBobs in both chassis. In another message, Billy reported that all these cables were length matched to within 1 cm. I believe (but am not 100% sure) that the problem was using mismatched cable types for an ibob's two ADC clocks rather than one cable type being unusable. Back to the present... On Mar 4, 2010, at 14:32 , Jason Manley wrote: I think Paul's on the right track: since you cannot calibrate off the sky, injecting a reference for the calibration is their most reliable option. Agreed. Dave
