Re: [ARTIQ] FW: initial specification of the project

[Thomas Harty]

Great! Sounds like a plan.

Tom


From: Slichter, Daniel H. (Fed) [daniel.slich...@nist.gov]
Sent: 12 April 2016 16:40
To: Thomas Harty; artiq@lists.m-labs.hk
Subject: RE: FW: initial specification of the project

Hi Tom,
Responses inline below!  Agreed that we are pretty much on the same page 
regarding our end goals, and I am flexible in how they are achieved (backplane 
clocks would be great!) as long as we aren’t sacrificing performance in a 
substantial way.
Cheers,
Daniel

It looks like we agree that backplane clock distribution is the best way to go, 
so long as we can overcome the phase noise/drift concerns that we've discussed 
-- which isn't a given. We've probably taken this discussion about as far as we 
can without having test data to see how well we can get the backplane clock to 
work practice. We (Oxford) are happy to have a go at getting this data and, as 
I mentioned, are currently designing hardware to that end.  Once we've got 
designs ready for the test hardware, we'll post them here along with a rough 
description of the tests we plan to perform.

Great, I look forward to the results of your measurements.

If you have a couple of test boards up and running, would it be easy to take 
some data on the propagation delay stability of these DACs at the same time? 
This would be a really useful reference point for thinking about how stable the 
clock distribution network needs to be -- there isn't much point worrying about 
the temp. co. of clock buffers/PLLs if they are small compared with the DAC...

I have one test board right now, but can measure the channel-to-channel noise 
within a single card pretty easily.  If we get another board we can compare two 
chips to each other as well.  I’ll share the data once I have it, lots of items 
on the plate though so it may not be instant.


Out of curiosity, what is the plan for thermal management (e.g. the DACs put 
out a fair amount of heat)? Is this something you've thought about much yet?

The plan here is forced air cooling at crate level, which is part of the uTCA 
crate design.

We are only trying to send a relatively small volume of data, and have the 
option of using a multi GBPS link. This gives us a lot of options in terms of 
spreading the noise power out/minimising the power at our reference frequency. 
If we do this properly, the numbers actually don't look so bad and I think we 
may be able to get the cross-talk noise below the DAC's noise floor. We'll have 
a proper think about this once we've got our test hardware designed...

It’s not clear to me that we will only be sending a relatively small volume of 
data.  We may want to send waveform data to and from DACs, or potentially 
stream raw or minimally processed ADC data out to the MCH.  I think it is a 
mistake to rely on the notion that we will only be sending small amounts of 
data.  Likewise, with Greg’s proposed solution of silencing the high speed 
digital transmissions during experiments, I think we may need to pipeline 
things such that data is being fed in and/or out of the cards while an 
experiment is running.  Any design which doesn’t allow for this seems like a 
bad idea to me.


I'm not so sure about that... It's been my experience that if one wants a 
source that can be programmed to go from 100kHz to 20GHz and -130dBm to +20dBm 
with crazy resolution and accuracy then the Agilent synth is the way to go. 
However, for a fixed-frequency, stable oscillator they're actually not that 
great. e.g. it's not that hard to outperform an Agilent synth at a few GHz with 
a cheap PLL phase noise wise. When it comes to phase stability, bigger is 
generally worse. I suspect that a small, if needs be ovenized, PLL is actually 
a pretty good solution. But, maybe I'll take that back when our test boards 
arrive.

Sounds like waiting for the test boards to “recalibrate our BS detectors” is 
the thing to do here :)


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

Hi Tom,
Responses inline below!  Agreed that we are pretty much on the same page 
regarding our end goals, and I am flexible in how they are achieved (backplane 
clocks would be great!) as long as we aren't sacrificing performance in a 
substantial way.
Cheers,
Daniel

It looks like we agree that backplane clock distribution is the best way to go, 
so long as we can overcome the phase noise/drift concerns that we've discussed 
-- which isn't a given. We've probably taken this discussion about as far as we 
can without having test data to see how well we can get the backplane clock to 
work practice. We (Oxford) are happy to have a go at getting this data and, as 
I mentioned, are currently designing hardware to that end.  Once we've got 
designs ready for the test hardware, we'll post them here along with a rough 
description of the tests we plan to perform.

Great, I look forward to the results of your measurements.

If you have a couple of test boards up and running, would it be easy to take 
some data on the propagation delay stability of these DACs at the same time? 
This would be a really useful reference point for thinking about how stable the 
clock distribution network needs to be -- there isn't much point worrying about 
the temp. co. of clock buffers/PLLs if they are small compared with the DAC...

I have one test board right now, but can measure the channel-to-channel noise 
within a single card pretty easily.  If we get another board we can compare two 
chips to each other as well.  I'll share the data once I have it, lots of items 
on the plate though so it may not be instant.


Out of curiosity, what is the plan for thermal management (e.g. the DACs put 
out a fair amount of heat)? Is this something you've thought about much yet?

The plan here is forced air cooling at crate level, which is part of the uTCA 
crate design.

We are only trying to send a relatively small volume of data, and have the 
option of using a multi GBPS link. This gives us a lot of options in terms of 
spreading the noise power out/minimising the power at our reference frequency. 
If we do this properly, the numbers actually don't look so bad and I think we 
may be able to get the cross-talk noise below the DAC's noise floor. We'll have 
a proper think about this once we've got our test hardware designed...

It's not clear to me that we will only be sending a relatively small volume of 
data.  We may want to send waveform data to and from DACs, or potentially 
stream raw or minimally processed ADC data out to the MCH.  I think it is a 
mistake to rely on the notion that we will only be sending small amounts of 
data.  Likewise, with Greg's proposed solution of silencing the high speed 
digital transmissions during experiments, I think we may need to pipeline 
things such that data is being fed in and/or out of the cards while an 
experiment is running.  Any design which doesn't allow for this seems like a 
bad idea to me.


I'm not so sure about that... It's been my experience that if one wants a 
source that can be programmed to go from 100kHz to 20GHz and -130dBm to +20dBm 
with crazy resolution and accuracy then the Agilent synth is the way to go. 
However, for a fixed-frequency, stable oscillator they're actually not that 
great. e.g. it's not that hard to outperform an Agilent synth at a few GHz with 
a cheap PLL phase noise wise. When it comes to phase stability, bigger is 
generally worse. I suspect that a small, if needs be ovenized, PLL is actually 
a pretty good solution. But, maybe I'll take that back when our test boards 
arrive.

Sounds like waiting for the test boards to "recalibrate our BS detectors" is 
the thing to do here :)


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

 a 
high-frequency communication clock. This could be a strong reason to consider 
using GTX for backplane communication...

 

Indeed, in this way we could move the noise spectrum much higher, where it’s 
easier to filter it out.

Greg

 

 

 

From: kaspr...@gmail.com <mailto:kaspr...@gmail.com>  
[mailto:kaspr...@gmail.com] On Behalf Of Grzegorz Kasprowicz
Sent: 08 April 2016 11:34
To: Thomas Harty
Cc: Slichter, Daniel H. (Fed); artiq@lists.m-labs.hk 
<mailto:artiq@lists.m-labs.hk> 
Subject: Re: [ARTIQ] FW: initial specification of the project

 

Hi


tl;dr: distributing a reference clock at something like 10MHz-100MHz over the 
backplane, and generating the required DAC/upconverter clocks locally using 
VCOs seems to be by far the nicest, most scalable and flexible solution. There 
are, however, potential issues with phase drift/noise which could cause 
problems with this approach. However, I'm not convinced that there aren't 
equally difficult problems with any other approach. And, in any case, I'm 
optimistic that this won't be the case and the VCOs will be fine. We've started 
doing some tests on this and have more planned, and hope to have at least a 
partial answer soon.

1) You're right that for the most critical applications the single-loop PLL I 
described previously could start to become a limitation (although, we did gates 
with 99.% fidelity with a clock source that had a noise level pretty 
similar to the VCO I suggested). In practice, one would probably want to go for 
a 2-loop solution anyway to avoid having such a high divider ratio (240) in a 
single loop, which can lead to a significant noise contribution from the PFD 
(scales as 10log10(N) ).

That's why I proposed dual PLL solution. In this way we can either use 
backplane clock or switch to external DAC and RF clocks deliver via MMCX 
connectors. 


An example 2-loop PLL might be: begin by locking a 100MHz VCO to the 10MHz 
backplane-distributed reference using a lock bandwidth of ~1kHz. This loop 
would be designed to "clean up" the 10MHz reference clock by rejecting 
cross-talk, and needs to use a VCO with extremely good close-in phase noise. A 
choice of VC(S)O could be a Crystek CVSS-945 ($20 in single quantity from 
Digi-Key 
http://www.digikey.co.uk/product-detail/en/crystek-corporation/CVSS-945-100.000/744-1612-ND/3711616
 although, there are fancier options if one really wants to push things). One 
would then lock the 2.4GHz VCO to the 100MHz with a loop bandwidth of ~500kHz.

For jitter cleaning we use good VCXOs from Crystek. You can look at FMC ADC 
250M ch4 design. THere is Crstek VCXO synchronized by backplane clock and in 
parallel, possibility to deliver the clock from MMCX to ADCs via passivle clock 
splitter. 


In this system, the phase noise should be within ~20dB of the Wenzel 1GHz 
oscillator at all frequencies, with the VCO and Wenzel oscillator being 
comparable by ~1MHz. This does assume that the PLL doesn't degrade the VCO 
noise significantly, which is something we plan to test.

well, you don't have to pass it via PLL chip. YOu can split the VCO output 
passively. 


While it's more engineering work to design and test this system, it's work that 
can be done once on a prototype board and then copied into the final design. 
I'd generally prefer that to requiring each user to sort out their own external 
oscillator, power splitters, amps, cables, etc.

Assuming we can get close to the VCO data sheet noise, I can't see the 
difference between the VCOs and Wenzel oscillator being significant, 
particularly given that...

2) Distributing the Wenzel oscillator's signal without degrading its phase 
noise is going to be pretty tough. For example, the best clock distribution IC 
that I know of is the HMC830. Looking at figure 22 of its data sheet suggests 
that even this IC will degrade the Wenzel oscillator's phase noise 
considerably. In general, clock switches tend to be far worse than this (which 
is one reason that I'd really rather not put a switch in the clock path to 
select between the front-panel and backplane clocks). I wouldn't trust a 
switch/distribution IC to work at those phase noise levels unless its data 
sheet explicitly gave a phase noise plot. 

Did you think about passive splitting? 


A similar remark goes for the ERA-4SM amplifier you suggest using on the DACs 
output. For close-in phase noise, amplifier noise figure is meaningless. In 
general, for work where phase noise really matters, no amplifiers should be 
used unless their datasheet actually gives a phase noise plot -- and I 
certainly wouldn't trust a cheap MCL "general purpose" Darlington amp with my 
Wenzel oscillator without some careful measurements!

3) The DAC's phase noise contribution is going to be worse than the locked 
VCO's noise (compare the 100MHz trace on figure 33 of the AD9154 data sheet to 
the VCO datasheets) for all frequencies above 1kHz. Below 1kHz, the locked 
VCO's noise should be

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Hi

In MTCA the clock signal is distributed from dedicated Tongue 2 module on MCH.

There are 3 clock signals and in new implementation of MTCA also 8 MLVDS lines 
that can be used as reset or interlock.

 

We can route the external DAC clock to the clock distributor as well which 
would synchronously apply reset pulse to all AMCs, exactly in the same ways as 
in case of clock distribution over the backplane .

 

Best  regards,

Greg

 

 

From: Thomas Harty [mailto:thomas.ha...@physics.ox.ac.uk] 
Sent: Friday, April 08, 2016 8:04 PM
To: Grzegorz Kasprowicz <g.kasprow...@elka.pw.edu.pl>
Cc: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>; artiq@lists.m-labs.hk
Subject: RE: [ARTIQ] FW: initial specification of the project

 

Greg,

 

 

In case of backplane, there is available WR-compatible version that I mentioned 
in the datasheet I atached.

It has all the clocks equalized between MCH and AMC with 20ps accuracy.

So providing that we use identical AMCs it should do the job.

But anyway, maybe we should provide option to calibrate the phase delay on each 
FMC ?

Many PLL chips have such feature. 

 

 

I think we’re talking about different things here (sorry if I wasn’t clear). 
I’m considering how we will distribute a t=0 signal (i.e. how we reset all 
FPGA/DAC state machines the same time). The model I have in mind is something 
like this:

 

1)  We distribute a common system clock (e.g. 10MHz) down the backplane to 
each AMC. We would like to select a rising edge of this clock to be t=0, so 
that all FPGA/DAC state machines are reset exactly on this edge of the system 
clock. We achieve this by distributing a RST signal to each AMC. RST is 
intended to be sampled from the system clock.

2)  I assume that the FPGA clocks are generated from the 10MHz by the FPGAs 
by integer-N multiplication, making resetting their state-machines relatively 
straight forwards

3)  If the DAC clock is generated on the AMC using an integer-N PLL, so 
that it is edge-aligned with the system clock, the it is also easy to determine 
which edge of the DAC clock corresponds to t=0

4)  However, if the DAC clock is generated externally and fed into the AMC 
via the front panel then it won’t be edge-aligned with the system clock. As a 
result, there is an ambiguity of 1 DAC clock cycle over when t=0 occurs. As a 
result, the delay between t=0 and the moment the DAC resets itself can vary 
randomly each time the hardware is reset/resynchronised. We are keen to avoid 
this non-deterministic latency if possible.

 

How do you plan to do this in your system?

 

Best wishes,

 

Tom

 

 

From: kaspr...@gmail.com <mailto:kaspr...@gmail.com>  
[mailto:kaspr...@gmail.com] On Behalf Of Grzegorz Kasprowicz
Sent: 08 April 2016 11:34
To: Thomas Harty
Cc: Slichter, Daniel H. (Fed); artiq@lists.m-labs.hk 
<mailto:artiq@lists.m-labs.hk> 
Subject: Re: [ARTIQ] FW: initial specification of the project

 

Hi


tl;dr: distributing a reference clock at something like 10MHz-100MHz over the 
backplane, and generating the required DAC/upconverter clocks locally using 
VCOs seems to be by far the nicest, most scalable and flexible solution. There 
are, however, potential issues with phase drift/noise which could cause 
problems with this approach. However, I'm not convinced that there aren't 
equally difficult problems with any other approach. And, in any case, I'm 
optimistic that this won't be the case and the VCOs will be fine. We've started 
doing some tests on this and have more planned, and hope to have at least a 
partial answer soon.

1) You're right that for the most critical applications the single-loop PLL I 
described previously could start to become a limitation (although, we did gates 
with 99.% fidelity with a clock source that had a noise level pretty 
similar to the VCO I suggested). In practice, one would probably want to go for 
a 2-loop solution anyway to avoid having such a high divider ratio (240) in a 
single loop, which can lead to a significant noise contribution from the PFD 
(scales as 10log10(N) ).

That's why I proposed dual PLL solution. In this way we can either use 
backplane clock or switch to external DAC and RF clocks deliver via MMCX 
connectors. 


An example 2-loop PLL might be: begin by locking a 100MHz VCO to the 10MHz 
backplane-distributed reference using a lock bandwidth of ~1kHz. This loop 
would be designed to "clean up" the 10MHz reference clock by rejecting 
cross-talk, and needs to use a VCO with extremely good close-in phase noise. A 
choice of VC(S)O could be a Crystek CVSS-945 ($20 in single quantity from 
Digi-Key 
http://www.digikey.co.uk/product-detail/en/crystek-corporation/CVSS-945-100.000/744-1612-ND/3711616
 although, there are fancier options if one really wants to push things). One 
would then lock the 2.4GHz VCO to the 100MHz with a loop bandwidth of ~500kHz.

For jitter cleaning we use good VCXOs from Crystek. You ca

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Tuesday, 12 April 2016 1:55:45 AM HKT Slichter, Daniel H. (Fed) wrote:
> On the DSP/"Sayma" boards, a hard SoC could have use for reducing the time
> it takes to perform feedback calculations (e.g. shifts of output signal
> frequency based on ADC readings).  

Can't this be done on the Metlino (where it *might* make more sense to have a 
MPSoC) or with gateware? What are those feedback calculations exactly?

> My comment in the previous email about "users" wanting this also pertains to
> the idea that people not using ARTIQ might want to buy these boards and use
> them for their own purposes.  For many of them, having a hard SoC on board
> may make it faster/easier for them to get the boards doing what they
> want.  Again, it's not totally necessary to have this but would enable more
> applications and a broader group of potentially interested customers.  

There is no shortage of general purpose FPGA boards for Zynq users. Do we 
really want to compete in this market?

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> On Saturday, 9 April 2016 6:10:36 PM HKT Grzegorz Kasprowicz wrote:
> > Why do you think that CPUs have negative value? You don't have to use
> > them at all.
> 
> I already explained that the MPSoC has to be dealt with and cannot be
> completely ignored. If we have two SDRAM systems, maybe we can to a
> reasonable extent, but then it does complicate the board.

On the DSP/"Sayma" boards, a hard SoC could have use for reducing the time it 
takes to perform feedback calculations (e.g. shifts of output signal frequency 
based on ADC readings).  

My comment in the previous email about "users" wanting this also pertains to 
the idea that people not using ARTIQ might want to buy these boards and use 
them for their own purposes.  For many of them, having a hard SoC on board may 
make it faster/easier for them to get the boards doing what they want.  Again, 
it's not totally necessary to have this but would enable more applications and 
a broader group of potentially interested customers.  
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

> > Why do you think that CPUs have negative value? You don't have to use
> them
> > at all.
>
> I already explained that the MPSoC has to be dealt with and cannot be
> completely ignored. If we have two SDRAM systems, maybe we can to a
> reasonable
> extent, but then it does complicate the board.
>
I clearly see your point. The only reason to use here the SoC is its price
and design reuse.
I can prepare bootloader configuration that will keep the CPUs on hold and
dedicate all the SDRAM controller to your needs. In this way you won't need
to waste any logic resources and forget about the CPUs.


>
> > Anyway, to build MCH, we need first tongue. We can either build it or buy
> > it.  Buying is the simplest option at the moment and we get gigabit
> switch
> > and management.
>
> > On the MCH Tongue 3, there is single port Ethernet connection that can be
> > used for slow control and remote upgrade.
>
> We do not need Ethernet (nor IPMI in fact) inside the crates. The only
> part of
> the system that has Ethernet is between the control PC and the root master
> (Metlino); this can go over one of its regular SFPs.
>
this can also go over embedded MAC, CPUs and SFPs as well. It's easy to use
this MAC in bare-metal way. You will save logic resources. This MAC is
working well, I managed to get 900Mbit/s under TCP/IP. And you get the
IEEE1588 for free.
And you have 4 such MACs.
I don't know how many logic resources do you plan to use in Metilno, but at
some point some non time-critical stuff can be offloaded by the CPUs.

>
> > 6. MCH Tongue 2 clock distributor(Metlino). We can use NAT clock module
> with
> > two low jitter crossbar switches.
>
> Why do we need a fancy clock module? Data transfer certainly does not
> require
> one. Are you reconsidering your position that the AMC backplane should not
> be
> used for DAC clocks?
>

Well, during last discussion there is an idea to use these clocks for less
demanding applications.
Moreover such clocks + MLVDS lines make sub-ns synchronisation of all AMC
cards trivial.

>
> Also, (5 data/clock signals)*(10 AMCs)*(2 for differential) = 100 pins,
> plus
> power supply. Do we really need all those "tongues"?
>
This is good question:)
Tongue 3/4 has pins for communication - x4 fat pipes between AMCs and MCH.
Tongue 2 has only clock pins -  3 sets of independent clocks to all AMCs
Tongue 1 has supplies, I2Cs and x1 Ethernet ports to all AMCs.

How do you count these 5 data/clock signals ?



> Sébastien
>
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Saturday, 9 April 2016 6:10:36 PM HKT Grzegorz Kasprowicz wrote:
> Why do you think that CPUs have negative value? You don't have to use them
> at all. 

I already explained that the MPSoC has to be dealt with and cannot be 
completely ignored. If we have two SDRAM systems, maybe we can to a reasonable 
extent, but then it does complicate the board.

> Anyway, to build MCH, we need first tongue. We can either build it or buy
> it.  Buying is the simplest option at the moment and we get gigabit switch
> and management.

> On the MCH Tongue 3, there is single port Ethernet connection that can be
> used for slow control and remote upgrade.

We do not need Ethernet (nor IPMI in fact) inside the crates. The only part of 
the system that has Ethernet is between the control PC and the root master 
(Metlino); this can go over one of its regular SFPs.

> 6. MCH Tongue 2 clock distributor(Metlino). We can use NAT clock module with
> two low jitter crossbar switches.

Why do we need a fancy clock module? Data transfer certainly does not require 
one. Are you reconsidering your position that the AMC backplane should not be 
used for DAC clocks?

Also, (5 data/clock signals)*(10 AMCs)*(2 for differential) = 100 pins, plus 
power supply. Do we really need all those "tongues"?

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Thomas Harty]

Dear Greg,

Thank you for the detailed Feedback.

The key question here seems to be: will the cross-talk between the clock line 
and the digital communications lines be sufficiently bad to prevent one from 
recovering a low-noise analog clock from a reference signal sent over the 
backplane? If it is then we will want an external clock, otherwise we won’t.

It seems to me that the answer is likely to be that, while there will be a 
significant level of cross-talk -- particularly at higher frequencies--, it 
shouldn’t be sufficiently bad to prevent one from recovering a clean clock. Let 
me make the following naïve argument, and maybe you can point out where I am 
wrong

In short, the argument will be this: we will model the digital communication as 
a constant stream of random data, with a white noise power spectral density 
(PSD). As a result, the effect of cross-talk will be to increase the phase 
noise of the 10MHz analog clock. We will estimate this phase noise contribution 
and compare it to the phase noise of the DAC chip to see if it’s significant. 
In detail:


1)Let’s assume that our reference clock frequency is 10MHz

2)Assume that on the AMC/FMC board we lock a 100MHz VCXO to the 10MHz with 
a 1kHz loop bandwidth

3)We are only sensitive to cross-talk-induced noise at offset frequencies 
of <~1kHz from the 10MHz reference, since higher offset frequencies are 
attenuated by the loop.

4)To avoid degrading the DAC’s performance, we would thus like the noise on 
the 10MHz signal to be lower than -155dBc/Hz, corresponding to the DAC’s noise 
@ 1kHz offset frequency, scaled to a 10MHz carrier.

5)Let’s assume that we do all our backplane communication at 2GSPS and that 
we send a constant stream of data.

6)Assume that all communication data is properly scrambled, so that its 
power spectrum is approximately white.

7)Further, make the approximation that all power from the digital 
communication lines is contained within a bandwidth between DC and 1GHz (the 
Nyquist frequency for the digital communications clock).

8)From (6) and (7) the power spectral density (PSD) for the backplane data 
is roughly -90dBc/Hz, taking our reference (0dBc) to be the power in the 10MHz 
reference clock signal (i.e. we assume the same power in both signals, but that 
the digital communication power is spread over a wide bandwidth due to 
scrambling)

9)Assuming a DC-balanced line encoding, the PSD for the communication data 
will actually roll-off at low frequencies. A quick-and-dirty MatLab simulation 
based on a PRBS of data @2GHz, encoded using 8b10b, suggests that the roll-off 
should be >30dB @ 10MHz

10)  As a result, so long as the cross-talk between the clock line and the 
digital communication lines (i.e. the S12 between the clock line and the data 
lines) for a 10MHz signal  is <~35dB (which would be extremely bad for a 10MHz 
signal!) no degradation of the DAC’s performance is expected as a result of 
cross-talk-related noise. In a well-designed backplane, the cross-talk should 
be far less than this @10MHz (see, e.g. 
http://www.harting-usa.com/fileadmin/harting/documents/public/40GBWhitePaper_01.pdf<https://owa.nexus.ox.ac.uk/owa/redir.aspx?SURL=d061emLP9wo2pUOgjIDJYtXdN1vWKNsgOUYamVWKKm27D6Jr01_TCGgAdAB0AHAAOgAvAC8AdwB3AHcALgBoAGEAcgB0AGkAbgBnAC0AdQBzAGEALgBjAG8AbQAvAGYAaQBsAGUAYQBkAG0AaQBuAC8AaABhAHIAdABpAG4AZwAvAGQAbwBjAHUAbQBlAG4AdABzAC8AcAB1AGIAbABpAGMALwA0ADAARwBCAFcAaABpAHQAZQBQAGEAcABlAHIAXwAwADEALgBwAGQAZgA.=http%3a%2f%2fwww.harting-usa.com%2ffileadmin%2fharting%2fdocuments%2fpublic%2f40GBWhitePaper_01.pdf>
 where the cross-talk was measured to be <60dB at 10MHz). Note that we are only 
sensitive to the cross-talk at frequencies close to 10MHz, since higher 
frequencies (e.g. 1GHz) can easily be filtered off. Thus, the fact that the 
cross-talk will be worse at higher frequencies isn’t a problem for us.

11)  Note that, in practice, most of the time the signal would actually just be 
a repeated ‘,’ (idle) as the volume of data is expected to be relatively low. 
The spectrum for the idle signal has no power at 10MHz at all (it’s periodic), 
so point (8) is actually probably very pessimistic

This is obviously just a quick order-of-magnitude calculation, but it does 
suggest that we should be able to get away with distributing the analog clock 
over the backplane if we’re careful with how we do our digital communication. 
As I said, we’re currently building hardware to test how well this works in 
practice though…

Note that to minimize EMI/cross-talk, it’s likely to be important to use a 
high-frequency communication clock. This could be a strong reason to consider 
using GTX for backplane communication...
Best,

Tom



From: kaspr...@gmail.com [mailto:kaspr...@gmail.com] On Behalf Of Grzegorz 
Kasprowicz
Sent: 08 April 2016 11:34
To: Thomas Harty
Cc: Slichter, Daniel H. (Fed); artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specificat

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

One more thing
In case of ZynQ, instead of ZU11, we have smaller GTX count version - ZU7 in
the same package.
Its cost is about 700-750$.
So we can install ZU7 by default, and when necessary, upgrade it to ZU11.
Greg

-Original Message-
From: Grzegorz Kasprowicz [mailto:gkasp...@elka.pw.edu.pl] 
Sent: Friday, April 08, 2016 2:00 PM
To: 'Sébastien Bourdeauducq' <s...@m-labs.hk>
Cc: 'Grzegorz Kasprowicz' <kaspr...@gmail.com>; 'Slichter, Daniel H. (Fed)'
<daniel.slich...@nist.gov>; artiq@lists.m-labs.hk
Subject: RE: [ARTIQ] FW: initial specification of the project

Well, I compared highest speed grades (-3) in large packages :)
XC7K325T-3FFG900E  costs more than ZU11 in similar speed grade.
Personally, I like this disgusting ZynQ stuff. I didn't have any special
troubles to make it running. And it gives very handy feature of FPGA upgrade
over Ethernet :) In this way you can keep all FPGA bit streams on single nfs
server and load them at the startup.
THE CPU and FPGA are independent, so you can boot CPU without the bitstream,
connect to nfs, grab the bitteam and load it. So you can treat it as CPU
which simply loads the FPGA and from programmable logic you treat it as
ordinary Kintex device.
And you get bunch of IOs for free which can be controlled over Ethernet.
In case of complex system it is very desirable because you keep all the
settings in single location and to upgrade it don't have to walk with
programmer from board to board.
We have a system which consists of 22 FPGAs in single box and all FPGA files
are kept on single USB drive. The system is installed in UK and upgraded
once a month and it is real pleasure to work in such manner.
Another system is CMS muon trigger which consists of thousands of FPGAs and
they all are loaded at the startup. It's much easier to keep control on
bitstream versions of individual chips.
The only alternative update channel in MTCA is either JTAG or I2C.
With I2C you need about 30hours to programm the config FLASH. In case of
JTAG, not every MTCA crate has JSM.
Greg


-Original Message-
From: Sébastien Bourdeauducq [mailto:s...@m-labs.hk] 
Sent: Friday, April 08, 2016 12:13 PM
To: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>
Cc: 'Grzegorz Kasprowicz' <kaspr...@gmail.com>; 'Slichter, Daniel H. (Fed)'
<daniel.slich...@nist.gov>; artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specification of the project

On Friday, 8 April 2016 11:53:25 AM HKT Grzegorz Kasprowicz wrote:
> Btw,
> ZU11 FPGA costs more or less the same as 7K325 and offers almost twice 
> more logic resources. The price of ZU11 is $1,376.00 at 100pcs.
> Since we will buy such quantity for our CBM project (Fair facility in 
> GSI), we can use that step pricing in this case as well.

Even on Digikey in single quantities, 7K325T starts at $898.75 (and the
cheapest one in FF which seems to be recommended for the full transceiver
bandwidth is $1122.50) and we don't have to deal with any of this disgusting
Zynq stuff.

Sébastien



___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Anyway, we have already plenty of things to do so I'd relay on existing
backplane at the moment:)

On 8 April 2016 at 12:51, Grzegorz Kasprowicz  wrote:

> Well, I don't have such template so all must be done from scratch.
> It's really big piece of PCB, we need at least 6..8 layers so just tooling
> is a few k$.
> The AMC backplane offered by Schroff has at least 20 layers.
> Greg
>
> On 8 April 2016 at 12:39, Sébastien Bourdeauducq  wrote:
>
>> On Friday, 8 April 2016 12:37:02 PM HKT Grzegorz Kasprowicz wrote:
>> > Modification of the backplane is quite difficult and expensive.
>>
>> If we have a minimalistic backplane with just power, maybe IPMI, 4x
>> differential pairs between MCH and each AMCs, and clock - why is that
>> particularly expensive?
>>
>> Sébastien
>>
>>
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Friday, 8 April 2016 12:37:02 PM HKT Grzegorz Kasprowicz wrote:
> Modification of the backplane is quite difficult and expensive.

If we have a minimalistic backplane with just power, maybe IPMI, 4x 
differential pairs between MCH and each AMCs, and clock - why is that 
particularly expensive?

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Modification of the backplane is quite difficult and expensive. You have to
have really good reason and money to pay to NAT, ELMA or other company.
But the good news is that together with NAT we are working on AMC backplane
optimized for timing distribution.
So additional layers of PCB are dedicated to the clock distribution.
Greg

On 8 April 2016 at 12:25, Sébastien Bourdeauducq  wrote:

> On Friday, 8 April 2016 12:02:43 PM HKT Grzegorz Kasprowicz wrote:
> > The MTCA backplane is not designed to distribute RF or analog clocks.
> >
> > Definitely they cannot be directly used to feed DACs or RF modules.
>
> Maybe we can modify the backplane, and get rid of what we do not need
> (Ethernet, AMC-to-AMC pairs, ...) and improve the clock distribution?
>
> Sébastien
>
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Friday, 8 April 2016 11:53:25 AM HKT Grzegorz Kasprowicz wrote:
> Btw,
> ZU11 FPGA costs more or less the same as 7K325 and offers almost twice more
> logic resources. The price of ZU11 is $1,376.00 at 100pcs.
> Since we will buy such quantity for our CBM project (Fair facility in GSI),
> we can use that step pricing in this case as well.

Even on Digikey in single quantities, 7K325T starts at $898.75 (and the 
cheapest one in FF which seems to be recommended for the full transceiver 
bandwidth is $1122.50) and we don't have to deal with any of this disgusting 
Zynq stuff.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

Just to clarify this: We do need a pretty large FPGA (7K325T) on the AMC cards 
due to the resource requirements of signal processing. Then we have 16 GTX 
12.5Gbps transceivers, but if we want to make the two FMCs identical and 
capable of receiving fast 4-channel DACs with 8 lanes, then we have no more 
transceivers. So the IOSERDES option is still interesting.

Greg - How many data lanes can we easily have between the MCH and each AMC 
with a star topology backplane?

Sébastien


On Wednesday, 30 March 2016 9:33:51 PM HKT Grzegorz Kasprowicz wrote:
> Hi all
> I like that idea - in this way we can do it really affordable with lower end
> FPGAs and still compatible with MTCA.
> If we place all RF stuff on the modules, with RF connectors mounted directly
> on the front panel, this could work.
> How we would deliver the clocks? Using external SMA connectors?
> The only think I worry about is performance of the DACs.
> ZynQ US+ can run IOserdes with almost 2Gbit/s performance - one can make
> 1000base-x directly on IO pins using oversampling technique.
> Greg
> 
> -Original Message-
> From: Sébastien Bourdeauducq [mailto:s...@m-labs.hk]
> Sent: Wednesday, March 30, 2016 1:50 PM
> To: Grzegorz Kasprowicz <kaspr...@gmail.com>
> Cc: 'Slichter, Daniel H. (Fed)' <daniel.slich...@nist.gov>; 'Grzegorz
> Kasprowicz' <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
> Subject: Re: [ARTIQ] FW: initial specification of the project
> 
> On Tuesday, 29 March 2016 11:55:19 PM HKT Grzegorz Kasprowicz wrote:
> > - they can be used immediately with existing OSHW carriers like AFC/AFCK.
> 
> AFCK may be overkill. We are still working on evaluating the FPGA resource
> requirements.
> 
> > - it could be hard to fit FPGA, supply, DACs and several RF modules,
> > all on single dual width AMC, especially when shielding is required.
> > RTM relaxes these constraints
> > - on AMC+RTM you can place 8 ADC channels + 8 DAC channels + 8 RF
> > modules. In case of single AMC board it would be hard to achieve such
> > channels density.
> 
> How about this:
> * we reduce the number of channels per AMC to 4 DACs + 4 ADCs
> * we can therefore use a smaller FPGA. Communication lanes to the master
> board are relatively cheap if we put them on IOSERDES.
> * the power density and cooling requirements are also reduced.
> * for the RF daughter cards, we use a custom form factor that can use at
> least
> 2/3 of the AMC front panel
> * connectors between the DSP card and the RF daughter card are 2mm header
> and 8x SMP
> * the rest of the AMC front panel used for (optional, runtime selectable)
> clock input and some TTLs.
> 
> Sébastien


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Thomas Harty]

Correction in point 2: The distributor I meant was HMC6832, not HMC830 and I 
meant to say "will degrade the Wenzel oscillator's phase noise somewhat at 
higher frequencies" (not "considerably")...

T



From: Thomas Harty
Sent: 04 April 2016 18:38
To: Slichter, Daniel H. (Fed); artiq@lists.m-labs.hk
Subject: RE: FW: initial specification of the project

Daniel, thanks for the feedback. "A few" comments in response:

tl;dr: distributing a reference clock at something like 10MHz-100MHz over the 
backplane, and generating the required DAC/upconverter clocks locally using 
VCOs seems to be by far the nicest, most scalable and flexible solution. There 
are, however, potential issues with phase drift/noise which could cause 
problems with this approach. However, I'm not convinced that there aren't 
equally difficult problems with any other approach. And, in any case, I'm 
optimistic that this won't be the case and the VCOs will be fine. We've started 
doing some tests on this and have more planned, and hope to have at least a 
partial answer soon.

1) You're right that for the most critical applications the single-loop PLL I 
described previously could start to become a limitation (although, we did gates 
with 99.% fidelity with a clock source that had a noise level pretty 
similar to the VCO I suggested). In practice, one would probably want to go for 
a 2-loop solution anyway to avoid having such a high divider ratio (240) in a 
single loop, which can lead to a significant noise contribution from the PFD 
(scales as 10log10(N) ).

An example 2-loop PLL might be: begin by locking a 100MHz VCO to the 10MHz 
backplane-distributed reference using a lock bandwidth of ~1kHz. This loop 
would be designed to "clean up" the 10MHz reference clock by rejecting 
cross-talk, and needs to use a VCO with extremely good close-in phase noise. A 
choice of VC(S)O could be a Crystek CVSS-945 ($20 in single quantity from 
Digi-Key 
http://www.digikey.co.uk/product-detail/en/crystek-corporation/CVSS-945-100.000/744-1612-ND/3711616
 although, there are fancier options if one really wants to push things). One 
would then lock the 2.4GHz VCO to the 100MHz with a loop bandwidth of ~500kHz.

In this system, the phase noise should be within ~20dB of the Wenzel 1GHz 
oscillator at all frequencies, with the VCO and Wenzel oscillator being 
comparable by ~1MHz. This does assume that the PLL doesn't degrade the VCO 
noise significantly, which is something we plan to test.

While it's more engineering work to design and test this system, it's work that 
can be done once on a prototype board and then copied into the final design. 
I'd generally prefer that to requiring each user to sort out their own external 
oscillator, power splitters, amps, cables, etc.

Assuming we can get close to the VCO data sheet noise, I can't see the 
difference between the VCOs and Wenzel oscillator being significant, 
particularly given that...

2) Distributing the Wenzel oscillator's signal without degrading its phase 
noise is going to be pretty tough. For example, the best clock distribution IC 
that I know of is the HMC830. Looking at figure 22 of its data sheet suggests 
that even this IC will degrade the Wenzel oscillator's phase noise 
considerably. In general, clock switches tend to be far worse than this (which 
is one reason that I'd really rather not put a switch in the clock path to 
select between the front-panel and backplane clocks). I wouldn't trust a 
switch/distribution IC to work at those phase noise levels unless its data 
sheet explicitly gave a phase noise plot.

A similar remark goes for the ERA-4SM amplifier you suggest using on the DACs 
output. For close-in phase noise, amplifier noise figure is meaningless. In 
general, for work where phase noise really matters, no amplifiers should be 
used unless their datasheet actually gives a phase noise plot -- and I 
certainly wouldn't trust a cheap MCL "general purpose" Darlington amp with my 
Wenzel oscillator without some careful measurements!

3) The DAC's phase noise contribution is going to be worse than the locked 
VCO's noise (compare the 100MHz trace on figure 33 of the AD9154 data sheet to 
the VCO datasheets) for all frequencies above 1kHz. Below 1kHz, the locked 
VCO's noise should be reduced close to the level of the 10MHz reference source, 
which could be a Wenzel oscillator if one wants. FWIW, the situation is the 
same even using something like an AD9914, which is about the lowest noise 
digital RF source I've seen.

Similarly, before one gets _too_ worried about the possibility of a few 
residual cross-talk spurs near the noise floor on the DAC clock, let's see what 
the DAC's output spectrum actually looks like. (but then, maybe I'm just 
scarred by the forest of low-level spurs on our current DDS boards...)

4) In practice, I think that very low-level phase noise isn't going to be the 
thing that stops us all getting gate errors <1e-6. For 

Re: [ARTIQ] FW: initial specification of the project

[Thomas Harty]

Daniel, thanks for the feedback. "A few" comments in response:

tl;dr: distributing a reference clock at something like 10MHz-100MHz over the 
backplane, and generating the required DAC/upconverter clocks locally using 
VCOs seems to be by far the nicest, most scalable and flexible solution. There 
are, however, potential issues with phase drift/noise which could cause 
problems with this approach. However, I'm not convinced that there aren't 
equally difficult problems with any other approach. And, in any case, I'm 
optimistic that this won't be the case and the VCOs will be fine. We've started 
doing some tests on this and have more planned, and hope to have at least a 
partial answer soon.

1) You're right that for the most critical applications the single-loop PLL I 
described previously could start to become a limitation (although, we did gates 
with 99.% fidelity with a clock source that had a noise level pretty 
similar to the VCO I suggested). In practice, one would probably want to go for 
a 2-loop solution anyway to avoid having such a high divider ratio (240) in a 
single loop, which can lead to a significant noise contribution from the PFD 
(scales as 10log10(N) ).

An example 2-loop PLL might be: begin by locking a 100MHz VCO to the 10MHz 
backplane-distributed reference using a lock bandwidth of ~1kHz. This loop 
would be designed to "clean up" the 10MHz reference clock by rejecting 
cross-talk, and needs to use a VCO with extremely good close-in phase noise. A 
choice of VC(S)O could be a Crystek CVSS-945 ($20 in single quantity from 
Digi-Key 
http://www.digikey.co.uk/product-detail/en/crystek-corporation/CVSS-945-100.000/744-1612-ND/3711616
 although, there are fancier options if one really wants to push things). One 
would then lock the 2.4GHz VCO to the 100MHz with a loop bandwidth of ~500kHz.

In this system, the phase noise should be within ~20dB of the Wenzel 1GHz 
oscillator at all frequencies, with the VCO and Wenzel oscillator being 
comparable by ~1MHz. This does assume that the PLL doesn't degrade the VCO 
noise significantly, which is something we plan to test.

While it's more engineering work to design and test this system, it's work that 
can be done once on a prototype board and then copied into the final design. 
I'd generally prefer that to requiring each user to sort out their own external 
oscillator, power splitters, amps, cables, etc.

Assuming we can get close to the VCO data sheet noise, I can't see the 
difference between the VCOs and Wenzel oscillator being significant, 
particularly given that...

2) Distributing the Wenzel oscillator's signal without degrading its phase 
noise is going to be pretty tough. For example, the best clock distribution IC 
that I know of is the HMC830. Looking at figure 22 of its data sheet suggests 
that even this IC will degrade the Wenzel oscillator's phase noise 
considerably. In general, clock switches tend to be far worse than this (which 
is one reason that I'd really rather not put a switch in the clock path to 
select between the front-panel and backplane clocks). I wouldn't trust a 
switch/distribution IC to work at those phase noise levels unless its data 
sheet explicitly gave a phase noise plot.

A similar remark goes for the ERA-4SM amplifier you suggest using on the DACs 
output. For close-in phase noise, amplifier noise figure is meaningless. In 
general, for work where phase noise really matters, no amplifiers should be 
used unless their datasheet actually gives a phase noise plot -- and I 
certainly wouldn't trust a cheap MCL "general purpose" Darlington amp with my 
Wenzel oscillator without some careful measurements!

3) The DAC's phase noise contribution is going to be worse than the locked 
VCO's noise (compare the 100MHz trace on figure 33 of the AD9154 data sheet to 
the VCO datasheets) for all frequencies above 1kHz. Below 1kHz, the locked 
VCO's noise should be reduced close to the level of the 10MHz reference source, 
which could be a Wenzel oscillator if one wants. FWIW, the situation is the 
same even using something like an AD9914, which is about the lowest noise 
digital RF source I've seen.

Similarly, before one gets _too_ worried about the possibility of a few 
residual cross-talk spurs near the noise floor on the DAC clock, let's see what 
the DAC's output spectrum actually looks like. (but then, maybe I'm just 
scarred by the forest of low-level spurs on our current DDS boards...)

4) In practice, I think that very low-level phase noise isn't going to be the 
thing that stops us all getting gate errors <1e-6. For example, I'd guess that 
duty-cycle-dependent thermal amplitude/phase transients in the DAC/amplifier 
will be far more of a problem in practice. Obviously, that's not a reason to be 
sloppy about the clock noise, but it does mean that it's probably not worth 
complicating the external wiring by introducing an external oscillator just to 
save a few dB...



As you say, one could 

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Friday, 1 April 2016 5:25:19 PM HKT Thomas Harty wrote:
> d) Something else I'm missing?

Low-frequency jitter of the PLL?

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Thomas Harty]

Agreed, clock phase noise is _very_ important, and backplane crosstalk will be 
a killer if clocks are distributed without due care. Having said that, I'm not 
convinced that clock distribution via the backplane isn't the right way to go 
if done properly -- even for the most demanding applications.

The plan would be to derive the backplane clock from a source with extremely 
good close-in phase noise/long term stability, such as an atomic reference. 
Transmission over the backplane will degrade the short-term stability (the 
phase noise far from the carrier) to an unacceptable level. To fix this, we 
would use an on-AMC (or FMC board) PLL based on a low-noise VCO. The PLL loop 
filter would be designed with narrow bandwidth in order to reduce the VCO phase 
noise to the reference level at low frequencies, while still rejecting noise 
further from the carrier.

Take the 2.4GHz DAC clock as an example. A decent (relatively small and 
inexpensive) choice of VCO for the PLL might be a UMX-630-D16-G 
(http://www.rfmd.com/store/products/all-products/oscillators/ultra-low-noise-cro-vcos/umx-630-d16-g-1.html).
 To get an idea of the lock bandwidth required, we can compare the phase noise 
of this VCO with a typical high-quality lab 10MHz source, such as a SRS FS725 
Rubidium Standard (http://www.thinksrs.com/products/FS725.htm) scaled from 
10MHz to 2.4GHz by adding 20*log10(240)=48dB. By 10kHz offset frequency, the 
typical VCO noise is -108dBc/Hz, compared with a (scaled) noise for the FS725 
of ~-105dBc/Hz. Beyond that, the VCO is significantly better*.

So, the lock bandwidth we need should only be a few kHz. With proper design of 
the loop filter, one should be able to heavily suppress any noise much outside 
this range (it might also be worth putting a low-Q low-pass/band-pass filter on 
the 10MHz clock to remove any noise far outside the loop bandwidth). So, as 
long as we don't put any signals down the backplane which have significant 
spectral content within, say, a few hundred kHz of the clock frequency, I 
wouldn't expect cross-talk to be an issue.

Presumably, your concern is some combination of:

a) We won't be able to build a PLL on the AMC/FMC board which has good enough 
phase noise @2.4GHz, regardless of the quality of the 10MHz reference we feed it
b) The PLL won't do a good enough job of removing noise on the reference clock 
far outside the loop bandwidth (say, outside the range 9MHz - 11MHz if one 
wants to be pessimistic)
c) Even with a narrow-bandwidth (~1kHz) PLL with carefully chosen loop filter, 
and taking care over what other signals we send down the backplane, there will 
be too much pickup on the clock line close to 10MHz that can't be filtered off
d) Something else I'm missing?

If you have any measurements relevant to this, I'd be interested to see them. 
We're currently building hardware, one purpose of which will be to test how 
well we can generate a low phase noise 1GHz DDS clock from a 10MHz signal 
distributed down a uTCA backplane.

Best,

Tom

* Of course, actually achieving this noise level will require some work in 
terms of simulation/testing the PLL design, and may need a multi-loop PLL to 
prevent the high divider ratio from causing problems with the PFD...

From: Slichter, Daniel H. (Fed) [daniel.slich...@nist.gov]
Sent: 31 March 2016 17:21
To: Thomas Harty; artiq@lists.m-labs.hk
Subject: RE: FW: initial specification of the project


Re DAC clock: presumably, the plan is to distribute a 10MHz (or 100MHz, or 
whatever) clock with really low close-in phase noise to each MCH, and then from 
each MCH to the AMCs via the backplane. Why not send this from the AMC to the 
FMC boards via the FMC connectors? Then, have the FMC board generate the DAC 
clock using an integer-N PLL. If done properly, an int-N PLL at a couple of GHz 
is cheap, has a small footprint and will give as good phase noise/phase 
stability as using an external synth plus coax. The loop filter bandwidth can 
be low enough to take out any cross-talk from the FMC/backplane. Plus, this way 
the DAC clock has a well-defined phase relationship to the main system clock 
(the 10MHz or whatever), which may come in handy for keeping everything 
synchronised...
Basically, it is not possible to generate a sufficiently high performance DAC 
clock for many applications from something that comes over the backplane, due 
to crosstalk/signal integrity issues.  Joe Britton has some data on clock noise 
over uTCA backplanes, I believe.  The idea would be to have a clock selection 
crossbar switch on the FMC card, which allows the user either to use an 
external clock brought in on an SMA connector, or to use a clock generated by a 
PLL on the AMC card referenced to a backplane clock (and there will be 
backplane clocks for digital timing purposes).  This way people who really care 
about phase noise can have it as good as they want, while people who don’t care 
as much can dispense with 

Re: [ARTIQ] FW: initial specification of the project

[Thomas Harty]

Re DAC clock: presumably, the plan is to distribute a 10MHz (or 100MHz, or 
whatever) clock with really low close-in phase noise to each MCH, and then from 
each MCH to the AMCs via the backplane. Why not send this from the AMC to the 
FMC boards via the FMC connectors? Then, have the FMC board generate the DAC 
clock using an integer-N PLL. If done properly, an int-N PLL at a couple of GHz 
is cheap, has a small footprint and will give as good phase noise/phase 
stability as using an external synth plus coax. The loop filter bandwidth can 
be low enough to take out any cross-talk from the FMC/backplane. Plus, this way 
the DAC clock has a well-defined phase relationship to the main system clock 
(the 10MHz or whatever), which may come in handy for keeping everything 
synchronised...


> No!  The clock coming in will be a sine wave from a low phase noise 
> oscillator somewhere.  The DACs and ADCs will threshold this clock to 
> determine their sample times.  Any amount of crosstalk will distort the clock 
> signal (adding or subtracting to the voltage at a given time), thus skewing 
> the time at which the threshold is reached and thus inducing jitter into the 
> sampling times.  This would also hold true even if you have an LVPECL clock 
> signal, because at frequencies like 2.4 GHz the rise and fall times (~100-200 
> ps) are similar to that of a sine wave.  Unlike for digital data signals, any 
> amount of crosstalk will degrade the jitter and/or edge time performance for 
> a clock signal.

Tom
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

well, there are verified OH designs of such cards:
32 channel TTL IO
http://www.ohwr.org/projects/fmc-dio-32chttla/wiki
slow DAC card with some IOs and slow ADCs.
http://www.ohwr.org/projects/fmc-dac100m14b16cha-adc2m14b4cha/wiki


On 31 March 2016 at 01:45, Slichter, Daniel H. (Fed) <
daniel.slich...@nist.gov> wrote:

> One further question: is there a plan to make a “TTL” card or a
> multichannel “slow” DAC card (e.g. for trap voltages), using a Centronics
> or d-sub type connector?  These could both be more readily accomplished
> with their own FMC modules if we go with this architecture.
>
>
>
> *From:* Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
> *Sent:* Wednesday, March 30, 2016 3:36 PM
>
> *To:* Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
> *Cc:* Robert Jördens <r...@m-labs.hk>; Grzegorz Kasprowicz <
> gkasp...@elka.pw.edu.pl>; Leibrandt, David R. (Fed) <
> david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>;
> artiq@lists.m-labs.hk
> *Subject:* Re: [ARTIQ] FW: initial specification of the project
>
>
>
> Here is example of CERN carrier with analogue voltages:
> http://www.ohwr.org/projects/fmc-pci-carrier/wiki
> "+5V, -2V, -5V2 and -12V optionally wired on HPC pins"
>
> look here
> https://edms.cern.ch/ui/file/1098639/1/EDA-02118-V1-1_sch.pdf
>
> page 3
>
>
>
> On 30 March 2016 at 23:17, Slichter, Daniel H. (Fed) <
> daniel.slich...@nist.gov> wrote:
>
> We definitely need +/- 15V for the low frequency (e.g. trap electrode)
> amplifiers.  Many low noise amplifiers and RF components run off +5V or +/-
> 5V and have substantial current draws, so if you pull everything from +/-
> 15V rails you are tripling your power dissipation and you end up with very
> hot regulators.  The DACs and ADCs will be dissipating a fair amount of
> power already so we want to try to keep the power budget under control.
> Other than that, I don’t see major reasons why one couldn’t run fewer
> analog rails.  I think we are better off with DC/DC converters on the AMC
> card making a number of rails, which then have some filtering/regulation on
> the AMC card and then a final stage of LDO regulation on the FMC
> daughtercard itself, as close to the amplifiers etc as possible.
>
>
>
> Using the alternating grounds a la CERN seems like a suitable solution to
> me for sending in these additional analog rails.
>
>
>
> *From:* Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
> *Sent:* Wednesday, March 30, 2016 3:13 PM
> *To:* Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
> *Cc:* Robert Jördens <r...@m-labs.hk>; Grzegorz Kasprowicz <
> gkasp...@elka.pw.edu.pl>; Leibrandt, David R. (Fed) <
> david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>;
> artiq@lists.m-labs.hk
> *Subject:* Re: [ARTIQ] FW: initial specification of the project
>
>
>
> Well, CERN does it in their FMC carriers. They use HPC routed like LPC and
> then some of grounds are used as symmetrical analog supplies. In this way
> if you plug wrong board, it will short the supply but no damage will occur.
>
> I assume that low nosie DC/DC converters + LDOs will be installed on the
> AMC board.
>
> Do we need all these voltages, especially +/-5 and +/- 15? Won't single
> +/- 8 or +/- 15V be sufficient?
>
> Greg
>
>
>
> On 30 March 2016 at 23:08, Slichter, Daniel H. (Fed) <
> daniel.slich...@nist.gov> wrote:
>
>
>
> Actually HPC with LPC IO assignment and 8 x GTP links is popular
> configuration
>
> So you have 34 LVDS pairs and 8 GTP links.
>
>
>
> If this works for you then I don’t have major objections.  The other issue
> to consider is power rails, since for the analog circuitry we will probably
> want +/- 5V, +/- 15V as well as +3.3V, +1.8V, +12V.  Can we put these
> through on the FMC without breaking back compatibility?  For example, one
> rail on each of the 4 VADJ pins?  I am sure this would not be VITA 57
> compliant….and we don’t want switching converters on the FMC daughtercard
> for space and noise reasons both.
>
>
>
>
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

On 31 March 2016 at 04:58, Sébastien Bourdeauducq  wrote:

> On Wednesday, 30 March 2016 9:37:51 PM HKT Grzegorz Kasprowicz wrote:
> > Well, you don't have to write it.
> > It is already available for RTOS and linux.
>
> We are not using RTOS or Linux.
>
you can also program it bare metal.
We have such solution in one of our projects where Ethernet is used.

>
> > But it's true - it occupies MIO bank and dedicated DDR port. But this is
> axi
> > and can be easily accessible from PL part.
> > How many IOs do you need?
>
> IO count is not the problem. The problem is you cannot switch between
> designs
> that use the fabric to talk directly to Ethernet and SDRAM, and designs
> that
> use those questionable hardened blocks.
>
well, you can talk from PL to Ethernet or SDRAM via AXI.
In this way we build AXI video streaming which goes directly to PS SDRAM.
Only MMU must be either switched off or programmed to give such access.
Greg

>
> Sébastien
>
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Wednesday, 30 March 2016 9:37:51 PM HKT Grzegorz Kasprowicz wrote:
> Well, you don't have to write it.
> It is already available for RTOS and linux.

We are not using RTOS or Linux.

> But it's true - it occupies MIO bank and dedicated DDR port. But this is axi
> and can be easily accessible from PL part.
> How many IOs do you need?

IO count is not the problem. The problem is you cannot switch between designs 
that use the fabric to talk directly to Ethernet and SDRAM, and designs that 
use those questionable hardened blocks.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

One further question: is there a plan to make a “TTL” card or a multichannel 
“slow” DAC card (e.g. for trap voltages), using a Centronics or d-sub type 
connector?  These could both be more readily accomplished with their own FMC 
modules if we go with this architecture.

From: Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
Sent: Wednesday, March 30, 2016 3:36 PM
To: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
Cc: Robert Jördens <r...@m-labs.hk>; Grzegorz Kasprowicz 
<gkasp...@elka.pw.edu.pl>; Leibrandt, David R. (Fed) 
<david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>; 
artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specification of the project

Here is example of CERN carrier with analogue voltages:
http://www.ohwr.org/projects/fmc-pci-carrier/wiki
"+5V, -2V, -5V2 and -12V optionally wired on HPC pins"
look here
https://edms.cern.ch/ui/file/1098639/1/EDA-02118-V1-1_sch.pdf
page 3

On 30 March 2016 at 23:17, Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>> wrote:
We definitely need +/- 15V for the low frequency (e.g. trap electrode) 
amplifiers.  Many low noise amplifiers and RF components run off +5V or +/- 5V 
and have substantial current draws, so if you pull everything from +/- 15V 
rails you are tripling your power dissipation and you end up with very hot 
regulators.  The DACs and ADCs will be dissipating a fair amount of power 
already so we want to try to keep the power budget under control.  Other than 
that, I don’t see major reasons why one couldn’t run fewer analog rails.  I 
think we are better off with DC/DC converters on the AMC card making a number 
of rails, which then have some filtering/regulation on the AMC card and then a 
final stage of LDO regulation on the FMC daughtercard itself, as close to the 
amplifiers etc as possible.

Using the alternating grounds a la CERN seems like a suitable solution to me 
for sending in these additional analog rails.

From: Grzegorz Kasprowicz [mailto:kaspr...@gmail.com<mailto:kaspr...@gmail.com>]
Sent: Wednesday, March 30, 2016 3:13 PM
To: Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>>
Cc: Robert Jördens <r...@m-labs.hk<mailto:r...@m-labs.hk>>; Grzegorz Kasprowicz 
<gkasp...@elka.pw.edu.pl<mailto:gkasp...@elka.pw.edu.pl>>; Leibrandt, David R. 
(Fed) <david.leibra...@nist.gov<mailto:david.leibra...@nist.gov>>; Sébastien 
Bourdeauducq <s...@m-labs.hk<mailto:s...@m-labs.hk>>; 
artiq@lists.m-labs.hk<mailto:artiq@lists.m-labs.hk>
Subject: Re: [ARTIQ] FW: initial specification of the project

Well, CERN does it in their FMC carriers. They use HPC routed like LPC and then 
some of grounds are used as symmetrical analog supplies. In this way if you 
plug wrong board, it will short the supply but no damage will occur.
I assume that low nosie DC/DC converters + LDOs will be installed on the AMC 
board.
Do we need all these voltages, especially +/-5 and +/- 15? Won't single +/- 8 
or +/- 15V be sufficient?
Greg

On 30 March 2016 at 23:08, Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>> wrote:

Actually HPC with LPC IO assignment and 8 x GTP links is popular configuration
So you have 34 LVDS pairs and 8 GTP links.

If this works for you then I don’t have major objections.  The other issue to 
consider is power rails, since for the analog circuitry we will probably want 
+/- 5V, +/- 15V as well as +3.3V, +1.8V, +12V.  Can we put these through on the 
FMC without breaking back compatibility?  For example, one rail on each of the 
4 VADJ pins?  I am sure this would not be VITA 57 compliant….and we don’t want 
switching converters on the FMC daughtercard for space and noise reasons both.


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

Looks like one could get a pretty good start on things by just copying the 
designs used by marki microwave:

http://www.markimicrowave.com/assets/packages/CTG.pdf
http://www.markimicrowave.com/assets/appnotes/an-ct-pcb.pdf

On another note, I would recommend the MM1-0320HSM mixer as a good option for 
the upconversion board if we decide not to go with an IQ modulator (there the 
ADL5375 would be best but I haven’t tested it to 12 GHz).

http://www.markimicrowave.com/MM1-0320HSM-MMIC-Double-Balanced-Mixer-P782.aspx


From: Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
Sent: Wednesday, March 30, 2016 3:09 PM
To: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
Cc: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; Leibrandt, David R. (Fed) 
<david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>; 
artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specification of the project

Well, it depends on trace width matching. We can simulate and characterize it 
even at much higher frequencies.
If we place them really close to the DAC and match widths, remove grounds under 
pads, we can match it not worse than SMA connectors.
We can even do even more crazy thing - try to make them mountable.
The only signals that would need to be transmitted is DAC output / ADC input.
We can use low profile board to board connectors for rest of the signals - I 
know ones that have 1mm stacking hight. For RF we could use PCB mounted UFL 
connectors.
This is just crazy idea that need to be verified.

On 30 March 2016 at 23:02, Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>> wrote:
This is an interesting potential solution although I am not sure how the signal 
integrity is at ~3 GHz, for example.

From: Grzegorz Kasprowicz [mailto:kaspr...@gmail.com<mailto:kaspr...@gmail.com>]
Sent: Wednesday, March 30, 2016 2:44 PM
To: Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>>
Cc: Grzegorz Kasprowicz 
<gkasp...@elka.pw.edu.pl<mailto:gkasp...@elka.pw.edu.pl>>; Leibrandt, David R. 
(Fed) <david.leibra...@nist.gov<mailto:david.leibra...@nist.gov>>; Sébastien 
Bourdeauducq <s...@m-labs.hk<mailto:s...@m-labs.hk>>; 
artiq@lists.m-labs.hk<mailto:artiq@lists.m-labs.hk>
Subject: Re: [ARTIQ] FW: initial specification of the project

Such assembly technique is called:
castellated PCB module
https://www.google.pl/search?q=castellated+RF+modules=1920=917=lnms=isch=X=0ahUKEwjRvuqko-nLAhWnnXIKHe_IARsQ_AUIBygB<https://www.google.pl/search?q=castellated+PCB+down+converter+modules=1920=917=lnms=isch=X=0ahUKEwj19L7younLAhUKvXIKHQVBCg8Q_AUIBigB#tbm=isch=castellated+RF+modules>

On 30 March 2016 at 22:40, Grzegorz Kasprowicz 
<kaspr...@gmail.com<mailto:kaspr...@gmail.com>> wrote:
One more thing - we can fit 5 SMA connectors on FMC panel, in case of 6 ones, 
we won't be able to screw them.
But we can install MMCX ones for clocks and fit in total of 7 RF connectors,
Look here
http://www.ohwr.org/attachments/3390/fmc_top.jpg
Greg

On 30 March 2016 at 22:38, Grzegorz Kasprowicz 
<kaspr...@gmail.com<mailto:kaspr...@gmail.com>> wrote:
Well, we can do another crazy thing - solder small module with RF stuff on the 
FMC board, under same shield.
In this way we keep 3 simple FMCs with expensive ADCs/DACs and define the 
functionality by soldering (automatic or manual) of just RF modules. WE can 
even design such modules to hold the front-end connectors of leave them on the 
FMC.
Such approach has also some attractive feature - we can make them using small 
pieces of Rogers material which is hell expensive and it's hard to make small 
vias and thin traces needed for JESD signals.
these modules could look like that
http://www.emcfastpass.com/wp-content/uploads/2014/09/rf_module_holes.gif
You can mount them by pick and place or manually.
IT is also possible to manually disassemble them.
This is a form factor of popular RF modules, i.e. wifi, GPS and LTE modems.
http://www.emcfastpass.com/rf-modules/
And is simply works
Greg


On 30 March 2016 at 22:25, Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>> wrote:
> Maybe we should come back to the roots:) What if we use standard FMCs
> (LPC) with DAC/ADC channels and RF stuff _on_ them.
> JESD204B and some pins would go to the FPGA while DAC and RF clock would
> be fed externally.
> In this way we leave general purpose AMC board and define its functionality
> by FMC boards If we make 3 flavours of FMCs: ADC+ADC,
> ADC+DAC,DAC+DAC, we would cover several use cases:
> Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
> FMCs with only DAC and RF stuff on it can be simple, 4 layer boards with
> external sield.
> Look at this shield (my project)
> http://www.ohwr.org/projects/fmc-adc-130m-16b-4cha/wiki
> In this way we could use existi

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

We could also use a fuzz button interposer, which would do the same kind of 
task (low stacking height connection w/ high frequency compatibility), but I 
think once we start heading too far down this road the cost and complexity of 
making a robust solution starts to outweigh the cost of just making more types 
of boards.  We should consider carefully before we make too fancy a solution.  
The notion of the SMP connectors is that they are robust, high performance, 
suited for the task, and cheap.

It does seem that microwave-frequency castellated solutions are used by serious 
industry folks:

https://www.markimicrowave.com/Assets/appnotes/Marki_Surface_mount_Guide_V1.pdf


From: Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
Sent: Wednesday, March 30, 2016 3:09 PM
To: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
Cc: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; Leibrandt, David R. (Fed) 
<david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>; 
artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specification of the project

Well, it depends on trace width matching. We can simulate and characterize it 
even at much higher frequencies.
If we place them really close to the DAC and match widths, remove grounds under 
pads, we can match it not worse than SMA connectors.
We can even do even more crazy thing - try to make them mountable.
The only signals that would need to be transmitted is DAC output / ADC input.
We can use low profile board to board connectors for rest of the signals - I 
know ones that have 1mm stacking hight. For RF we could use PCB mounted UFL 
connectors.
This is just crazy idea that need to be verified.

On 30 March 2016 at 23:02, Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>> wrote:
This is an interesting potential solution although I am not sure how the signal 
integrity is at ~3 GHz, for example.

From: Grzegorz Kasprowicz [mailto:kaspr...@gmail.com<mailto:kaspr...@gmail.com>]
Sent: Wednesday, March 30, 2016 2:44 PM
To: Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>>
Cc: Grzegorz Kasprowicz 
<gkasp...@elka.pw.edu.pl<mailto:gkasp...@elka.pw.edu.pl>>; Leibrandt, David R. 
(Fed) <david.leibra...@nist.gov<mailto:david.leibra...@nist.gov>>; Sébastien 
Bourdeauducq <s...@m-labs.hk<mailto:s...@m-labs.hk>>; 
artiq@lists.m-labs.hk<mailto:artiq@lists.m-labs.hk>
Subject: Re: [ARTIQ] FW: initial specification of the project

Such assembly technique is called:
castellated PCB module
https://www.google.pl/search?q=castellated+RF+modules=1920=917=lnms=isch=X=0ahUKEwjRvuqko-nLAhWnnXIKHe_IARsQ_AUIBygB<https://www.google.pl/search?q=castellated+PCB+down+converter+modules=1920=917=lnms=isch=X=0ahUKEwj19L7younLAhUKvXIKHQVBCg8Q_AUIBigB#tbm=isch=castellated+RF+modules>

On 30 March 2016 at 22:40, Grzegorz Kasprowicz 
<kaspr...@gmail.com<mailto:kaspr...@gmail.com>> wrote:
One more thing - we can fit 5 SMA connectors on FMC panel, in case of 6 ones, 
we won't be able to screw them.
But we can install MMCX ones for clocks and fit in total of 7 RF connectors,
Look here
http://www.ohwr.org/attachments/3390/fmc_top.jpg
Greg

On 30 March 2016 at 22:38, Grzegorz Kasprowicz 
<kaspr...@gmail.com<mailto:kaspr...@gmail.com>> wrote:
Well, we can do another crazy thing - solder small module with RF stuff on the 
FMC board, under same shield.
In this way we keep 3 simple FMCs with expensive ADCs/DACs and define the 
functionality by soldering (automatic or manual) of just RF modules. WE can 
even design such modules to hold the front-end connectors of leave them on the 
FMC.
Such approach has also some attractive feature - we can make them using small 
pieces of Rogers material which is hell expensive and it's hard to make small 
vias and thin traces needed for JESD signals.
these modules could look like that
http://www.emcfastpass.com/wp-content/uploads/2014/09/rf_module_holes.gif
You can mount them by pick and place or manually.
IT is also possible to manually disassemble them.
This is a form factor of popular RF modules, i.e. wifi, GPS and LTE modems.
http://www.emcfastpass.com/rf-modules/
And is simply works
Greg


On 30 March 2016 at 22:25, Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>> wrote:
> Maybe we should come back to the roots:) What if we use standard FMCs
> (LPC) with DAC/ADC channels and RF stuff _on_ them.
> JESD204B and some pins would go to the FPGA while DAC and RF clock would
> be fed externally.
> In this way we leave general purpose AMC board and define its functionality
> by FMC boards If we make 3 flavours of FMCs: ADC+ADC,
> ADC+DAC,DAC+DAC, we would cover several use cases:
> Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
> FMCs with only DAC and RF stuff on it can be simple, 4 laye

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

OK, I see your point

On 30 March 2016 at 23:17, Slichter, Daniel H. (Fed) <
daniel.slich...@nist.gov> wrote:

> We definitely need +/- 15V for the low frequency (e.g. trap electrode)
> amplifiers.  Many low noise amplifiers and RF components run off +5V or +/-
> 5V and have substantial current draws, so if you pull everything from +/-
> 15V rails you are tripling your power dissipation and you end up with very
> hot regulators.  The DACs and ADCs will be dissipating a fair amount of
> power already so we want to try to keep the power budget under control.
> Other than that, I don’t see major reasons why one couldn’t run fewer
> analog rails.  I think we are better off with DC/DC converters on the AMC
> card making a number of rails, which then have some filtering/regulation on
> the AMC card and then a final stage of LDO regulation on the FMC
> daughtercard itself, as close to the amplifiers etc as possible.
>
>
>
> Using the alternating grounds a la CERN seems like a suitable solution to
> me for sending in these additional analog rails.
>
>
>
> *From:* Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
> *Sent:* Wednesday, March 30, 2016 3:13 PM
> *To:* Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
> *Cc:* Robert Jördens <r...@m-labs.hk>; Grzegorz Kasprowicz <
> gkasp...@elka.pw.edu.pl>; Leibrandt, David R. (Fed) <
> david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>;
> artiq@lists.m-labs.hk
> *Subject:* Re: [ARTIQ] FW: initial specification of the project
>
>
>
> Well, CERN does it in their FMC carriers. They use HPC routed like LPC and
> then some of grounds are used as symmetrical analog supplies. In this way
> if you plug wrong board, it will short the supply but no damage will occur.
>
> I assume that low nosie DC/DC converters + LDOs will be installed on the
> AMC board.
>
> Do we need all these voltages, especially +/-5 and +/- 15? Won't single
> +/- 8 or +/- 15V be sufficient?
>
> Greg
>
>
>
> On 30 March 2016 at 23:08, Slichter, Daniel H. (Fed) <
> daniel.slich...@nist.gov> wrote:
>
>
>
> Actually HPC with LPC IO assignment and 8 x GTP links is popular
> configuration
>
> So you have 34 LVDS pairs and 8 GTP links.
>
>
>
> If this works for you then I don’t have major objections.  The other issue
> to consider is power rails, since for the analog circuitry we will probably
> want +/- 5V, +/- 15V as well as +3.3V, +1.8V, +12V.  Can we put these
> through on the FMC without breaking back compatibility?  For example, one
> rail on each of the 4 VADJ pins?  I am sure this would not be VITA 57
> compliant….and we don’t want switching converters on the FMC daughtercard
> for space and noise reasons both.
>
>
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

So now we are moving towards complying with the AMC FMC physical standard.  Is 
this something we want to do?  There are pluses (able to use existing AMC FMC 
cards) and minuses (squeezing things in more tightly than we otherwise might 
have to).

From: Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
Sent: Wednesday, March 30, 2016 2:41 PM
To: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
Cc: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; Leibrandt, David R. (Fed) 
<david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>; 
artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specification of the project

One more thing - we can fit 5 SMA connectors on FMC panel, in case of 6 ones, 
we won't be able to screw them.
But we can install MMCX ones for clocks and fit in total of 7 RF connectors,
Look here
http://www.ohwr.org/attachments/3390/fmc_top.jpg
Greg

On 30 March 2016 at 22:38, Grzegorz Kasprowicz 
<kaspr...@gmail.com<mailto:kaspr...@gmail.com>> wrote:
Well, we can do another crazy thing - solder small module with RF stuff on the 
FMC board, under same shield.
In this way we keep 3 simple FMCs with expensive ADCs/DACs and define the 
functionality by soldering (automatic or manual) of just RF modules. WE can 
even design such modules to hold the front-end connectors of leave them on the 
FMC.
Such approach has also some attractive feature - we can make them using small 
pieces of Rogers material which is hell expensive and it's hard to make small 
vias and thin traces needed for JESD signals.
these modules could look like that
http://www.emcfastpass.com/wp-content/uploads/2014/09/rf_module_holes.gif
You can mount them by pick and place or manually.
IT is also possible to manually disassemble them.
This is a form factor of popular RF modules, i.e. wifi, GPS and LTE modems.
http://www.emcfastpass.com/rf-modules/
And is simply works
Greg


On 30 March 2016 at 22:25, Slichter, Daniel H. (Fed) 
<daniel.slich...@nist.gov<mailto:daniel.slich...@nist.gov>> wrote:
> Maybe we should come back to the roots:) What if we use standard FMCs
> (LPC) with DAC/ADC channels and RF stuff _on_ them.
> JESD204B and some pins would go to the FPGA while DAC and RF clock would
> be fed externally.
> In this way we leave general purpose AMC board and define its functionality
> by FMC boards If we make 3 flavours of FMCs: ADC+ADC,
> ADC+DAC,DAC+DAC, we would cover several use cases:
> Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
> FMCs with only DAC and RF stuff on it can be simple, 4 layer boards with
> external sield.
> Look at this shield (my project)
> http://www.ohwr.org/projects/fmc-adc-130m-16b-4cha/wiki
> In this way we could use existing AFCK for quick tests

We have been working with the notion that should be many possible front ends 
for each of the DACs or ADCs, depending on what the particular application is, 
and so we want to separate the analog daughtercards from whatever board has the 
DACs and ADCs on it.  This way, you can reconfigure the hardware for 
high-frequency or low-frequency applications, for example, by changing 
daughtercards and not having to build entire new AMC cards.  The modularity 
principle lets one have a single design for the AMC card (aka DSP card) that 
can be used for many different applications, by shifting the analog signal 
processing circuitry onto a separate card.

Now, as you suggest we could just change the level at which we make this break 
from the AMC card, shift the DACs and ADCs onto the daughter card as well, and 
use FMC to communicate with the whole thing.  This makes it a bit more 
expensive/difficult to reconfigure the analog front end, but the DAC and ADC 
costs are not so high that it is impossible to do.  I had envisioned the notion 
of making the daughtercards simple enough that end users could redesign/respin 
easily to accommodate their own applications, or we could ship unstuffed or 
partially stuffed boards that they could complete with the particular filters 
etc they desire.

However, I agree that there are compelling arguments for using the architecture 
you propose.  We would need to pick just a few board styles (I suggest quad 
DAC, 2 DAC/2 ADC, and quad ADC), and for each of these board styles we would 
need to make several different variants with different analog front ends (3 
types for DAC - low frequency, baseband RF, upconverted RF - and 2 types for 
ADC - baseband RF and downconverted RF, both likely including switchable gain). 
 So now we are looking at making 3 types of quad DAC boards, 2 types of ADC 
board, and probably 3 types of DAC/ADC board (upconvert DAC/downconvert ADC, 
baseband RF DAC/baseband RF ADC, and low frequency DAC/baseband RF ADC).  So 
now there are 8 different daughterboard designs.  If we restrict ourselves to 
just quad DAC or quad ADC on a given daughtercard, then there are 5 designs, 
same as in the current proposal 

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

Actually HPC with LPC IO assignment and 8 x GTP links is popular configuration
So you have 34 LVDS pairs and 8 GTP links.

If this works for you then I don’t have major objections.  The other issue to 
consider is power rails, since for the analog circuitry we will probably want 
+/- 5V, +/- 15V as well as +3.3V, +1.8V, +12V.  Can we put these through on the 
FMC without breaking back compatibility?  For example, one rail on each of the 
4 VADJ pins?  I am sure this would not be VITA 57 compliant….and we don’t want 
switching converters on the FMC daughtercard for space and noise reasons both.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Well, it depends on trace width matching. We can simulate and characterize
it even at much higher frequencies.
If we place them really close to the DAC and match widths, remove grounds
under pads, we can match it not worse than SMA connectors.
We can even do even more crazy thing - try to make them mountable.
The only signals that would need to be transmitted is DAC output / ADC
input.
We can use low profile board to board connectors for rest of the signals -
I know ones that have 1mm stacking hight. For RF we could use PCB mounted
UFL connectors.
This is just crazy idea that need to be verified.

On 30 March 2016 at 23:02, Slichter, Daniel H. (Fed) <
daniel.slich...@nist.gov> wrote:

> This is an interesting potential solution although I am not sure how the
> signal integrity is at ~3 GHz, for example.
>
>
>
> *From:* Grzegorz Kasprowicz [mailto:kaspr...@gmail.com]
> *Sent:* Wednesday, March 30, 2016 2:44 PM
> *To:* Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
> *Cc:* Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; Leibrandt, David R.
> (Fed) <david.leibra...@nist.gov>; Sébastien Bourdeauducq <s...@m-labs.hk>;
> artiq@lists.m-labs.hk
> *Subject:* Re: [ARTIQ] FW: initial specification of the project
>
>
>
> Such assembly technique is called:
>
> castellated PCB module
>
> https://www.google.pl/search?q=castellated+RF+modules=1920=917=lnms=isch=X=0ahUKEwjRvuqko-nLAhWnnXIKHe_IARsQ_AUIBygB
> <https://www.google.pl/search?q=castellated+PCB+down+converter+modules=1920=917=lnms=isch=X=0ahUKEwj19L7younLAhUKvXIKHQVBCg8Q_AUIBigB#tbm=isch=castellated+RF+modules>
>
>
>
> On 30 March 2016 at 22:40, Grzegorz Kasprowicz <kaspr...@gmail.com> wrote:
>
> One more thing - we can fit 5 SMA connectors on FMC panel, in case of 6
> ones, we won't be able to screw them.
>
> But we can install MMCX ones for clocks and fit in total of 7 RF
> connectors,
>
> Look here
> http://www.ohwr.org/attachments/3390/fmc_top.jpg
>
> Greg
>
>
>
> On 30 March 2016 at 22:38, Grzegorz Kasprowicz <kaspr...@gmail.com> wrote:
>
> Well, we can do another crazy thing - solder small module with RF stuff on
> the FMC board, under same shield.
>
> In this way we keep 3 simple FMCs with expensive ADCs/DACs and define the
> functionality by soldering (automatic or manual) of just RF modules. WE can
> even design such modules to hold the front-end connectors of leave them on
> the FMC.
>
> Such approach has also some attractive feature - we can make them using
> small pieces of Rogers material which is hell expensive and it's hard to
> make small vias and thin traces needed for JESD signals.
>
> these modules could look like that
> http://www.emcfastpass.com/wp-content/uploads/2014/09/rf_module_holes.gif
>
> You can mount them by pick and place or manually.
>
> IT is also possible to manually disassemble them.
>
> This is a form factor of popular RF modules, i.e. wifi, GPS and LTE modems.
> http://www.emcfastpass.com/rf-modules/
>
> And is simply works
>
> Greg
>
>
>
>
>
> On 30 March 2016 at 22:25, Slichter, Daniel H. (Fed) <
> daniel.slich...@nist.gov> wrote:
>
> > Maybe we should come back to the roots:) What if we use standard FMCs
> > (LPC) with DAC/ADC channels and RF stuff _on_ them.
> > JESD204B and some pins would go to the FPGA while DAC and RF clock would
> > be fed externally.
> > In this way we leave general purpose AMC board and define its
> functionality
> > by FMC boards If we make 3 flavours of FMCs: ADC+ADC,
> > ADC+DAC,DAC+DAC, we would cover several use cases:
> > Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
> > FMCs with only DAC and RF stuff on it can be simple, 4 layer boards with
> > external sield.
> > Look at this shield (my project)
> > http://www.ohwr.org/projects/fmc-adc-130m-16b-4cha/wiki
> > In this way we could use existing AFCK for quick tests
>
> We have been working with the notion that should be many possible front
> ends for each of the DACs or ADCs, depending on what the particular
> application is, and so we want to separate the analog daughtercards from
> whatever board has the DACs and ADCs on it.  This way, you can reconfigure
> the hardware for high-frequency or low-frequency applications, for example,
> by changing daughtercards and not having to build entire new AMC cards.
> The modularity principle lets one have a single design for the AMC card
> (aka DSP card) that can be used for many different applications, by
> shifting the analog signal processing circuitry onto a separate card.
>
> Now, as you suggest we could just change the level at which we make this
> break from the AMC card, shift the DACs and ADCs o

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Actually HPC with LPC IO assignment and 8 x GTP links is popular
configuration
So you have 34 LVDS pairs and 8 GTP links.

On 30 March 2016 at 23:00, Slichter, Daniel H. (Fed) <
daniel.slich...@nist.gov> wrote:

> > On Wed, Mar 30, 2016 at 10:25 PM, Slichter, Daniel H. (Fed)
> >  wrote:
> > > Now, as you suggest we could just change the level at which we make
> this
> > break from the AMC card, shift the DACs and ADCs onto the daughter card
> as
> > well, and use FMC to communicate with the whole thing.  This makes it a
> bit
> > more expensive/difficult to reconfigure the analog front end, but the DAC
> > and ADC costs are not so high that it is impossible to do.  I had
> envisioned the
> > notion of making the daughtercards simple enough that end users could
> > redesign/respin easily to accommodate their own applications, or we could
> > ship unstuffed or partially stuffed boards that they could complete with
> the
> > particular filters etc they desire.
> >
> > It makes letting unused mezzanines collect dust on the shelf more
> > expensive.
>
> The point is you buy X number of daughtercards for Y AMC cards, where X>Y
> (perhaps X=2*Y or X=3*Y), and the daughtercards either do or don't have
> ADC/DAC on them.  If they do, it costs you more than if the ADC/DAC were on
> the AMC modules.
>
> > > However, I agree that there are compelling arguments for using the
> > architecture you propose.  We would need to pick just a few board styles
> (I
> > suggest quad DAC, 2 DAC/2 ADC, and quad ADC), and for each of these board
> > styles we would need to make several different variants with different
> > analog front ends (3 types for DAC - low frequency, baseband RF,
> > upconverted RF - and 2 types for ADC - baseband RF and downconverted RF,
> > both likely including switchable gain).  So now we are looking at making
> 3
> > types of quad DAC boards, 2 types of ADC board, and probably 3 types of
> > DAC/ADC board (upconvert DAC/downconvert ADC, baseband RF
> > DAC/baseband RF ADC, and low frequency DAC/baseband RF ADC).  So now
> > there are 8 different daughterboard designs.  If we restrict ourselves
> to just
> > quad DAC or quad ADC on a given daughtercard, then there are 5 designs,
> > same as in the current proposal for analog-only daughtercards.  I would
> still
> > want to have boards be partially stuffed (or stuffed in different
> > configurations on demand) to allow users to choose the frequencies of
> > interest for analog filters etc.
> > >
> > > If we proceed this way, we will need an external clock SMA for each FMC
> > module, because we don't want the high-quality external clock going down
> > one FMC connector, across the AMC, and up the other FMC connector for
> > signal integrity/crosstalk reasons.
> >
> > For a digital clock with fast edges 60 dB of crosstalk is _much_ less of
> a
> > problem.
>
> No!  The clock coming in will be a sine wave from a low phase noise
> oscillator somewhere.  The DACs and ADCs will threshold this clock to
> determine their sample times.  Any amount of crosstalk will distort the
> clock signal (adding or subtracting to the voltage at a given time), thus
> skewing the time at which the threshold is reached and thus inducing jitter
> into the sampling times.  This would also hold true even if you have an
> LVPECL clock signal, because at frequencies like 2.4 GHz the rise and fall
> times (~100-200 ps) are similar to that of a sine wave.  Unlike for digital
> data signals, any amount of crosstalk will degrade the jitter and/or edge
> time performance for a clock signal.
>
> > > Are we thinking we would try to implement the actual VITA 57 standard
> on
> > these connectors?  Or just use them as convenient high-speed-capable
> > connectors?  I agree with the second idea, but I don't like the first
> one.
> >
> > What from the VITA 57 pin assignmend do you not like?
>
> If you comply with VITA 57, we will need to use an HPC connector to get
> enough gigabit transceivers into the connector, and we will have to
> populate all the other digital lines.  We could use an HPC connector with
> the VITA 57 LPC connections, plus adding on the gigabit transceivers in the
> locations where they would go for an HPC connector, and thus have an LPC
> compliant connector with "extra features" for the needed gigabit
> transceivers.  I am just trying to reduce unnecessary complexity in routing
> the AMC board to the FMC connectors, since HPC is much more of a pain than
> LPC in this regard.
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> On Wed, Mar 30, 2016 at 10:25 PM, Slichter, Daniel H. (Fed)
>  wrote:
> > Now, as you suggest we could just change the level at which we make this
> break from the AMC card, shift the DACs and ADCs onto the daughter card as
> well, and use FMC to communicate with the whole thing.  This makes it a bit
> more expensive/difficult to reconfigure the analog front end, but the DAC
> and ADC costs are not so high that it is impossible to do.  I had envisioned 
> the
> notion of making the daughtercards simple enough that end users could
> redesign/respin easily to accommodate their own applications, or we could
> ship unstuffed or partially stuffed boards that they could complete with the
> particular filters etc they desire.
> 
> It makes letting unused mezzanines collect dust on the shelf more
> expensive.

The point is you buy X number of daughtercards for Y AMC cards, where X>Y 
(perhaps X=2*Y or X=3*Y), and the daughtercards either do or don't have ADC/DAC 
on them.  If they do, it costs you more than if the ADC/DAC were on the AMC 
modules.  

> > However, I agree that there are compelling arguments for using the
> architecture you propose.  We would need to pick just a few board styles (I
> suggest quad DAC, 2 DAC/2 ADC, and quad ADC), and for each of these board
> styles we would need to make several different variants with different
> analog front ends (3 types for DAC - low frequency, baseband RF,
> upconverted RF - and 2 types for ADC - baseband RF and downconverted RF,
> both likely including switchable gain).  So now we are looking at making 3
> types of quad DAC boards, 2 types of ADC board, and probably 3 types of
> DAC/ADC board (upconvert DAC/downconvert ADC, baseband RF
> DAC/baseband RF ADC, and low frequency DAC/baseband RF ADC).  So now
> there are 8 different daughterboard designs.  If we restrict ourselves to just
> quad DAC or quad ADC on a given daughtercard, then there are 5 designs,
> same as in the current proposal for analog-only daughtercards.  I would still
> want to have boards be partially stuffed (or stuffed in different
> configurations on demand) to allow users to choose the frequencies of
> interest for analog filters etc.
> >
> > If we proceed this way, we will need an external clock SMA for each FMC
> module, because we don't want the high-quality external clock going down
> one FMC connector, across the AMC, and up the other FMC connector for
> signal integrity/crosstalk reasons.
> 
> For a digital clock with fast edges 60 dB of crosstalk is _much_ less of a
> problem.

No!  The clock coming in will be a sine wave from a low phase noise oscillator 
somewhere.  The DACs and ADCs will threshold this clock to determine their 
sample times.  Any amount of crosstalk will distort the clock signal (adding or 
subtracting to the voltage at a given time), thus skewing the time at which the 
threshold is reached and thus inducing jitter into the sampling times.  This 
would also hold true even if you have an LVPECL clock signal, because at 
frequencies like 2.4 GHz the rise and fall times (~100-200 ps) are similar to 
that of a sine wave.  Unlike for digital data signals, any amount of crosstalk 
will degrade the jitter and/or edge time performance for a clock signal.  

> > Are we thinking we would try to implement the actual VITA 57 standard on
> these connectors?  Or just use them as convenient high-speed-capable
> connectors?  I agree with the second idea, but I don't like the first one.
> 
> What from the VITA 57 pin assignmend do you not like?

If you comply with VITA 57, we will need to use an HPC connector to get enough 
gigabit transceivers into the connector, and we will have to populate all the 
other digital lines.  We could use an HPC connector with the VITA 57 LPC 
connections, plus adding on the gigabit transceivers in the locations where 
they would go for an HPC connector, and thus have an LPC compliant connector 
with "extra features" for the needed gigabit transceivers.  I am just trying to 
reduce unnecessary complexity in routing the AMC board to the FMC connectors, 
since HPC is much more of a pain than LPC in this regard.  
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Such assembly technique is called:
castellated PCB module
https://www.google.pl/search?q=castellated+RF+modules=1920=917=lnms=isch=X=0ahUKEwjRvuqko-nLAhWnnXIKHe_IARsQ_AUIBygB


On 30 March 2016 at 22:40, Grzegorz Kasprowicz  wrote:

> One more thing - we can fit 5 SMA connectors on FMC panel, in case of 6
> ones, we won't be able to screw them.
> But we can install MMCX ones for clocks and fit in total of 7 RF
> connectors,
> Look here
> http://www.ohwr.org/attachments/3390/fmc_top.jpg
> Greg
>
> On 30 March 2016 at 22:38, Grzegorz Kasprowicz  wrote:
>
>> Well, we can do another crazy thing - solder small module with RF stuff
>> on the FMC board, under same shield.
>> In this way we keep 3 simple FMCs with expensive ADCs/DACs and define the
>> functionality by soldering (automatic or manual) of just RF modules. WE can
>> even design such modules to hold the front-end connectors of leave them on
>> the FMC.
>> Such approach has also some attractive feature - we can make them using
>> small pieces of Rogers material which is hell expensive and it's hard to
>> make small vias and thin traces needed for JESD signals.
>> these modules could look like that
>> http://www.emcfastpass.com/wp-content/uploads/2014/09/rf_module_holes.gif
>> You can mount them by pick and place or manually.
>> IT is also possible to manually disassemble them.
>> This is a form factor of popular RF modules, i.e. wifi, GPS and LTE
>> modems.
>> http://www.emcfastpass.com/rf-modules/
>> And is simply works
>> Greg
>>
>>
>> On 30 March 2016 at 22:25, Slichter, Daniel H. (Fed) <
>> daniel.slich...@nist.gov> wrote:
>>
>>> > Maybe we should come back to the roots:) What if we use standard FMCs
>>> > (LPC) with DAC/ADC channels and RF stuff _on_ them.
>>> > JESD204B and some pins would go to the FPGA while DAC and RF clock
>>> would
>>> > be fed externally.
>>> > In this way we leave general purpose AMC board and define its
>>> functionality
>>> > by FMC boards If we make 3 flavours of FMCs: ADC+ADC,
>>> > ADC+DAC,DAC+DAC, we would cover several use cases:
>>> > Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
>>> > FMCs with only DAC and RF stuff on it can be simple, 4 layer boards
>>> with
>>> > external sield.
>>> > Look at this shield (my project)
>>> > http://www.ohwr.org/projects/fmc-adc-130m-16b-4cha/wiki
>>> > In this way we could use existing AFCK for quick tests
>>>
>>> We have been working with the notion that should be many possible front
>>> ends for each of the DACs or ADCs, depending on what the particular
>>> application is, and so we want to separate the analog daughtercards from
>>> whatever board has the DACs and ADCs on it.  This way, you can reconfigure
>>> the hardware for high-frequency or low-frequency applications, for example,
>>> by changing daughtercards and not having to build entire new AMC cards.
>>> The modularity principle lets one have a single design for the AMC card
>>> (aka DSP card) that can be used for many different applications, by
>>> shifting the analog signal processing circuitry onto a separate card.
>>>
>>> Now, as you suggest we could just change the level at which we make this
>>> break from the AMC card, shift the DACs and ADCs onto the daughter card as
>>> well, and use FMC to communicate with the whole thing.  This makes it a bit
>>> more expensive/difficult to reconfigure the analog front end, but the DAC
>>> and ADC costs are not so high that it is impossible to do.  I had
>>> envisioned the notion of making the daughtercards simple enough that end
>>> users could redesign/respin easily to accommodate their own applications,
>>> or we could ship unstuffed or partially stuffed boards that they could
>>> complete with the particular filters etc they desire.
>>>
>>> However, I agree that there are compelling arguments for using the
>>> architecture you propose.  We would need to pick just a few board styles (I
>>> suggest quad DAC, 2 DAC/2 ADC, and quad ADC), and for each of these board
>>> styles we would need to make several different variants with different
>>> analog front ends (3 types for DAC - low frequency, baseband RF,
>>> upconverted RF - and 2 types for ADC - baseband RF and downconverted RF,
>>> both likely including switchable gain).  So now we are looking at making 3
>>> types of quad DAC boards, 2 types of ADC board, and probably 3 types of
>>> DAC/ADC board (upconvert DAC/downconvert ADC, baseband RF DAC/baseband RF
>>> ADC, and low frequency DAC/baseband RF ADC).  So now there are 8 different
>>> daughterboard designs.  If we restrict ourselves to just quad DAC or quad
>>> ADC on a given daughtercard, then there are 5 designs, same as in the
>>> current proposal for analog-only daughtercards.  I would still want to have
>>> boards be partially stuffed (or stuffed in different 

Re: [ARTIQ] FW: initial specification of the project

[Robert Jördens]

On Wed, Mar 30, 2016 at 10:25 PM, Slichter, Daniel H. (Fed)
 wrote:
> Now, as you suggest we could just change the level at which we make this 
> break from the AMC card, shift the DACs and ADCs onto the daughter card as 
> well, and use FMC to communicate with the whole thing.  This makes it a bit 
> more expensive/difficult to reconfigure the analog front end, but the DAC and 
> ADC costs are not so high that it is impossible to do.  I had envisioned the 
> notion of making the daughtercards simple enough that end users could 
> redesign/respin easily to accommodate their own applications, or we could 
> ship unstuffed or partially stuffed boards that they could complete with the 
> particular filters etc they desire.

It makes letting unused mezzanines collect dust on the shelf more expensive.

> However, I agree that there are compelling arguments for using the 
> architecture you propose.  We would need to pick just a few board styles (I 
> suggest quad DAC, 2 DAC/2 ADC, and quad ADC), and for each of these board 
> styles we would need to make several different variants with different analog 
> front ends (3 types for DAC - low frequency, baseband RF, upconverted RF - 
> and 2 types for ADC - baseband RF and downconverted RF, both likely including 
> switchable gain).  So now we are looking at making 3 types of quad DAC 
> boards, 2 types of ADC board, and probably 3 types of DAC/ADC board 
> (upconvert DAC/downconvert ADC, baseband RF DAC/baseband RF ADC, and low 
> frequency DAC/baseband RF ADC).  So now there are 8 different daughterboard 
> designs.  If we restrict ourselves to just quad DAC or quad ADC on a given 
> daughtercard, then there are 5 designs, same as in the current proposal for 
> analog-only daughtercards.  I would still want to have boards be partially 
> stuffed (or stuffed in different configurations on demand) to allow users to 
> choose the frequencies of interest for analog filters etc.
>
> If we proceed this way, we will need an external clock SMA for each FMC 
> module, because we don't want the high-quality external clock going down one 
> FMC connector, across the AMC, and up the other FMC connector for signal 
> integrity/crosstalk reasons.

For a digital clock with fast edges 60 dB of crosstalk is _much_ less
of a problem.

> Are we thinking we would try to implement the actual VITA 57 standard on 
> these connectors?  Or just use them as convenient high-speed-capable 
> connectors?  I agree with the second idea, but I don't like the first one.

What from the VITA 57 pin assignmend do you not like?

Robert.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Well, we can do another crazy thing - solder small module with RF stuff on
the FMC board, under same shield.
In this way we keep 3 simple FMCs with expensive ADCs/DACs and define the
functionality by soldering (automatic or manual) of just RF modules. WE can
even design such modules to hold the front-end connectors of leave them on
the FMC.
Such approach has also some attractive feature - we can make them using
small pieces of Rogers material which is hell expensive and it's hard to
make small vias and thin traces needed for JESD signals.
these modules could look like that
http://www.emcfastpass.com/wp-content/uploads/2014/09/rf_module_holes.gif
You can mount them by pick and place or manually.
IT is also possible to manually disassemble them.
This is a form factor of popular RF modules, i.e. wifi, GPS and LTE modems.
http://www.emcfastpass.com/rf-modules/
And is simply works
Greg

On 30 March 2016 at 22:25, Slichter, Daniel H. (Fed) <
daniel.slich...@nist.gov> wrote:

> > Maybe we should come back to the roots:) What if we use standard FMCs
> > (LPC) with DAC/ADC channels and RF stuff _on_ them.
> > JESD204B and some pins would go to the FPGA while DAC and RF clock would
> > be fed externally.
> > In this way we leave general purpose AMC board and define its
> functionality
> > by FMC boards If we make 3 flavours of FMCs: ADC+ADC,
> > ADC+DAC,DAC+DAC, we would cover several use cases:
> > Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
> > FMCs with only DAC and RF stuff on it can be simple, 4 layer boards with
> > external sield.
> > Look at this shield (my project)
> > http://www.ohwr.org/projects/fmc-adc-130m-16b-4cha/wiki
> > In this way we could use existing AFCK for quick tests
>
> We have been working with the notion that should be many possible front
> ends for each of the DACs or ADCs, depending on what the particular
> application is, and so we want to separate the analog daughtercards from
> whatever board has the DACs and ADCs on it.  This way, you can reconfigure
> the hardware for high-frequency or low-frequency applications, for example,
> by changing daughtercards and not having to build entire new AMC cards.
> The modularity principle lets one have a single design for the AMC card
> (aka DSP card) that can be used for many different applications, by
> shifting the analog signal processing circuitry onto a separate card.
>
> Now, as you suggest we could just change the level at which we make this
> break from the AMC card, shift the DACs and ADCs onto the daughter card as
> well, and use FMC to communicate with the whole thing.  This makes it a bit
> more expensive/difficult to reconfigure the analog front end, but the DAC
> and ADC costs are not so high that it is impossible to do.  I had
> envisioned the notion of making the daughtercards simple enough that end
> users could redesign/respin easily to accommodate their own applications,
> or we could ship unstuffed or partially stuffed boards that they could
> complete with the particular filters etc they desire.
>
> However, I agree that there are compelling arguments for using the
> architecture you propose.  We would need to pick just a few board styles (I
> suggest quad DAC, 2 DAC/2 ADC, and quad ADC), and for each of these board
> styles we would need to make several different variants with different
> analog front ends (3 types for DAC - low frequency, baseband RF,
> upconverted RF - and 2 types for ADC - baseband RF and downconverted RF,
> both likely including switchable gain).  So now we are looking at making 3
> types of quad DAC boards, 2 types of ADC board, and probably 3 types of
> DAC/ADC board (upconvert DAC/downconvert ADC, baseband RF DAC/baseband RF
> ADC, and low frequency DAC/baseband RF ADC).  So now there are 8 different
> daughterboard designs.  If we restrict ourselves to just quad DAC or quad
> ADC on a given daughtercard, then there are 5 designs, same as in the
> current proposal for analog-only daughtercards.  I would still want to have
> boards be partially stuffed (or stuffed in different configurations on
> demand) to allow users to choose the frequencies of interest for analog
> filters etc.
>
> If we proceed this way, we will need an external clock SMA for each FMC
> module, because we don't want the high-quality external clock going down
> one FMC connector, across the AMC, and up the other FMC connector for
> signal integrity/crosstalk reasons.
>
> Are we thinking we would try to implement the actual VITA 57 standard on
> these connectors?  Or just use them as convenient high-speed-capable
> connectors?  I agree with the second idea, but I don't like the first one.
>
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> Maybe we should come back to the roots:) What if we use standard FMCs
> (LPC) with DAC/ADC channels and RF stuff _on_ them.
> JESD204B and some pins would go to the FPGA while DAC and RF clock would
> be fed externally.
> In this way we leave general purpose AMC board and define its functionality
> by FMC boards If we make 3 flavours of FMCs: ADC+ADC,
> ADC+DAC,DAC+DAC, we would cover several use cases:
> Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
> FMCs with only DAC and RF stuff on it can be simple, 4 layer boards with
> external sield.
> Look at this shield (my project)
> http://www.ohwr.org/projects/fmc-adc-130m-16b-4cha/wiki
> In this way we could use existing AFCK for quick tests

We have been working with the notion that should be many possible front ends 
for each of the DACs or ADCs, depending on what the particular application is, 
and so we want to separate the analog daughtercards from whatever board has the 
DACs and ADCs on it.  This way, you can reconfigure the hardware for 
high-frequency or low-frequency applications, for example, by changing 
daughtercards and not having to build entire new AMC cards.  The modularity 
principle lets one have a single design for the AMC card (aka DSP card) that 
can be used for many different applications, by shifting the analog signal 
processing circuitry onto a separate card.  

Now, as you suggest we could just change the level at which we make this break 
from the AMC card, shift the DACs and ADCs onto the daughter card as well, and 
use FMC to communicate with the whole thing.  This makes it a bit more 
expensive/difficult to reconfigure the analog front end, but the DAC and ADC 
costs are not so high that it is impossible to do.  I had envisioned the notion 
of making the daughtercards simple enough that end users could redesign/respin 
easily to accommodate their own applications, or we could ship unstuffed or 
partially stuffed boards that they could complete with the particular filters 
etc they desire.  

However, I agree that there are compelling arguments for using the architecture 
you propose.  We would need to pick just a few board styles (I suggest quad 
DAC, 2 DAC/2 ADC, and quad ADC), and for each of these board styles we would 
need to make several different variants with different analog front ends (3 
types for DAC - low frequency, baseband RF, upconverted RF - and 2 types for 
ADC - baseband RF and downconverted RF, both likely including switchable gain). 
 So now we are looking at making 3 types of quad DAC boards, 2 types of ADC 
board, and probably 3 types of DAC/ADC board (upconvert DAC/downconvert ADC, 
baseband RF DAC/baseband RF ADC, and low frequency DAC/baseband RF ADC).  So 
now there are 8 different daughterboard designs.  If we restrict ourselves to 
just quad DAC or quad ADC on a given daughtercard, then there are 5 designs, 
same as in the current proposal for analog-only daughtercards.  I would still 
want to have boards be partially stuffed (or stuffed in different 
configurations on demand) to allow users to choose the frequencies of interest 
for analog filters etc.

If we proceed this way, we will need an external clock SMA for each FMC module, 
because we don't want the high-quality external clock going down one FMC 
connector, across the AMC, and up the other FMC connector for signal 
integrity/crosstalk reasons.  

Are we thinking we would try to implement the actual VITA 57 standard on these 
connectors?  Or just use them as convenient high-speed-capable connectors?  I 
agree with the second idea, but I don't like the first one.  
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Maybe we should come back to the roots:)
What if we use standard FMCs (LPC) with DAC/ADC channels and RF stuff _on_ them.
JESD204B and some pins would go to the FPGA while DAC and RF clock would be fed 
externally.
In this way we leave general purpose AMC board and define its functionality by 
FMC boards
If we make 3 flavours of FMCs: ADC+ADC, ADC+DAC,DAC+DAC, we would cover several 
use cases:
Quad ADC, quad DAC, 1xADC 3xDAC, 3xADC 1xDAC.
FMCs with only DAC and RF stuff on it can be simple, 4 layer boards with 
external sield.
Look at this shield (my project)
http://www.ohwr.org/projects/fmc-adc-130m-16b-4cha/wiki
In this way we could use existing AFCK for quick tests

-Original Message-
From: Slichter, Daniel H. (Fed) [mailto:daniel.slich...@nist.gov] 
Sent: Wednesday, March 30, 2016 5:46 PM
To: Leibrandt, David R. (Fed) <david.leibra...@nist.gov>; Sébastien 
Bourdeauducq <s...@m-labs.hk>; Grzegorz Kasprowicz <kaspr...@gmail.com>
Cc: 'Grzegorz Kasprowicz' <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
Subject: RE: [ARTIQ] FW: initial specification of the project

> I like this plan.  I think 4 + 4 channels will also make the front 
> panel connector density more reasonable.  What are you thinking for 
> number of daughter cards?  I suppose that more would give us more 
> flexibility, but less would be more economical in terms of cost and 
> layout area.  Perhaps two daughter cards would be reasonable: one for 
> all of the inputs and one for all of the outputs?

Agreed that ~ 8 front panel SMA connections, plus one clock SMA connection, is 
about what one can tolerate in a reasonable way in terms of physical footprint. 
 If we split this as 4 DAC + 4 ADC, it makes a nice symmetry although my 
suspicion is that most applications would prefer something more asymmetric with 
a few more DAC channels than ADC channels (6 DAC/2 ADC or 6 DAC/4 ADC, for 
example). One could go a simple as 4 DAC/2 ADC and make the space requirements 
even simpler to fulfill on the cards.  All of these modifications will increase 
the price per channel, even though we may be able to save on FPGA costs.  For 
example, if we do 4 DAC/2 ADC on a card with 1x AD9154 and 1x ADC16DX370 (our 
currently planned parts), we only need 10 GTX transceivers on the FPGA.  With 4 
DAC/4 ADC, this would be 12 GTX transceivers.  With these numbers we could look 
at the ZU5EV Zynq Ultrascale, instead of the ZU9EG, which we had been 
discussing.

To Greg's question on the RTM, we have had a number of extensive internal 
discussions about pros and cons of RTM and it seems that we don't want to 
pursue this avenue for a number of carefully considered reasons.  Let's just 
take it as a given that we will need to put everything on the AMC card or its 
analog daughtercards.  As stated, the DAC and ADC chips themselves will be on 
the AMC card, not the analog daughtercards.  

I am in favor of separate analog daughtercards for the inputs and the outputs, 
this seems sensible.  


> On Tuesday, 29 March 2016 11:55:19 PM HKT Grzegorz Kasprowicz wrote:
> > - they can be used immediately with existing OSHW carriers like AFC/AFCK.
> 
> AFCK may be overkill. We are still working on evaluating the FPGA 
> resource requirements.
> 
> > - it could be hard to fit FPGA, supply, DACs and several RF modules, 
> > all on single dual width AMC, especially when shielding is required.
> > RTM relaxes these constraints
> > - on AMC+RTM you can place 8 ADC channels + 8 DAC channels + 8 RF 
> > modules. In case of single AMC board it would be hard to achieve 
> > such channels density.
> 
> How about this:
> * we reduce the number of channels per AMC to 4 DACs + 4 ADCs
> * we can therefore use a smaller FPGA. Communication lanes to the 
> master board are relatively cheap if we put them on IOSERDES.
> * the power density and cooling requirements are also reduced.
> * for the RF daughter cards, we use a custom form factor that can use 
> at least
> 2/3 of the AMC front panel
> * connectors between the DSP card and the RF daughter card are 2mm 
> header and 8x SMP
> * the rest of the AMC front panel used for (optional, runtime 
> selectable) clock input and some TTLs.



___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Are we talking about double width AMCs?
Two DAC channels and 2 RF modules should fit.
Greg

-Original Message-
From: Sébastien Bourdeauducq [mailto:s...@m-labs.hk] 
Sent: Wednesday, March 30, 2016 5:46 PM
To: Leibrandt, David R. (Fed) <david.leibra...@nist.gov>
Cc: Grzegorz Kasprowicz <kaspr...@gmail.com>; 'Grzegorz Kasprowicz'
<gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk; Slichter, Daniel H. (Fed)
<daniel.slich...@nist.gov>
Subject: Re: [ARTIQ] FW: initial specification of the project

On Wednesday, 30 March 2016 3:15:59 PM HKT Leibrandt, David R. (Fed) wrote:
> What are you thinking for number of daughter cards?  I suppose that 
> more would give us more flexibility, but less would be more economical 
> in terms of cost and layout area.  Perhaps two daughter cards would be
reasonable:
> one for all of the inputs and one for all of the outputs?

Just one, the space on the AMC is rather limited.

As for splitting inputs and outputs - maybe, but then it makes sense to
allow resource sharing and/or space to be redistributed within one card.
Similar to double FMC, e.g.
http://www.euvis.com/products/mod/fmc/image/FMC2653.JPG

Sébastien


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

On Tuesday, 29 March 2016 11:14:14 PM HKT Grzegorz Kasprowicz wrote:
> [GK] If you don't use ARM, you still get hardened SDRAM controller and 
> GBE MACs.

Yes, that's what I was saying: you cannot get rid of them (i.e. use their
pins like other IOs). So you need to use the Zynq-specific features of
Vivado, interface your design to the hardened AXI system, use some obscure
Vivado "wizard" to set up the SDRAM, and write a software driver for
Xilinx's GbE MAC. All doable, but annoying.

[GK]
Well, you don't have to write it.
It is already available for RTOS and linux.
But it's true - it occupies MIO bank and dedicated DDR port. But this is axi
and can be easily accessible from PL part.
How many IOs do you need?

> > * we want to avoid RTMs and instead put the DAC/ADCs on the AMC card 
> > and have analog plug-ins using the FMC form factor (see my document).
> > **Are you sure you would get noise performance from such setup that 
> > satisfies you?
> 
> Unless the FMC connector is particularly bad with analog signals, I 
> think it should not be worse than the current hardware. [GK] All 
> depends how you deliver the clock for such DAC. This is the weakest point
of such solution.

The DACs will be mounted on the DSP cards (AMC directly), not the RF
daughtercards. And the DSP cards will have an external clocking option, that
may use semi-rigid coax if need be.

[GK]
OK, if you make clock splitter and deliver the clocks for both DACs and RF
modules externally, this should work.
Another issue are ground loops, so clock and RF SMA outputs may need to be
isolated.


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

Hi all
I like that idea - in this way we can do it really affordable with lower end
FPGAs and still compatible with MTCA.
If we place all RF stuff on the modules, with RF connectors mounted directly
on the front panel, this could work.
How we would deliver the clocks? Using external SMA connectors?
The only think I worry about is performance of the DACs.
ZynQ US+ can run IOserdes with almost 2Gbit/s performance - one can make
1000base-x directly on IO pins using oversampling technique.
Greg

-Original Message-
From: Sébastien Bourdeauducq [mailto:s...@m-labs.hk] 
Sent: Wednesday, March 30, 2016 1:50 PM
To: Grzegorz Kasprowicz <kaspr...@gmail.com>
Cc: 'Slichter, Daniel H. (Fed)' <daniel.slich...@nist.gov>; 'Grzegorz
Kasprowicz' <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specification of the project

On Tuesday, 29 March 2016 11:55:19 PM HKT Grzegorz Kasprowicz wrote:
> - they can be used immediately with existing OSHW carriers like AFC/AFCK. 

AFCK may be overkill. We are still working on evaluating the FPGA resource
requirements.

> - it could be hard to fit FPGA, supply, DACs and several RF modules, 
> all on single dual width AMC, especially when shielding is required. 
> RTM relaxes these constraints
> - on AMC+RTM you can place 8 ADC channels + 8 DAC channels + 8 RF 
> modules. In case of single AMC board it would be hard to achieve such 
> channels density.

How about this:
* we reduce the number of channels per AMC to 4 DACs + 4 ADCs
* we can therefore use a smaller FPGA. Communication lanes to the master
board are relatively cheap if we put them on IOSERDES.
* the power density and cooling requirements are also reduced.
* for the RF daughter cards, we use a custom form factor that can use at
least
2/3 of the AMC front panel
* connectors between the DSP card and the RF daughter card are 2mm header
and 8x SMP
* the rest of the AMC front panel used for (optional, runtime selectable)
clock input and some TTLs.

Sébastien


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Robert Jördens]

On Wed, Mar 30, 2016 at 5:57 PM, Slichter, Daniel H. (Fed)
 wrote:
>> > What are you thinking for number of daughter cards?  I suppose that
>> > more would give us more flexibility, but less would be more economical
>> > in terms of cost and layout area.  Perhaps two daughter cards would be
>> reasonable:
>> > one for all of the inputs and one for all of the outputs?
>>
>> Just one, the space on the AMC is rather limited.
>
> We should be able to do two side by side.  I think it would be nice to be 
> able to select the input daughtercard separately from the output 
> daughtercard.  This would require having two sets of pin headers for sending 
> power and digital signals, but I think the input daughtercards would likely 
> have much less in terms of digital control signal requirements; there would 
> be two basic designs, one for baseband digitization and one for 
> downconversion before digitization.  Both designs would just have analog 
> components, no digital attenuators/switches/etc would be necessary or desired.

To be useful as a digital servo platform the analog inputs need to
have quite a bit of configurability to them e.g. switchable gain.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> > What are you thinking for number of daughter cards?  I suppose that
> > more would give us more flexibility, but less would be more economical
> > in terms of cost and layout area.  Perhaps two daughter cards would be
> reasonable:
> > one for all of the inputs and one for all of the outputs?
> 
> Just one, the space on the AMC is rather limited.

We should be able to do two side by side.  I think it would be nice to be able 
to select the input daughtercard separately from the output daughtercard.  This 
would require having two sets of pin headers for sending power and digital 
signals, but I think the input daughtercards would likely have much less in 
terms of digital control signal requirements; there would be two basic designs, 
one for baseband digitization and one for downconversion before digitization.  
Both designs would just have analog components, no digital 
attenuators/switches/etc would be necessary or desired.  

This would especially hold true if we go to a 4 DAC / 2 ADC design, but a 4/4 
design should work fine too.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> I like this plan.  I think 4 + 4 channels will also make the front panel 
> connector
> density more reasonable.  What are you thinking for number of daughter
> cards?  I suppose that more would give us more flexibility, but less would be
> more economical in terms of cost and layout area.  Perhaps two daughter
> cards would be reasonable: one for all of the inputs and one for all of the
> outputs?

Agreed that ~ 8 front panel SMA connections, plus one clock SMA connection, is 
about what one can tolerate in a reasonable way in terms of physical footprint. 
 If we split this as 4 DAC + 4 ADC, it makes a nice symmetry although my 
suspicion is that most applications would prefer something more asymmetric with 
a few more DAC channels than ADC channels (6 DAC/2 ADC or 6 DAC/4 ADC, for 
example). One could go a simple as 4 DAC/2 ADC and make the space requirements 
even simpler to fulfill on the cards.  All of these modifications will increase 
the price per channel, even though we may be able to save on FPGA costs.  For 
example, if we do 4 DAC/2 ADC on a card with 1x AD9154 and 1x ADC16DX370 (our 
currently planned parts), we only need 10 GTX transceivers on the FPGA.  With 4 
DAC/4 ADC, this would be 12 GTX transceivers.  With these numbers we could look 
at the ZU5EV Zynq Ultrascale, instead of the ZU9EG, which we had been 
discussing.

To Greg's question on the RTM, we have had a number of extensive internal 
discussions about pros and cons of RTM and it seems that we don't want to 
pursue this avenue for a number of carefully considered reasons.  Let's just 
take it as a given that we will need to put everything on the AMC card or its 
analog daughtercards.  As stated, the DAC and ADC chips themselves will be on 
the AMC card, not the analog daughtercards.  

I am in favor of separate analog daughtercards for the inputs and the outputs, 
this seems sensible.  


> On Tuesday, 29 March 2016 11:55:19 PM HKT Grzegorz Kasprowicz wrote:
> > - they can be used immediately with existing OSHW carriers like AFC/AFCK.
> 
> AFCK may be overkill. We are still working on evaluating the FPGA resource
> requirements.
> 
> > - it could be hard to fit FPGA, supply, DACs and several RF modules,
> > all on single dual width AMC, especially when shielding is required.
> > RTM relaxes these constraints
> > - on AMC+RTM you can place 8 ADC channels + 8 DAC channels + 8 RF
> > modules. In case of single AMC board it would be hard to achieve such
> > channels density.
> 
> How about this:
> * we reduce the number of channels per AMC to 4 DACs + 4 ADCs
> * we can therefore use a smaller FPGA. Communication lanes to the master
> board are relatively cheap if we put them on IOSERDES.
> * the power density and cooling requirements are also reduced.
> * for the RF daughter cards, we use a custom form factor that can use at least
> 2/3 of the AMC front panel
> * connectors between the DSP card and the RF daughter card are 2mm
> header and 8x SMP
> * the rest of the AMC front panel used for (optional, runtime selectable)
> clock input and some TTLs.


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Leibrandt, David R. (Fed)]

I like this plan.  I think 4 + 4 channels will also make the front panel 
connector density more reasonable.  What are you thinking for number of 
daughter cards?  I suppose that more would give us more flexibility, but less 
would be more economical in terms of cost and layout area.  Perhaps two 
daughter cards would be reasonable: one for all of the inputs and one for all 
of the outputs?

Best,
Dave

-Original Message-
From: ARTIQ [mailto:artiq-boun...@lists.m-labs.hk] On Behalf Of Sébastien 
Bourdeauducq
Sent: Wednesday, March 30, 2016 5:50 AM
To: Grzegorz Kasprowicz <kaspr...@gmail.com>
Cc: 'Grzegorz Kasprowicz' <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk; 
Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
Subject: Re: [ARTIQ] FW: initial specification of the project

On Tuesday, 29 March 2016 11:55:19 PM HKT Grzegorz Kasprowicz wrote:
> - they can be used immediately with existing OSHW carriers like AFC/AFCK. 

AFCK may be overkill. We are still working on evaluating the FPGA resource 
requirements.

> - it could be hard to fit FPGA, supply, DACs and several RF modules, 
> all on single dual width AMC, especially when shielding is required. 
> RTM relaxes these constraints
> - on AMC+RTM you can place 8 ADC channels + 8 DAC channels + 8 RF 
> modules. In case of single AMC board it would be hard to achieve such 
> channels density.

How about this:
* we reduce the number of channels per AMC to 4 DACs + 4 ADCs
* we can therefore use a smaller FPGA. Communication lanes to the master board 
are relatively cheap if we put them on IOSERDES.
* the power density and cooling requirements are also reduced.
* for the RF daughter cards, we use a custom form factor that can use at least
2/3 of the AMC front panel
* connectors between the DSP card and the RF daughter card are 2mm header and 
8x SMP
* the rest of the AMC front panel used for (optional, runtime selectable) clock 
input and some TTLs.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Tuesday, 29 March 2016 11:14:14 PM HKT Grzegorz Kasprowicz wrote:
> [GK] If you don't use ARM, you still get hardened SDRAM controller and GBE
> MACs.

Yes, that's what I was saying: you cannot get rid of them (i.e. use their pins 
like other IOs). So you need to use the Zynq-specific features of Vivado, 
interface your design to the hardened AXI system, use some obscure Vivado 
"wizard" to set up the SDRAM, and write a software driver for Xilinx's GbE 
MAC. All doable, but annoying.

> > * we want to avoid RTMs and instead put the DAC/ADCs on the AMC card
> > and have analog plug-ins using the FMC form factor (see my document).
> > **Are you sure you would get noise performance from such setup that
> > satisfies you?
> 
> Unless the FMC connector is particularly bad with analog signals, I think it
> should not be worse than the current hardware. [GK] All depends how you
> deliver the clock for such DAC. This is the weakest point of such solution.

The DACs will be mounted on the DSP cards (AMC directly), not the RF 
daughtercards. And the DSP cards will have an external clocking option, that 
may use semi-rigid coax if need be.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

On Friday, 25 March 2016 12:24:02 PM HKT you wrote:
> * whether or not we use Zynq remains to be decided.
> **The price difference is not that high (a few tens of $) and we get 
> plenty of CPU power

Yes, but Zynq chips are annoying to program (even if we do not use the ARM
cores) and more proprietary. The question is really whether the Zynq 
architecture has technical benefits to the end application that justify the 
trouble. The answer is not straightforward, e.g. CPU power doesn't translate 
linearly to latency reductions.

[GK] If you don't use ARM, you still get hardened SDRAM controller and GBE MACs.

Maybe we should do a quick experiment and map the ARTIQ RTIO core in the 
address space of a Zynq device, and compile (with just gcc or clang, not the 
ARTIQ compiler) the equivalent of the current runtime+kernel system that 
measures the maximum pulse rate. Then we have hard numbers to discuss.

[GK]For time critical apps, one can program these CPUs bare metal. We did it in 
our projects.

> * transceivers on the backplane are acceptable but not necessary; 
> since we have multiple signals we can transmit a clock and use the 
> regular IOSERDES to lower FPGA requirements.
> **Well, IOSERDES can go up to 1.2Gbit n synchronous mode.
> **GTH can go 12Gbit/s easily
> **Are you sure you want to relay only on IOSERDES?

Yes, I think they would be sufficient. The bulk of the data will be processed 
locally on-the-fly by the AMC cards, and a couple Gbps (I suppose we can route 
extra pairs easily) of bandwidth should be plenty for controlling the AMCs.

> * what is the other large BGA chip on the MCH rendering? Is it the 
> Macom
> M21048 48x48 crossbar? Let's not use such a device, which looks 
> unnecessary, complicated, and very proprietary. There isn't even a 
> public datasheet for it.
> We do not need direct communication between the AMCs.
> 
> **At the moment not, but remember that it's going to be research 
> platform

If we use the IOSERDES, the FPGA has plenty of IO pins to perform such direct 
routing (on wider buses and/or with less bandwidth) .

[GK] That' true

> **I can make an option with this chip not mounted. I simply see 
> another potential of this board and the place on the PCB does not cost 
> single $ since we have to use standard module **dimensions:)

OK, but what about the additional routing difficulty, and any additional PCB 
layers?

[GK] It will need 2 extra layers

What is the other application you are thinking of?

[GK] The other application is not related to ARTIQ project - we will need it 
for another project at the WUT related with GEM detectors.
[GK] Generally, it is detector processing electronics

> **In case of the datasheet, I got it easily after I signed the NDA.

The fewer NDAs I sign, the better. And it would make the board less accessible.
[GK] I understand your point. It is not compatible with Open Hardware

> * we want to avoid RTMs and instead put the DAC/ADCs on the AMC card 
> and have analog plug-ins using the FMC form factor (see my document).
> **Are you sure you would get noise performance from such setup that 
> satisfies you?

Unless the FMC connector is particularly bad with analog signals, I think it 
should not be worse than the current hardware.
[GK] All depends how you deliver the clock for such DAC. This is the weakest 
point of such solution.
[GK] Is current HW fully sufficient in terms of SFDR?

Greg


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Grzegorz Kasprowicz]

For slow control signals and supplies one can use standard 2.54 golpins and 
headers. 
I saw such solution in the RF group at the university. They pass RF using 
dedicated board to board coaxial connectors, rest is transmitted over standard 
pin headers.
Greg

-Original Message-
From: ARTIQ [mailto:artiq-boun...@lists.m-labs.hk] On Behalf Of Slichter, 
Daniel H. (Fed)
Sent: Monday, March 28, 2016 5:25 PM
To: Sébastien Bourdeauducq <s...@m-labs.hk>
Cc: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
Subject: Re: [ARTIQ] FW: initial specification of the project

The beauty of SMP connectors is that they are explicitly designed for 
board-to-board applications with substantial tolerance for misalignment.  Each 
board has a male SMP connector, with a female-female "bullet" in between.  
These connectors, with the bullet in between, can tolerate substantial axial 
and radial misalignment (in excess of .010" - basically considerably larger 
than the fabrication tolerances of typical PCBs and PCB assembly) without 
degrading RF performance.  Then one would have one multipin connector for power 
and digital signals that fixes the board-to-board alignment, and the SMP 
connectors can absorb the slack as necessary based on fabrication/assembly 
tolerances for their placement.  

I agree that if we can get suitable performance out of an FMC or other 
single-connector solution, that would be better from a physical simplicity 
standpoint.  We need to test this.  I may try putting together a board to do so 
in the near term.  

> -Original Message-
> From: Sébastien Bourdeauducq [mailto:s...@m-labs.hk]
> Sent: Saturday, March 26, 2016 10:19 AM
> To: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
> Cc: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; 
> artiq@lists.m-labs.hk
> Subject: Re: [ARTIQ] FW: initial specification of the project
> 
> On Saturday, 26 March 2016 4:06:17 PM HKT Slichter, Daniel H. (Fed) wrote:
> > The cost savings from using FMC, which might amount to $50 per AMC, 
> > are not worth if the crosstalk will make the cards not useful for 
> > researchers.
> 
> It's not only about cost of the connector - the RF daughter cards will 
> often need some digital signals (e.g. SPI) for control. FMC provides 
> plenty of pins for that. Mixing two different types of board-to-board 
> connectors will cause mechanical problems and I think we should not do 
> that. If two types of connectors are needed, one of them should use cables.
> 
> Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> FMC has mounting height of 8 or 10mm.
> Other heights are also possible in case of SEARAY series, but one must order
> large number of them Greg

FMC per VITA 57.1 standard is 8.5 mm stacking height or 10 mm stacking height 
(see section 3.4).  You can get the SEARAY connectors (of which FMC compliant 
connectors are a subset) in a variety of other stacking heights too.  

At the end of the day, I really think that something simple and straightforward 
like pin headers is going to work well and will save us a lot of hassle in 
finding specialized connectors/worrying about longevity/connector 
availability/etc.


> -Original Message-
> From: ARTIQ [mailto:artiq-boun...@lists.m-labs.hk] On Behalf Of Slichter,
> Daniel H. (Fed)
> Sent: Tuesday, March 29, 2016 10:39 PM
> To: Robert Jördens <r...@m-labs.hk>
> Cc: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
> Subject: Re: [ARTIQ] FW: initial specification of the project
> 
> Let me make clear that I don't have any specific opposition to FMC for the
> power/digital signals.  The only reason for considering other types of
> connectors would be either if the desired stackup height is not doable (this 
> is
> unlikely -- one .224" bullet e.g. corning A1A1-0001-02, plus two male
> connetors with .051" reference plane distance, e.g. Molex 74315-3312, gives
> a total SMP height of 8.28 mm, which works perfectly with the standard FMC
> stackup height of 8.5 mm board to board), or if there is not enough real
> estate on the board to meet the physical footprint requirements (FMC is
> larger than the QSE connectors I sent along, for example).
> 
> > -Original Message-
> > From: Robert Jördens [mailto:r...@m-labs.hk]
> > Sent: Tuesday, March 29, 2016 11:08 AM
> > To: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
> > Cc: Sébastien Bourdeauducq <s...@m-labs.hk>; Grzegorz Kasprowicz
> > <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
> > Subject: Re: [ARTIQ] FW: initial specification of the project
> >
> > On Tue, Mar 29, 2016 at 5:16 PM, Slichter, Daniel H. (Fed)
> > <daniel.slich...@nist.gov> wrote:
> > > Yes, FMC could work but it's overkill in terms of pin count; one
> > > might be
> > able to find a smaller footprint connector with fewer pins, which
> > would be advantageous.  Frankly something as dumb and cheap as 2 mm
> > pin headers would do the job.
> >
> > Yes. A pin header would be dumb if there are better suggestions.
> > LPC FMC with 64 single ended signals is definitely not overkill if 40
> > signals is the estimated need. The grounding is unlikely to hurt.
> > There are certainly many other offers. But if there is such a strong
> > opposition to FMC, I would look for something where we can at least
> > expect a long term availability that is similar to the FMC usage lifespan.
> ___
> ARTIQ mailing list
> https://ssl.serverraum.org/lists/listinfo/artiq

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

Let me make clear that I don't have any specific opposition to FMC for the 
power/digital signals.  The only reason for considering other types of 
connectors would be either if the desired stackup height is not doable (this is 
unlikely -- one .224" bullet e.g. corning A1A1-0001-02, plus two male connetors 
with .051" reference plane distance, e.g. Molex 74315-3312, gives a total SMP 
height of 8.28 mm, which works perfectly with the standard FMC stackup height 
of 8.5 mm board to board), or if there is not enough real estate on the board 
to meet the physical footprint requirements (FMC is larger than the QSE 
connectors I sent along, for example).  

> -Original Message-
> From: Robert Jördens [mailto:r...@m-labs.hk]
> Sent: Tuesday, March 29, 2016 11:08 AM
> To: Slichter, Daniel H. (Fed) <daniel.slich...@nist.gov>
> Cc: Sébastien Bourdeauducq <s...@m-labs.hk>; Grzegorz Kasprowicz
> <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
> Subject: Re: [ARTIQ] FW: initial specification of the project
> 
> On Tue, Mar 29, 2016 at 5:16 PM, Slichter, Daniel H. (Fed)
> <daniel.slich...@nist.gov> wrote:
> > Yes, FMC could work but it's overkill in terms of pin count; one might be
> able to find a smaller footprint connector with fewer pins, which would be
> advantageous.  Frankly something as dumb and cheap as 2 mm pin headers
> would do the job.
> 
> Yes. A pin header would be dumb if there are better suggestions.
> LPC FMC with 64 single ended signals is definitely not overkill if 40 signals 
> is
> the estimated need. The grounding is unlikely to hurt.
> There are certainly many other offers. But if there is such a strong 
> opposition
> to FMC, I would look for something where we can at least expect a long term
> availability that is similar to the FMC usage lifespan.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> > OK. Then mixing SMP with something else is fine IMO.

>

> The other connector can well be FMC. We need at least ~40 signals other

> than the analog ones going to the cards. A bunch of different power supplies,

> SPI control lines, identification buses, switching, attenuation settings etc.

> Also since reliably reflowing SMP connectors manually to three hair widths is

> rather tricky, manual assembly is of the table now anyway. And FMC give us

> at least a form factor that has been tested very well. No need to reinvent the

> mezzanine here.



Yes, FMC could work but it's overkill in terms of pin count; one might be able 
to find a smaller footprint connector with fewer pins, which would be 
advantageous.  Frankly something as dumb and cheap as 2 mm pin headers would do 
the job.



Manual assembly was never really one of the major concerns with these 
daughtercards, as many of the desired components are leadless or QFN packages 
with center ground pads, which are tricky for manual assembly anyway, 
especially if one cares about microwave performance.  However, many SMP 
connectors have a through hole design (see below for an example from Corning 
Gilbert, or from Molex at http://www.molex.com/pdm_docs/sd/734153320_sd.pdf ), 
so it's entirely possible to achieve the kind of lateral alignment tolerances 
required with manual assembly.  Cost-wise, the molex part quoted is less than 
$3 apiece on digikey at qty 750 (and $5 at qty 1), so it’s not like these 
connectors will be a major cost driver on the cards either.



[cid:image001.png@01D1899A.9C6CA5F0]
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Slichter, Daniel H. (Fed)]

> If 65 dB between neighboring channels is the requirement, then
> comprehensive board level shielding appears to be required.

Yes, this will be necessary.  See my previous emails.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Robert Jördens]

On Tue, Mar 29, 2016 at 11:56 AM, Sébastien Bourdeauducq  wrote:
> On Monday, 28 March 2016 6:30:52 PM HKT Slichter, Daniel H. (Fed) wrote:
>> Thus for the two examples above, using digital connectors with a 9 mm or 11
>> mm total stackup height would give 250 um axial misalignment (.010"), which
>> is a typical tolerance one would want to use anyway with SMP.
>>
>> More information is available in this application note:
>> https://www.corning.com/media/worldwide/coc/documents/applications/microwave
>> ApplicationNotes.pdf
>
> OK. Then mixing SMP with something else is fine IMO.

The other connector can well be FMC. We need at least ~40 signals
other than the analog ones going to the cards. A bunch of different
power supplies, SPI control lines, identification buses, switching,
attenuation settings etc. Also since reliably reflowing SMP connectors
manually to three hair widths is rather tricky, manual assembly is of
the table now anyway. And FMC give us at least a form factor that has
been tested very well. No need to reinvent the mezzanine here.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Robert Jördens]

On Mon, Mar 28, 2016 at 9:14 PM, Slichter, Daniel H. (Fed)
 wrote:
> http://suddendocs.samtec.com/testreports/hsc-report-seam-seaf-07mm_web.pdf
>
> There is too much crosstalk in FMC connectors (~40-65 dB typical @ 3 GHz) for 
> us to use them for the RF/analog daughterboards.  Recommend dedicated RF 
> connectors (typical crosstalk better than 80 dB) for analog signal 
> transmission.

If 65 dB between neighboring channels is the requirement, then
comprehensive board level shielding appears to be required.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq

Re: [ARTIQ] FW: initial specification of the project

[Sébastien Bourdeauducq]

On Saturday, 26 March 2016 4:06:17 PM HKT Slichter, Daniel H. (Fed) wrote:
> The cost savings from using FMC, which might amount to $50 per AMC, are not
> worth if the crosstalk will make the cards not useful for researchers.  

It's not only about cost of the connector - the RF daughter cards will often 
need some digital signals (e.g. SPI) for control. FMC provides plenty of pins 
for that. Mixing two different types of board-to-board connectors will cause 
mechanical problems and I think we should not do that. If two types of 
connectors are needed, one of them should use cables.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq