[ARTIQ] closing mailing lists on Dec. 24

2020-12-16 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

since nobody is using these mailing lists anymore (these days the 
preferred channels seem to be GitHub/Gitea issues, IRC/mattermost, and 
the forum), I will close them next week, Dec 24th - unless someone objects.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ-5 released

2019-11-14 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

ARTIQ-5 is released today. To update, follow the stable branch manual at 
https://m-labs.hk/artiq/manual/installing.html.


Highlights of this new release (compared to 4.0):

* Performance improvements:
  - Faster RTIO event submission (1.5x improvement in pulse rate test)
See: https://github.com/m-labs/artiq/issues/636
  - Faster compilation times (3 seconds saved on kernel compilation 
time on a typical medium-size experiment)
See: 
https://github.com/m-labs/artiq/commit/611bcc4db4ed604a32d9678623617cd50e968cbf

* Improved packaging and build system:
  - new continuous integration/delivery infrastructure based on Nix and 
Hydra, providing reproducibility, speed and independence.

  - rolling release process (https://github.com/m-labs/artiq/issues/1326).
  - firmware, gateware and device database templates are automatically 
built for all supported Kasli variants.

  - new JSON description format for generic Kasli systems.
  - Nix packages are now supported.
  - many Conda problems worked around.
  - controllers are now out-of-tree.
  - split packages that enable lightweight applications that 
communicate with ARTIQ,  e.g. controllers running on non-x86 
single-board computers.

* Improved Urukul support:
  - AD9910 RAM mode.
  - Configurable refclk divider and PLL bypass.
  - More reliable phase synchronization at high sample rates.
  - Synchronization calibration data can be read from EEPROM.
* A gateware-level input edge counter has been added, which offers 
higher throughput and increased flexibility over the usual TTL input 
PHYs where edge timestamps are not required. See 
`artiq.coredevice.edge_counter` for the core device driver and 
`artiq.gateware.rtio.phy.edge_counter`/`artiq.gateware.eem.DIO.add_std` 
for the gateware components.

* With DRTIO, Siphaser uses a better calibration mechanism.
  See: 
https://github.com/m-labs/artiq/commit/cc58318500ecfa537abf24127f2c22e8fe66e0f8

* Schedule updates can be sent to influxdb (artiq_influxdb_schedule).
* Experiments can now programatically set their default pipeline, 
priority, and flush flag.
* List datasets can now be efficiently appended to from experiments 
using `artiq.language.environment.HasEnvironment.append_to_dataset`.

* The core device now supports IPv6.
* To make development easier, the bootloader can receive firmware and 
secondary FPGA gateware from the network.

* Python 3.7 compatibility (Nix and source builds only, no Conda).
* Various other bugs from 4.0 fixed.
* Preliminary Sayma v2 and Metlino hardware support.

Breaking changes:

* The `artiq.coredevice.ad9910.AD9910` and
  `artiq.coredevice.ad9914.AD9914` phase reference timestamp parameters
  have been renamed to ``ref_time_mu`` for consistency, as they are in 
machine units.

* The controller manager now ignores device database entries without the
  ``"command"`` key set to facilitate sharing of devices between 
multiple masters.
* The meaning of the ``-d/--dir`` and ``--srcbuild`` options of 
``artiq_flash`` has changed.

* Controllers for third-party devices are now out-of-tree.
* ``aqctl_corelog`` now filters log messages below the ``WARNING`` level 
by default.

* On Kasli the firmware now starts with a unique default MAC address
  from EEPROM if `mac` is absent from the flash config.
* The ``-e/--experiment`` switch of ``artiq_run`` and ``artiq_compile``
  has been renamed ``-c/--class-name``.
* ``artiq_devtool`` has been removed.
* Much of ``artiq.protocols`` has been moved to a separate package 
``sipyco``.``artiq_rpctool`` has been renamed to ``sipyco_rpctool``.
* ``artiq_ctlmgr`` and the influxdb tools have moved to a separate 
package ``artiq-comtools`` (normally installed by default).


(Also posted on: https://forum.m-labs.hk/d/51-artiq-5-released)

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] ARTIQ 5 release timeline?

2019-09-12 Thread Sébastien Bourdeauducq via ARTIQ
On 9/12/19 11:05 PM, Andrew Risinger via ARTIQ wrote:
> Is there a timeline for the release of ARTIQ 5?

https://github.com/m-labs/artiq/milestone/14
The main item is Sayma v2 support.

> Also, is there any reason that the conda builds only still support
> python >=3.5.3 <3.6, when Nix supports python 3.7?

Yes, the poor quality of conda, which makes it frustratingly difficult
to upgrade Python.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] web forum

2019-05-28 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

I have set up a web-based forum as another place (that some may find 
friendlier and easier to use) to discuss all things ARTIQ, (n)Migen, 
MiSoC and HeavyX with the community.


Visit it here: https://forum.m-labs.hk/

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Frame grabber

2019-01-31 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

The Grabber card is not compatible with CoaXpress.

Using CoaXpress with Kasli should be possible via its SATA connector, 
but requires nontrivial development in hardware and gateware.


Sébastien


On 2/1/19 12:02 AM, Harry Parke via ARTIQ wrote:

Dear ARTIQ list members,


Does anybody know if the Frame Grabber Interface is only compatible with 
Camera Link or whether anything can be done so that it works with other 
fast data transfer interfaces such as CoaXpress?



Many thanks,

Harry Parke

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] status report

2018-12-11 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

The attached PDF covers the work since the last report sent on October 9th.

Sébastien



2018-November-December.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Elementary problems with Kasli

2018-10-01 Thread Sébastien Bourdeauducq via ARTIQ

On 10/01/2018 10:30 PM, Hanhijärvi Kalle via ARTIQ wrote:

On Kasli, I'm using SFP0 port for the fiber.


Is the LED next to SFP0 turned on? That's the Ethernet connection status 
indicator.

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] monthly report

2018-08-09 Thread Sébastien Bourdeauducq via ARTIQ

Please see the attached PDF.


2018-August.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] SAWG

2018-08-09 Thread Sébastien Bourdeauducq via ARTIQ

On Thursday, August 09, 2018 03:12 PM, Thomas Harty wrote:
So, the questions are: how much do we need to simplify the SAWG to make 
it okay to debug and maintain at the 1GSPS data rate? and, what's the 
way of doing that, which has the least impact on users?


"In anything at all, perfection is finally attained not when there is no 
longer anything to add, but when there is no longer anything to take away".

 - Antoine de Saint Exupéry
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] SAWG

2018-08-08 Thread Sébastien Bourdeauducq via ARTIQ

On Thursday, August 09, 2018 03:15 AM, Thomas Harty via ARTIQ wrote:
it's a big FPGA and IIRC we're not 
really pushing the resources limits yet (but maybe I'm wrong about 
that?), so it's not clear that's actually a problem. I get that 
multi-hour compile times are death, but at least we haven't seen any 
Sayma bugs that depend on whether the Sawg is enabled or not for quite a 
while (since we fixed the power supplies). So, maybe it's not actually 
such an issue if the majority of Sawg testing can be done in simulation, 
and any non-SAWG issues on Sayma can be debugging in non-SAWG builds?


Let's not get too optimistic. There can be difficult problems with 
timing closure and routing congestion, which are already a bit sensitive 
with the current design. Those cannot be solved with a partial design 
only. Also, even if there are no more Sayma-bugs, simulation/hardware 
mismatches might still happen (due to Vivado miscompilations, Verilog 
simulation issues, or Migen bugs). Having a smaller, faster to compile 
design is valuable.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] ARTIQ Digest, Vol 50, Issue 2

2018-07-13 Thread Sébastien Bourdeauducq via ARTIQ

On Wednesday, July 11, 2018 04:59 AM, Thomas Harty via ARTIQ wrote:

My view is that we shouldn't give up the flexibility of being able to
 fine-tune the DUC frequency unless there is a good reason to do so.
For example: if the complexity/compile times of the current code make
 development/maintenance problematic(*), or if the current code is
going to have issues achieving the full 1GSPS data rate. It would be
good to hear a bit more from SB/RJ on the advantages of moving to a
simpler DUC parametrization.


Right now we are producing 4 DAC samples per cycle (150MHz RTIO, 600MHz 
DAC). Moving to 1GSPS will need 8 samples per cycle (125MHz RTIO), i.e. 
requiring twice the FPGA resources plus adding interconnection paths 
between the parallel units. This can only exacerbate issues of 
compilation time, routing congestion, and timing closure. The last two 
can probably be addressed but there is no free lunch - it'll take 
significant work. And the compilation time would always remain problematic.


Having the fixed DUC frequencies would simplify the sample generation 
logic and reduce the problems.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Sayma DAC frequencies

2018-07-08 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

I'm trying to determine what is the best way forward to support sample 
rates better than the current 600MHz with the Sayma DAC and SAWG.


What sample rate(s) would you like to see and why?

With high sample rates, there are two ways to ease the FPGA resource burden:
* use the DAC interpolation modes (2X, 4X, 8X) if the goal is simply to 
improve spectral purity.
* drastically reduce the SAWG digital upconverter resolution to a few 
frequencies (use the other NCOs to for fine tuning).


Are those acceptable? We have a large FPGA, but high resource 
utilization means long compilation times and potential difficulties to 
meet timing, so it is still a good idea to make the design as light as 
possible.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Sayma input frequency

2018-06-18 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

any objections to supporting only the RTIO clock frequency (currently 
150MHz) at the Sayma input, instead of 100MHz?

Are you using non-programmable 100MHz references?

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] We have ARTIQ RF out of Sayma!

2018-05-23 Thread Sébastien Bourdeauducq via ARTIQ

On Thursday, May 24, 2018 02:16 AM, Joe Britton wrote:
Has M-Labs tested SAWG on Sayma since the DRAM, Ethernet, gateware bug 
patching in recent months?


Sure, I did it just before sending the board to Duke two weeks ago. Why?
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] monthly status report

2018-05-05 Thread Sébastien Bourdeauducq via ARTIQ

Please see the attached PDF.


2018-May.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] March 2018 report

2018-03-05 Thread Sébastien Bourdeauducq via ARTIQ

Please see the attached PDF.


2018-March.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 3.4 released

2018-02-05 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

We have released ARTIQ 3.4 to fix an intermittent core device crash (#902).

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Single-board ARTIQ/Kasli system now operational

2018-01-15 Thread Sébastien Bourdeauducq via ARTIQ

Hi everyone,

We have completed the ARTIQ core development for Kasli in single-board 
configurations (i.e. without DRTIO). This includes DDR3 support, and 
1000BASE-X Ethernet PHY using Artix-7 GTP transceivers. The full ARTIQ 
runtime works properly on the board and is ready to execute kernels over 
Ethernet. Development of the Urukul driver (both AD9910 and AD9912) is 
under way and progressing rapidly; a sizable part of it can already be 
found in the repository.


Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] January 2018 report

2018-01-03 Thread Sébastien Bourdeauducq via ARTIQ

Here is the current report. Happy new year everyone!

Sébastien



2018-January.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] SFP/SATA cables for connecting Ethernet on Sayma

2017-11-13 Thread Sébastien Bourdeauducq via ARTIQ

Hi Greg,

just a quick note about the SFP/SATA cable that is necessary to connect 
Ethernet on a Sayma directly. I suggest building them from a passive SFP 
copper cable (e.g. http://www.fs.com/products/36649.html) cut in half, 
with a male (motherboard or disk) connector soldered at the end.


Some switches or media converters require a EEPROM or the LOS signal 
that the copper cable provides, and the male connector is more solid 
mechanically than soldering a SATA cable inside a SFP.


The cable I got from you had its transceiver line connections damaged, 
plus it cannot work on my media converter due to the missing EEPROM or 
LOS (not sure which - LOS would be relatively easy to fix, just connect 
to ground). The FS.com cable above is properly detected.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] test

2017-11-06 Thread Sébastien Bourdeauducq via ARTIQ

Please disregard this message.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] November status report

2017-11-06 Thread Sébastien Bourdeauducq via ARTIQ

Please see the attached PDF.


2017-November.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] scans and generators on core device?

2017-10-20 Thread Sébastien Bourdeauducq via ARTIQ

On Friday, October 20, 2017 12:36 AM, Slichter, Daniel H. (Fed) wrote:

Judging from the absence of replies to this email, we will not
support generators on the core device nor MultiScanManager.

My main question with this is about time efficiency -- if you were to
go to the effort to support this on the core device
Note that scans can be supported on the device with just iterators and 
not generators (as explained in the issue).


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] scans and generators on core device?

2017-10-18 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

Judging from the absence of replies to this email, we will not support 
generators on the core device nor MultiScanManager.


Sébastien


On Tuesday, October 10, 2017 02:34 PM, Sébastien Bourdeauducq wrote:

Hi,

to implement scans on the core device 
(https://github.com/m-labs/artiq/issues/118) in the best way possible, 
we need some information about how ARTIQ is used and will be used:
* are Python generators (i.e. using "yield") something that you know 
about, use, and would like to see supported inside kernels?

* are you using MultiScanManager?

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] scans and generators on core device?

2017-10-10 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

to implement scans on the core device 
(https://github.com/m-labs/artiq/issues/118) in the best way possible, 
we need some information about how ARTIQ is used and will be used:
* are Python generators (i.e. using "yield") something that you know 
about, use, and would like to see supported inside kernels?

* are you using MultiScanManager?

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Instantiating tri-state buffer in migen

2017-10-06 Thread Sébastien Bourdeauducq via ARTIQ

On Saturday, October 07, 2017 09:13 AM, Arpit Agrawal via ARTIQ wrote:

 return Instance("IOBUFDS",
 i_I=self.i, o_O=self.o, i_T=self.oe,


OE means "output enable". T means "tristate", i.e. not driving. You need 
to invert that signal.


Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] October status report

2017-10-03 Thread Sébastien Bourdeauducq via ARTIQ

Please see the attached PDF.


2017-October.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 2.5 released

2017-10-02 Thread Sébastien Bourdeauducq via ARTIQ

Hello,

We have just released ARTIQ 2.5 and new conda packages are available. 
This is a bugfix release that you can use if you do not wish to move to 
ARTIQ-3 yet.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 3.0 released

2017-09-29 Thread Sébastien Bourdeauducq via ARTIQ

Hello,

After a year since the last major release, we are pleased to announce 
ARTIQ 3.0.


There were ~1300 commits since 2.0, for many different features such as 
RTIO DMA that can dramatically improve the throughput of long pulse 
sequences, and asynchronous RPCs to speed up the reporting results from 
the core device, and dashboard applet control from experiments.


ARTIQ-3 contains demonstrations on the KC705 board for two new major 
features that will be fully developed in ARTIQ-4 on the Sinara hardware: 
distributed RTIO, that enables multi-FPGA many-channel RTIO systems, and 
the SAWG, a "DDS on steroids" using multi-GSPS DACs connected directly 
to the FPGA.


We made major improvements to the PDQ waveform generator, which requires 
ARTIQ-3 but now lives in its own repository:

https://github.com/m-labs/pdq

The core device runtime saw a complete rewrite in Rust, and it now uses 
a new TCP/IP stack that we developed that should fix stability and 
performance issues encountered with lwIP.


The complete commit history is at
https://github.com/m-labs/artiq/commits/release-3

Functional changes that merit attention and may require user action are 
described in the release notes:

https://m-labs.hk/artiq/manual-release-3/release_notes.html

We recommend that users of ARTIQ 2.Y upgrade to 3.0. We plan to release 
ARTIQ 2.5 soon, after which we will cease maintainance on the 2.Y releases.


The pre-built packages are available under the "main" conda label.
Instructions on how to get started with ARTIQ are at
https://m-labs.hk/artiq/manual-release-3/installing.html

Due to conda problems, 32-bit Windows packages are no longer available, 
and may come back after Python 3.6 support lands (#652).


We do not plan to add features to the release-3 branch and subsequent
3.Y release that change behavior or APIs: the focus will be on bugs.
We will continue to track those at:
https://github.com/m-labs/artiq/issues

As always, please report issues and bugs through the usual channels.

Meanwhile work is continuing towards ARTIQ 4.0 and several new
features are already implemented. ARTIQ-4 will contain support for the 
new Sinara hardware (Sayma, Metlino, Kasli and their peripherals) plus a 
new, more scalable RTIO architecture (#778).


Please feel free to forward this message to other interested users.

The ARTIQ team.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] September status report

2017-09-04 Thread Sébastien Bourdeauducq via ARTIQ

Please see the attached PDF.


2017-September.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Should we use StackExchange?

2017-08-13 Thread Sébastien Bourdeauducq via ARTIQ

Joe,

Why is this better than the mailing list? And why add another place to 
get support?


On Sunday, August 13, 2017 05:10 PM, Joe Britton via ARTIQ wrote:

- news and topics of community-wide interest
(https://ssl.serverraum.org/lists/listinfo/artiq)


The mailing list was never intended for news only, otherwise not every 
subscriber could post.


Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] August 2017 status report

2017-08-02 Thread Sébastien Bourdeauducq via ARTIQ

Please see the attached PDF.


2017-August.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Kasli FPGA selection

2017-07-01 Thread Sébastien Bourdeauducq via ARTIQ

On Friday, June 30, 2017 07:54 PM, Grzegorz Kasprowicz via ARTIQ wrote:

additional 30$ does not make any difference.


OK, fine.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] ARTIQ Digest, Vol 37, Issue 6

2017-06-29 Thread Sébastien Bourdeauducq via ARTIQ

On Thursday, June 29, 2017 06:16 PM, Thomas Harty via ARTIQ wrote:

The fact that going for a 75T/100T gives us access to 12EEMs/Kasli (4
on the BP) rather than 10EEMs/Kasli (only 2 on the BP) for the 50T is
an added benefit.
Kasli was meant to be a simple and low-cost board without a backplane, 
and you are now using the backplane as an argument...

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Kasli FPGA selection

2017-06-28 Thread Sébastien Bourdeauducq via ARTIQ

On Wednesday, June 28, 2017 04:52 PM, Thomas Harty wrote:
Have we settled on the 50T as the FPGA for the first version of Kasli, 
and what speed grade?


I would advocate for the 50T in -2 speed grade for two main reasons:
a) I don't think we need that much FPGA resources for the 100T to be needed.
b) -2 speed grade transceivers go to 6.25Gbps whereas -1 speed grade 
ones go to 3.75Gbps. In addition to a significant increase in bandwidth, 
the -2 transceivers can use the same configuration on the Metlino/Sayma 
side which is used for the backplane (5Gbps). Otherwise we would have to 
generate another set of Ultrascale transceiver settings (and shave a 
yak) and potentially deal with weird RTIO frequency ratios in a hybrid 
MTCA/Eurocard Sinara system.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Fwd: North American Conference on Trapped Ions

2017-05-18 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

I would like to relay the information about this upcoming conference on 
trapped ions organized at NIST Boulder. M-Labs will be participating 
with an exhibit (including some Sinara boards) and the hosting of a 
networking event and panel discussion at Sanitas Brewery one evening. 
The panel will be on the future of control systems for quantum 
information experiments.


Conference registration has recently opened and costs US$431 catered, 
US$250 non-catered.


Abstract deadline: July 1
Registration deadline: August 1
Conference dates: August 14-18

The conference would be a great opportunity for the ARTIQ community to 
get together. We are looking forward to seeing you there!


Sébastien


 Forwarded Message 
Subject:North American Conference on Trapped Ions
Date:   Sun, 19 Feb 2017 17:40:35 +
From:   Allcock, David T. (IntlAssoc) 
To: Wilson, Andrew C. (Fed) 



Dear Members of the International Ion Trap Community,

The Ion Storage group at NIST Boulder will be holding a new Ion Trap 
Conference at our laboratory.  The conference will be similar in scope 
and aims to the European Conference on Trapped Ions (ECTI) but with a 
corresponding emphasis on North American research.  We hope for this to 
become a regular series, running on alternate years to ECTI.  If you are 
interested in attending, please add the following dates to your diary:


1st North American Conference on Trapped Ions
NIST Boulder Campus, Colorado USA
August 14-18 2017
https://www.nist.gov/news-events/events/2017/08/1st-north-american-conference-trapped-ions

More details and a registration link will be posted on the website in 
about a month and we will email you again at that time. Until then 
please feel free to make suggestions and pass this email on to 
colleagues and team members who may be interested.


Sincerely,
David Allcock & Andrew Wilson
(For the NIST Ions)

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 2.3 released

2017-04-23 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

ARTIQ 2.3 is available and fixes various bugs that were present in 2.2. 
We encourage all users to update to 2.3. When using conda, make sure to 
add the conda-forge channel before updating, as ARTIQ now depends on the 
new pyqtgraph 0.10 package available there.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] [SINARA] hardware arrived!

2017-03-17 Thread Sébastien Bourdeauducq via ARTIQ

On Friday, March 17, 2017 07:34 AM, Grzegorz Kasprowicz via ARTIQ wrote:

look here
https://cloud.githubusercontent.com/assets/4325054/24015076/98f7653a-0a87-11e7-93d2-7df1831b2422.jpg


Looks nice! Thanks for all your work!
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] [RFC] remove RTIOCollision

2017-03-15 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

RTIOCollision is a bit tedious to implement with DRTIO, since the master 
does not know if a given channel should do replace or collision. The 
satellite would need to report this information for each of its channels 
(and this also needs to be passed from the gateware scripts to the 
satellite manager firmware), then the master should program it into its 
gateware. Do we strongly need it, or can we simply have the replace 
behavior at all times? According to this mailing list discussion, the 
replace behavior is actually wanted:

https://ssl.serverraum.org/lists-archive/artiq/2016-November/001052.html

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] starter ARTIQ hardware for neutral atoms

2017-03-08 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

On Wednesday, March 08, 2017 06:11 AM, Neal Pisenti via ARTIQ wrote:

* For ARTIQ core device, we would ideally jump straight to using a
Kasli, but as that isn't likely to be done in the next few months, I was
planning to use a KC705 as the core.


The "EEM" DDS/synth Kasli extensions may not necessarily be ready before 
Kasli. So I don't see how the KC705 helps - is it because you want more 
extensions that one Kasli would support? Supporting this KC705 scheme is 
more gateware development and one more configuration that needs to be 
documented, packaged and maintained. Maintainance means that we need to 
check regularly (preferably automatically) that it keeps working when we 
modify ARTIQ and fix any bugs that pop up. It takes work.



* KC705 has 2x FMC headers, which would drive 1x VHDCI carrier card,
providing 8x IDC for EEMs. We would buy FMC -> VHDCI adapters for this
interconnect.


What adapters in particular? http://www.ohwr.org/projects/fmc-vhdci? We 
didn't check compatibility of any of those.



**Specific questions**:

* what limitations are there (latency/bandwidth/etc) on daisy-chaining
additional Kasili DRTIO modules off of the single KC705 SFP?


While the hardware could do it, daisy-chaining Kaslis is not supported
by the current gateware plans. The plan is to use a Metlino, which has
many available transceiver links (mostly to the microTCA backplane, but 
there are also 3 SFPs), as a central device with a direct link to every 
satellite device. If daisy-chains are implemented, there would

be virtually no impact on bandwidth, and the latency would increase by
roughly ~100-200ns per hop.

Instead of the daisy chain, it is also possible to have one Kasli as 
central device driving directly other Kaslis with its SFPs (note that 
one SFP will be used for Ethernet). There are no current gateware plans 
for this either (so this would need funding), but it is less difficult 
to develop and has less latency.



* Is there an estimate on the timescale for finished Kasli?


It is not funded yet, but I think this should happen in a few months. 
Then there will be another few months before it begins to become usable.


Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 2.2 released

2017-01-31 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

ARTIQ 2.2 is available, and contains bugfixes (SPI and some compiler 
corner cases) and a relicensing under LGPL. The new license resolves 
concerns about experiments being potentially considered derived works 
under the GPL. We encourage all users to update to 2.2.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] ARTIQ users meeting

2016-12-25 Thread Sébastien Bourdeauducq via ARTIQ

On Monday, December 19, 2016 10:08 PM, Slichter, Daniel H. (Fed) via
ARTIQ wrote:

It seems that JQI would be a good potential location because:


Sounds good. Jonathan, Joe, Jason - what do you think about hosting such 
a meeting?



* Would you present something?

Depending on the structure and goals of the conference, the NIST
group would likely be interested in presenting as appropriate.


Noted, thanks for your support.

Any other groups?

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] plans for clock chip, JESD and DAC initialization/configuration

2016-12-16 Thread Sébastien Bourdeauducq via ARTIQ
On Saturday, December 17, 2016 12:02 PM, Sébastien Bourdeauducq via 
ARTIQ wrote:

Before DRTIO can operate, the clock chip (HMC* on Sayma and AD9516 on
KC705) needs to be running.


Strictly speaking: this is needed only for the two-KC705 system. But we 
might as well use the same scheme everywhere, and it avoids the corner 
case of operating the kernel CPU with no RTIO clock running.


The generic chip configuration code should go in firmware/libbsp.

With the RTM FPGA, SPI will have to go over the SERDES link. I'm still 
thinking about a generic "I/O expander" similar to mini-DRTIO; we can 
have half-rate, quarter-rate, etc. updates for some pins in order to 
save bandwidth.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] plans for clock chip, JESD and DAC initialization/configuration

2016-12-16 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

Before DRTIO can operate, the clock chip (HMC* on Sayma and AD9516 on 
KC705) needs to be running.


This setup should be done by the comms CPU on the DRTIO master, and the 
management CPU on a DRTIO satellite.


For initialization, the comms or management CPU would configure the 
clock chips and bring up the JESD links. The clock chip and JESD code 
currently in 
https://github.com/m-labs/artiq/blob/master/artiq/examples/phaser/startup_kernel.py 
should be moved to the Rust runtime and the Rust DRTIO satellite manager.


The SPI core would be connected to the comms CPU.

It seems desirable to alter DAC settings in the experiment. Perhaps this 
feature can be deferred. When we need it, it can be done as follows:
* the kernel CPU sends a request to the comms CPU via the mailbox. Since 
the comms CPU already knows about the DAC as it needs to initialize it, 
the request should be on a similar level of abstraction, i.e. it should 
be something like "enable mix mode" and not "write X to SPI register Y".
* if the DAC is on the local board, the comms CPU performs the SPI 
transaction.
* if the DAC is on a remote board, the comms CPU sends an auxiliary 
DRTIO packet to the relevant board, and its management CPU performs the 
SPI transaction, then sends an acknowledgement auxiliary packet back.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ users meeting

2016-12-16 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

The idea of organizing an in-person meeting for current and prospective 
ARTIQ users has been floating for a while. We'd like to get a 
conversation started on this topic. Some of this email is taken from 
Joe's GitHub issue #582.


* Venue? some ideas: NIST, JQI, DESY, Oxford, CERN, Warsaw University of 
Technology, Chinese University of Hong Kong.
* It would be convenient to have it before or after an existing physics 
conference (e.g. APS DAMOP 2017 is June 5-9 in Sacramento, CA)


* We need organizers taking care of logistics, publicity, conference 
program, social activities, catering.

* M-Labs can hold several tutorials/workshops.

* What would you like to see at such a conference?
* Would you present something?
* Would you attend a conference if it were on your continent? Another 
continent?


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 2.1 released

2016-12-11 Thread Sébastien Bourdeauducq via ARTIQ

Hi,

ARTIQ 2.1 is out and is a simple bugfix release. We recommend that all 
2.0 users upgrade.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] test

2016-11-30 Thread Sébastien Bourdeauducq via ARTIQ
--- Begin Message ---

Another test

On Thursday, December 01, 2016 01:24 PM, Sébastien Bourdeauducq via 
ARTIQ wrote:

Testing new Mailman options that hopefully will stop triggerring the
NIST/Microsoft email spoofing detector. Can someome from NIST reply to
this message and confirm that they don't get "This sender failed our
fraud detection checks" messages anymore?
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq
--- End Message ---
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] test

2016-11-30 Thread Sébastien Bourdeauducq via ARTIQ
Testing new Mailman options that hopefully will stop triggerring the 
NIST/Microsoft email spoofing detector. Can someome from NIST reply to 
this message and confirm that they don't get "This sender failed our 
fraud detection checks" messages anymore?

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] [RFC] remove output event replacement feature

2016-11-24 Thread Sébastien Bourdeauducq

On Thursday, November 24, 2016 06:24 PM, Robert Jördens wrote:

How do you want to do the replacement?
Optimizing the sequence in advance in CPU/runtime would be
channel-dependent and mode-dependent special casing.
Doing so in gateware before it enters the DMA buffer would require
gateware there that depends on channel features on remote DRTIO
devices.


All software, and yes there would be some channel-based special cases. I 
don't see advantages to writing DMA buffers in gateware, only inconvenience.

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] [RFC] remove output event replacement feature

2016-11-23 Thread Sébastien Bourdeauducq

On Thursday, November 24, 2016 04:54 AM, Srinivas, Raghavendra
(IntlAssoc) wrote:

Replacing is very channel dependent. It can only happen on certain
data and certain state of the channel. Not generally. It needs to be
fine tuned for each channel. And it needs to happen at the input side
of the RTIO FIFO. In DRTIO we can't have such channel dependent
gateware close to the kernel CPU. Whether a channel supports
replacing is done by adding to the RTIO CSR API a logical "address"
that discriminates replaceable "register data". That additional
address field directly impacts and increases the amount of data that
is written on each RTIO event submission. We expect a significant
speed up in event rate by removing it. It also bloats DRTIO packets.


Well there is nothing unmanageable there.
* If we keep your "reset address CSR on timestamp modification" feature, 
those writes that always have address zero (e.g. set TTL level) can use 
a different syscall that ignores the address and saves the associated 
CPU overhead.
* In DRTIO packets, the address field can be transmitted in a handful of 
nanoseconds.
* For replacements in DRTIO, we can just do it on the remote side. The 
inefficiency of sending multiple packets is fine as the throughput will 
be limited by the CPU anyway. For DMA, we can optimize the sequences in 
advance to remove internal replacements.
* The remote side also has a synchronous FIFO which is easier to handle 
than the asynchronous FIFO of local RTIO.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Sinara multi-crate / DRTIO switches

2016-11-08 Thread Sébastien Bourdeauducq

Joe,

I think some clarification is badly required about what DRTIO does and 
does not.


DRTIO gives you:
1) time transfer
2) low-latency low-level control of remote RTIO channels
3) an auxiliary low-bandwidth low-priority general-purpose data channel 
(which can be used for moninj, flashing boards, monitoring temperature, 
etc.)


It is *not* a general-purpose networking or distributed computing protocol.

On Tuesday, November 08, 2016 11:13 PM, Joe Britton wrote:

Crossing each switch will incur 100ns-200ns of latency


This has implications for some experiments. 10 m (10 km) fiber
propagation is 48 ns (48 us). Demonstration experiments involving
heralded entanglement of a pair of nodes (2 crates) have a low
probability of success (~1e-6) and are repeated continuously (~1 MHz).


Why does it have to be 2 crates? Are the hundreds of channels of a 
single crate not enough to drive a few ion traps? You'll have slow 
entanglement in your system at some point anyway as you plan to go long 
distances.



1) slower response times.
2) blocking the kernel CPU by twice the latency (round-trip) when it needs to 
enquire about the space available in a remote RTIO FIFO.


Any implementation that requires round-trip communication to complete
DRTIO is very bad due to fiber/free-space propagation delays. To first
order all DRTIO should assume receiving devices are ready to receive
and handle errors by a) reporting to master crate b) logging for
post-processing. To second order it's fine for low-traffic advisory
signaling like "FIFO 80% full." Plan for future deployments where
communication propagation delays are 100's us.


I advise against running DRTIO over such high-latency links. Even if we 
find all sorts of clever tricks to hide the latency in the "write to a 
remote FIFO" case, any branching would unavoidably require a roundtrip. 
Even toggling an output TTL in response to an input TTL edge would take 
2x 100's us.


Instead the nodes should have more autonomy (e.g. contain their own 
CPUs) and the links should be just time transfer + general purpose 
networking, i.e. White Rabbit. (The reasons we don't do DRTIO over White 
Rabbit are latency, Ethernet overhead for small packets, and 
difficulties in prioritizing traffic)


> A current implementation using soft-core switching seems an adequate
> compromise provided the system is designed in such a way that a future
> gateware implementation is straight forward.


In anticipation of a future all-gateware implementation of DRTIO
routing is use of a dedicated soft-core CPU helpful?


Not at all.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Sinara multi-crate / DRTIO switches

2016-11-05 Thread Sébastien Bourdeauducq

On Saturday, November 05, 2016 09:57 PM, Grzegorz Kasprowicz wrote:

We can use multiple 10Gbit links in parallel between Metlinos


That's doable, but we'd have to write the gateware for inter-lane 
synchronization while keeping deterministic latency, which is a bit tricky.

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Sinara multi-crate / DRTIO switches

2016-11-05 Thread Sébastien Bourdeauducq

Hi,

does anyone have serious plans to use more than one Sinara crate?

A crate already contains one Metlino and up to 12 Sayma cards, which 
means 96 DAC and 96 ADC channels. A Metlino could also be connected to 
several Kaslis (if the Kasli ends up being made).


Multi-crate configurations require slightly complicated gateware support 
for DRTIO switches. The bandwidth between crates will also be limited to 
10Gbps. Crossing each switch will incur 100ns-200ns of latency which 
will manifest itself in two significant ways:

1) slower response times.
2) blocking the kernel CPU by twice the latency (round-trip) when it 
needs to enquire about the space available in a remote RTIO FIFO.


There is currently a plan to support multi-crate in the hardware (this 
future-proofing simply means adding some SFPs, essentially) but no plan 
to support it in the gateware.


DRTIO switch support is also beneficial to the serial protocol between 
the Sayma AMC and Sayma RTM FPGAs - the current plan is to use a dumb 
protocol that doesn't have good timing resolution and is inefficient for 
things like SPI transfers, essentially a more open version of Channel 
Link II.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] shared SPI clock

2016-10-28 Thread Sébastien Bourdeauducq

On Saturday, October 29, 2016 12:00 AM, Sébastien Bourdeauducq wrote:

The last line assumes that a non-transmitting device (weakly) pulls down
MISO.


You can enable a pull-down resistor inside the FPGA.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] shared SPI clock

2016-10-28 Thread Sébastien Bourdeauducq

On Friday, October 28, 2016 11:55 PM, Jonathan Mizrahi wrote:

How do you recommend I do that?


Something like:
self.comb += [
  mosi_dev1.eq(mosi_spi_core),
  mosi_dev2.eq(mosi_spi_core),
  miso_spi_core.eq(miso_dev_1 | miso_dev_2)
]

The last line assumes that a non-transmitting device (weakly) pulls down 
MISO. If this is not the case, you need to mux using CS instead.


There will also be some subtleties with the tristate.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] shared SPI clock

2016-10-28 Thread Sébastien Bourdeauducq

On Friday, October 28, 2016 08:49 PM, Jonathan Mizrahi wrote:

In the platform extension file, when I specify the SPI buses, there is a
clock subsignal "clk." If I want to have multiple SPI buses that share
the same clock, I'm going to need to reference the same clock subsignal
in each bus, which means that that specific pin will appear multiple
times in the file. Will this cause any problems?


Yes. There will be multiple drivers for the same FPGA pin, and the FPGA 
tools cannot do anything reasonable with that.



Do I need to do something else to have a shared SPI clock?


The code assumes one clock per bus. There would be a bit of work to do. 
If you are using the SPI devices one by one, I suggest putting them all 
on a single RTIO channel and mux/demux MISO/MOSI.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Pipistrello TTL Output

2016-10-17 Thread Sébastien Bourdeauducq

Hi,

See here:
https://github.com/m-labs/artiq/blob/master/artiq/gateware/nist_qc1.py#L4

The A/B/C connectors and numbers match the silkscreen of the board.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Sinara clocking

2016-10-09 Thread Sébastien Bourdeauducq

On Sunday, October 09, 2016 10:20 PM, Grzegorz Kasprowicz wrote:

I mean connection of HMC7043 RFSYNCIN pins.


And it's HMC7044, not HMC7043.
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] ARTIQ implementation

2016-10-04 Thread Sébastien Bourdeauducq

Hi,

On Wednesday, October 05, 2016 12:17 PM, Cornelius Hempel wrote:

At this stage, we are just trying to get an understanding of the size and 
functional requirements (FPGA space and features) at the verilog level - which 
both Trung, our FPGA engineer, and Moglabs speak.


you can simply look at the synthesis logs on our buildserver, e.g.
http://buildbot.m-labs.hk/builders/artiq-board/builds/66/steps/conda_build/logs/stdio
http://buildbot.m-labs.hk/waterfall?show=artiq-board

Note that most of our builds are for the KC705, which uses significantly 
more resources than the Pipistrello target.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] ARTIQ implementation

2016-10-04 Thread Sébastien Bourdeauducq

Hi,

On Wednesday, October 05, 2016 09:49 AM, Trung Nguyen wrote:

My goal is to extract the Verilog comprising that gets compiled into the
FPGA bitstream in order to establish how and if we can deploy it on
another SPARTAN 6 board (= the pipistrello FPGA) made commercially by
moglabs here in Australia.

It appears that I need to intercept the output of both Migen and MiSoC
as they build the gateway bitstream, BIOS and runtime.


Better port Migen/MiSoC to your board, otherwise you will have to take 
apart/reinvent the ARTIQ build system.



I have tried two ways to install ARTIQ, (1) via Anaconda and (2) from
source.
(1) fails as Migen and MiSoC are not part of the distribution and I
presume this branch is just to compile user control commands to send to
the FPGA, correct?


Migen and MiSoC are packaged for conda and it is possible to build a 
bitstream using only the conda packages. But since you are intending to 
develop I recommend you use (and learn) git.



Trying Artiq 2.0, as pointed out in your previous email,
I tried to build gateware bitstream but I got this error:


AttributeError: type object 'BaseSoC' has no attribute 'csr_map'


https://github.com/m-labs/artiq/issues/565
If you use MiSoC from Git (not conda), "git checkout 0.3" also gives you 
the correct version.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] HMC7044 stability

2016-10-03 Thread Sébastien Bourdeauducq

Dear Thomas,

On Monday, October 03, 2016 11:18 PM, Thomas Harty wrote:

I guess you mean propagation delay temp. co. + long-term stability,
right?


Yes. Your document mentions a <0.1deg error at 3GHz which corresponds to 
100fs, but it doesn't say over what period of time that error is 
accumulated.



I haven't tested this for the HMC7044 yet, but I'm happy to
add it to the list of ICs to test.

I have a couple questions/comments for you, but first let me check I
understand your plan at a high level:

- We have a rack with both AMC (digital) and RTM (RF) backplanes -
The MCH (Metlino) receives a 100MHz clock from one of various
sources


There are only two sources possible for the Metlino:
1) 100MHz received directly when in root mode, and converted to 200MHz 
RTIO clock
2) 200MHz recovered from DRTIO fiber when in satellite mode (with more 
than one crate). Using the recovered clock here avoids a CDC in gateware 
and associated non-deterministic-latency issues.



- The MCH communicates with the AMCs via the AMC backplane
using Gigabit transceivers clocked at n*100MHz (say, 1GHz-10GHz)


Correct, or fiber to another crate or to a Kasli, which hopefully will 
get built.



-
Each AMC recovers its RTIO clock by integer division of the recovered
gigabit transceiver clock (e.g. to 125MHz)



Yes, and the satellite Metlinos (if any) also do.


- The RTIO clock shall be
the same for all devices, but is globally tunable over some specified
range


Yes. If we want tunability, we may have to use PLL synthesizer chips 
instead of XOs; as I mentioned in my email the Xilinx transceivers are 
not very flexible when it comes to synthesizing frequencies.



- The DAC data rates/TTL phy clocks are power-of-two multiples
of the RTIO clock


Multiple yes, power-of-two helps but is not strictly necessary, Robert 
already wrote on this topic.



- The DAC is clocked from one of various sources,
such as an integer-N PLL locked to the 100MHz RTM clock
- The ADC is
clocked from an integer division of the DAC clock


Both would be synthesized from 100MHz by one HMC7044, so yes.


For the synchronisation:

- SYS_REF is generated by integer division of the DAC clock, and is
routed from the divider (HMC704x) to the DAC using length-matched
traces, such that it's phase relationship to the DAC clock is
well-known
- SYS_REF is operated at an integer division of the RTIO
clock
- SYS_REF is also routed to the FPGA using a length-matched
trace
- As a result, SYS_REF has the same phase at the FPGA as at the
DAC
- The FPGA measures the phase of SYS_REF w.r.t. the RTIO clock by
scanning its IDELAY
- The FPGA then programs the phase of the SYS_REF
output of the HMC704x to align the DAC/ADC clocks + SYS_REF signals
with the RTIO clock. This alignment needs to be good to better than 1
DAC clock cycle (i.e. <<400ps)


Not necessarily length-matched, but with a known length.

Robert also pointed out that with the HMC7044 we can scan its fine 
delays (which are potentially better calibrated than the Ultrascale's 
IDELAY) instead of using the IDELAY. My email was assuming the AD9516-1 
there, which has a limited number of fine delay blocks.



Also, - Why do you require a 125MHz XO on the Sayma? Is this just
used for startup until the RTIO clock can be recovered from the
gigabit link?


Simple convenience and the 125MHz frequency is not critical. This 
oscillator is not meant to be used as transceiver clock.


The transceivers do however always need an external clock even when 
receiving, to provide a reference to the CDR. In the scheme I proposed 
it is the same frequency as the RTIO clock, typically 200MHz.


Regards,
Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Sinara clocking

2016-09-30 Thread Sébastien Bourdeauducq

On Friday, September 30, 2016 07:18 PM, Sébastien Bourdeauducq wrote:

AD9516-1 has more coarse delay range.


Scratch this. I just noticed the HMC has another "slip" mechanism that 
essentially gives it infinite range. So there is no reason to use the 
AD9516-1, other than compatibility with the FMC card.

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Sinara clocking

2016-09-30 Thread Sébastien Bourdeauducq

Hi,

Here is a plan for clocking the Sinara system. Please comment.

Crate clock distribution


The crate distributes a 100MHz clock on a RTM RF backplane. This clock 
is typically externally supplied from a high quality source, but it is 
desirable to include a 100MHz oscillator on the clock module for 
turnkey/standalone operation (with limited timing performance).


In a multi-crate system, all crates should receive the same 100MHz clock.

RTIO clock on Metlino
-

In root mode, the Metlino receives the 100MHz clock and turns it into a 
200MHz RTIO clock that it uses as reference clock for its DRTIO 
transmitters.


In satellite mode, the Metlino recovers the RTIO clock from the fiber.

The following clock resources should be available on Metlino to support 
this operation:
* Si5324 for 100->200MHz in root mode, and CDR jitter filtering in 
satellite mode.
* 200MHz XO to transceiver clock pin for providing a CDR reference in 
satellite mode.


The Si5324 shall have its two clock outputs connected to a transceiver 
clock input (so that we can transmit back synchronously and at fixed 
latency) and to a general purpose clock input on the FPGA. 
Transceiver<->fabric clock routing inside the FPGA is of poor quality, 
so we want to be able to mitigate that.


There are three options for connecting the 100MHz RTM clock to the 
Metlino (required in root mode):
1) make the Metlino double-width and connect it to RTM (preferred option 
IMO)
2) add one clock cable inside the crate - could be mechanically 
difficult or impossible
3) require two 100MHz inputs in root mode - this makes using the 
built-in oscillator unwieldy, and will affect single-crate systems


RTIO clock on Sayma
---

Sayma cards recover their RTIO clock from the backplane's transceiver 
links. This requires the same hardware as the Metlino in root mode: XO + 
Si5324, connected in the same way.


RTIO clock frequency flexibility


The FPGA's built-in transceiver PLLs are not very flexible, so if easy 
changing of the RTIO clock frequency is desired, we should consider 
replacing the XOs with PLL synthesizer chips.


General purpose oscillator
--

Sayma and Metlino shall include a general purpose XO of e.g. 125MHz, 
connected to a general purpose FPGA clock input pin. Transceiver clock 
inputs can be routed to the fabric (with degraded timing), so this is 
not absolutely necessary, but this is a simple addition that make the 
boards a bit friendlier to developers. This XO becomes necessary if we 
use a transceiver PLL chip that needs to be configured before it outputs 
a clock.


JESD204 synchronization procedure
-

The FPGA shall align SYSREF with designated RTIO clock edges. The 
alignment should be better than a DAC clock cycle and reproducible 
across reboots.


The FPGA first roughly aligns SYSREF within one cycle before a desired 
RTIO clock edge by asserting the synchronization signal of the clock 
chip, which resets its dividers. This alignment has an uncertainty of 
one DAC clock cycle. The FPGA then analyzes SYSREF by repeatedly 
sampling it with a known clock while scanning a calibrated I/O input 
delay (IDELAYE3) element to measure its phase with a high precision. It 
then derives a phase correction value that it programs into the relevant 
registers of the clock chip to delay SYSREF by the duration that aligns 
it with the RTIO clock.


This mechanism is limited by the resolution of the IDELAY scanning 
technique and of the clock chip's fine delay block. For this to work, 
those resolutions should be good enough, i.e. significantly smaller than 
a DAC clock period.


The clock chip we should use to generate SYSREF and the DAC clock is 
AD9516-1 or HMC7044. HMC7044 looks like a better choice.


AD9516-1 has more coarse delay range. HMC7044 has more outputs, less 
noise, and a higher resolution fine delay block.


Other clock chips that were considered but not selected were:
* AD9528 - maximum output frequency too low.
* HMC7043 - no PLL, cannot multiply the input clock.

This technique can be implemented on the AD9154 FMC cards, which also 
use a AD9516-1 (assuming its fine delay block has enough resolution).


Alternate synchronization procedure #1
--

In case we encounter problems with the fine delay, we may be able to get 
away by using the coarse delay only (which has a resolution of one DAC 
clock period). This will require storing some information about the 
choice of clock edge made from one boot to the next in order to ensure 
constant latency across reboots. This technique was proposed by Robert 
on IRC yesterday.


Alternate synchronization procedure #2
--

If the above techniques fail, we can distribute SYSREF at the source and 
use it to sync HMC7044 clock chips (which contain a SYSREF retimer that 
makes 

Re: [ARTIQ] ARTIQ implementation

2016-09-28 Thread Sébastien Bourdeauducq

Hi,

the mailing list I was referring to is the ARTIQ list 
(https://ssl.serverraum.org/lists/listinfo/artiq).


You need to use a modified Rust:
https://github.com/m-labs/rust

Full compilation instructions for the upcoming ARTIQ 3.0 are here:
https://m-labs.hk/artiq/manual-master/installing_from_source.html#preparing-the-build-environment-for-the-core-device

Alternatively you can use the already released ARTIQ 2.0 which does not 
require Rust.


Sébastien


On Thursday, September 29, 2016 10:28 AM, Trung Nguyen wrote:

Hi Developers,

I'm trying to run the Artiq environment to compile the gateware file.
My OS is Linux. Distribution version is Ubuntu 14.04LTS

I have done step-by-step installation of the following manual but
excepts the openOCD because we don't have FPGA board yet:
https://m-labs.hk/artiq/manual-release-2/installing_from_source.html#install-from-source

The installation process is almost ok but I have some problems with
libraries and packages that I had to install when I installed Artiq.
I installed the following libraries and packages:

hdf5-1.8.17
pygit2==0.19.1
numpy-1.11.2rc1
h5py-2.6.0
libffi6 Version: 3.1~rc1+r3.0.13-12ubuntu0.1
Package: libhdf5-dev Version: 1.8.11-5ubuntu7
cargo 0.13.0-nightly (7964e94 2016-09-27)
rustc 1.11.0 (9b21dcd6a 2016-08-15)


I'm struggling with an error when I run this code:

python3.5 -m artiq.gateware.targets.pipistrello

And here is the error message:

make: Entering directory

`/home/trungnguyen/artiq-dev/misoc_nist_qc1_pipistrello/software/runtime'

CARGO_TARGET_DIR="./cargo" \

cargo rustc --verbose \

--manifest-path
/home/trungnguyen/artiq-dev/artiq/artiq/runtime/../runtime.rs/Cargo.toml
 \

--target=or1k-linux -- \

-C target-feature=+mul,+div,+ffl1,+cmov,+addc -C opt-level=s \

-L../libcompiler-rt

error: failed to run `rustc` to learn about target-specific
information


Caused by:

  process didn't exit successfully: `rustc - --crate-name _
--print=file-names --crate-type bin --crate-type rlib
--crate-type staticlib --target or1k-linux` (exit code: 101)

--- stderr

error: Error loading target specification: Could not find
specification for target "or1k-linux"


make: *** [libartiq_rust.a] Error 101


The target is incorrect. It was "or1k-unknown" but I read the note in
released manual as below:

We’re using an |or1k-linux| target because it is necessary to enable
shared library support in |ld|, not because Linux is involved.

from this link:
https://m-labs.hk/artiq/manual-release-2/installing_from_source.html#install-from-source
So I tried to change the target to "or1k-linux" but it does work.

Could you please have a look? Any comments are appreciated.
Thank you.

Best Regards,
Trung Vu Nguyen

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] 3.3 V I/O on kc705

2016-09-19 Thread Sébastien Bourdeauducq

Hi,

On Tuesday, September 20, 2016 09:35 AM, Jonathan Mizrahi wrote:

Thanks! I assume this is the part I need to buy?
http://www.ti.com/tool/usb-to-gpio


Yes.


The ARTIQ manual doesn't say anything about removing and replacing the
J65 jumper, which the kc705 manual talks about (p. 74). Is this unnecessary?


No. The technique described in the manual is for when the FPGA 
reprograms dynamically the power chip. Since we didn't want to bother 
with the details of programming proprietary TI power chips, we used a 
different technique that requires only existing software and hardware. 
The proposed technique will permanently program the TI chip to output 
3.3V, without FPGA intervention.



Also, are all of the I/O pins on the FMC connectors in HR banks (and
thus able to go to 3.3 V)?


IIRC yes, all HP banks go to on-board peripherals (mostly the SDRAM), 
and it would not be smart if reprogramming the FMC voltage only changed 
it for some of the pins. You can check using the schematics.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Applets and Datasets

2016-08-02 Thread Sébastien Bourdeauducq

On Tuesday, August 02, 2016 11:55 PM, Stevens, Kelly E. M. wrote:

Double click the content in the "Command" column and edit the line
such that "Y_DATASET" is replaced with "parabola".  Also, you can
remove --x, --error, and --fit options for this example.


The problem with this kind of detail is that if someone makes a change - 
however trivial - to the command line interface of the applet, then this 
tutorial becomes outdated, which is worse than no tutorial.


It would be better to explain the general principles of how applets work 
and the -h option (as I did in my email), which respectively are part of 
the ARTIQ architecture and a widely adopted convention and therefore are 
much less likely to change. Let the users figure out how they should 
edit the command line in the GUI based on that.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Applets and Datasets

2016-08-01 Thread Sébastien Bourdeauducq

Hi,

On Tuesday, August 02, 2016 02:07 AM, Stevens, Kelly E. M. wrote:

I ran the example run() function and tried to get a plot generated
but wasn’t able to figure it out.  The manual says I need to edit the
applet command line so that it retrieves the parabola dataset.  How
do I do that?


For the XY plot applet, the Y dataset name is a positional argument. The 
full command would be something like:


{python} -m artiq.applets.plot_xy --embed {ipc_address} 
name_of_my_parabola_dataset



Also, the default XY applet has this in the text:
“{ipc_address}” what is that?


Before running the applet command, the GUI replaces {ipc_address} that 
by a string that tells the applet how to connect to it, for the applet 
to embed itself and get dataset access. Same with {python} that gets 
replaced with the path to the Python interpreter that runs the GUI.



Also, in general, where can I find
documentation on what environment variables


Environment variables are not involved here, it's just a command line.


are available on the applet command line?


The applets are independent Python programs and you get their command 
line options by running them in a shell with the usual -h:


$ python -m artiq.applets.plot_xy -h
usage: plot_xy.py [-h] [--update-delay UPDATE_DELAY] [--server SERVER]
  [--port PORT] [--embed IPC_ADDRESS] [--title TITLE] 
[--x X]

  [--error ERROR] [--fit FIT]
  y
...
datasets:
  y Y values
  --x X X values
  --error ERROR Error bars for each X value
  --fit FIT Fit values for each X value

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] sayma gateware updates

2016-07-26 Thread Sébastien Bourdeauducq

On Tuesday, July 26, 2016 06:20 AM, j arl wrote:

Kintex UltraScale SYSMONE1 interface supports digitization of 4 power
supply voltages and up to 17 external analog signals. The resulting
data can be readout by FPGA (DRP), exposed to MMC using I2C and
readout using JTAG.

My read of the docs indicates that readout by I2C could be done by MMC
without involving FPGA gateware.


But then this requires the MMC to be used. I thought that the plan was 
to program the power supply to be on at all times, and then ignore the MMC?


Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] sayma gateware updates

2016-07-24 Thread Sébastien Bourdeauducq

On Sunday, July 24, 2016 10:04 PM, Grzegorz Kasprowicz wrote:

This can be also done using single FLASH and multi boot. Xilinx has
such option but never tested it.


I have used some of the multiboot features on Spartan-6, and it worked 
fine. Never tried on other FPGAs.



Well, we already have Ethernet switch in the crate, delvered to all
interesting places.


But we do not have it on any of our boards. And if we don't depend on 
Ethernet, it means we can remove it completely from the crate and/or 
backplane, should we decide to design our own. And Ethernet does not 
work in multi-crate systems, and Kasli, unless a) a separate cable is 
connected and b) Ethernet cruft is added to Kasli.



Since diagnostic data are not timing critical, you could devote main
fibre channel to real time traffic.


Diagnostic data is also low-bandwidth, and dedicating some fixed small
amount of the fiber traffic to such functions is also a solution.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] sayma gateware updates

2016-07-24 Thread Sébastien Bourdeauducq

Hi,

On Sunday, July 24, 2016 07:33 PM, Grzegorz Kasprowicz wrote:

> When receiving a bitfile, a board (Metlino/Sayma) writes it to its
> flash memory (via a SPI master core inside the FPGA) and then reboots
> (using ICAP) to load the new bitfile.
We used this approach some
time ago in AFC/AFCK designs. But it had no failover protection. For
this reason I added second FLASH that MMC can switch and do failsave
recovery of communication link.


In the rare case of broken flash contents, one can reflash using the USB
link. And for development I guess the USB link will be used anyway.


MTCA supports remote upgrade via IPMB, but it takes ~20h to write to
config FLASH.


This is another reason we should ignore IPMB.


In our case, main remote update channel is over Ethernet, where FPGA
programs its own FLASH by custom gateware while keeping fail-safe
FLASH intact.


I don't think we really need the complexity of a fail-safe flash. A 
board that fails to boot will be clearly shown (or rather, not shown) 
during DRTIO device discovery, and it can then be manually serviced by 
reflashing over USB. And I expect this situation to be rare.



In case of AMC boards, we consider adding Ethernet PHY
to keep compatibility of AMC standard (PORT0). Together with simple
open source MAC we gain communication channel over PORT0 and MCH
switch. This can be used for setup, diagnistics and update.


So can the existing DRTIO links. What does in-crate Ethernet bring to 
the table? And if the system has multiple crates, we'd need to connect 
an Ethernet cable between them as opposed to just the fiber, or invent 
another gadget to somehow tunnel the in-crate Ethernet into the fiber.



Since the PHY chip supports RGMII and MII standards and on the other
side has 1000BASE-X, we can use its MII port to connect to MMC chip
and in this way gain for free additional update channel where MMC can
do FLASH programming, direct FPGA ocnfig, diagnistic and fail
recovery functions.


It's not free: it needs to be developed, tested and maintained. And it 
bloats the design.



So the procedure would be simple: MMC diables
FPGA by PROGRAM_B line, than takes control over MII interface of PHY
chip. The chip does translation between 100mbit and 1Gbit link speed
and in this way we can have quick access to both FLASH and all
resources of the AMC board (supply currents and voltages,
thermometers, ID, etc)


The FPGA can handle all this already.


But in final production system where several AMC boards are present,
it may make sense to boot all FPGAs over Ethernet at the startup
from single file location. In this way it's easier to keep control
over file versions. FLASH chips won't be used at all in this
scenario.


I agree that this approach to bitstream management nice. But it does 
require a lot of gadgetry (microcontroller, in-crate Ethernet, Ethernet 
links between crates, the tricks you mentioned to switch Ethernet 
between FPGA and uC), which is way beyond the complexity of a flash 
chip. We can embed version numbers in bitstreams and have the software 
automatically check them and reflash as needed, which would give similar 
behaviour.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] sayma gateware updates

2016-07-23 Thread Sébastien Bourdeauducq

On Sunday, July 24, 2016 11:27 AM, Sébastien Bourdeauducq wrote:



It is reasonable to use the same approach to (a) and (c) on
metlino_motherboard?


I suggest (a) and (b) on all boards. (c) involves gadgetry.


And by "all boards" I also mean Kasli, which (c) would not support.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] sayma gateware updates

2016-07-23 Thread Sébastien Bourdeauducq

Hi,

On Sunday, July 24, 2016 07:59 AM, j arl wrote:

The bitfiles for the sayma_motherboard FPGA will be stored in flash.
We've discussed several ways of updating the bitfiles. a) serial JTAG
b) DRTIO over microTCA Port4 c) Ethernet microTCA Port0.

Update method (a) is implemented by a directly connecting a serial
cable to the board. This is well suited to factory debug and loading
the initial bitfiles. However, it's not convenient for in-field
upgrades.


For this technique, one could also build the JTAG adapter into the board 
- use a FT2232H chip and have a small USB connector on the front panel.



(b) and (c) permit in-field upgrades without external connectors.

What implementation of (b) does m-labs have in mind for sayma? Since
it's DRTIO it involves the Port4 GTH from metlino_motherboard.


The PC sends bitfiles to the root Metlino over the Metlino's front panel 
Ethernet connector (the same used in regular operation of the system). 
The Metlino receives its own bitfile from there, and distributes the 
other bitfiles to the whole DRTIO system using the existing 
communications links (fiber/backplane).


When receiving a bitfile, a board (Metlino/Sayma) writes it to its flash 
memory (via a SPI master core inside the FPGA) and then reboots (using 
ICAP) to load the new bitfile.



Greg advocates for use of ARM Cortex M3 (LPC1700 family from NXP) to
implement the microTCA MMC functionality (IPMI).


This is gadgetry that solves no real problem. And how do you program the 
microcontroller? It's not easier than loading a FPGA bitfile.


If you want temperature sensors (the only arguably useful monitoring 
feature IMO) I suggest connecting them to the FPGA and reading them out 
over DRTIO. The FPGA already contains one built-in temperature sensor, 
and an ADC with additional external channels.



Greg points out that
the M3 has a built-in MAC. If this were connected to Port0 the MMC
could be reconfigured over Ethernet. The M3 could also be connected to
the FGPA to permit loading of bitfiles.


I would not bother with this and also with Ethernet inside the crate. 
The built-in FPGA configuration mechanisms are good enough and do not 
require additional hardware.



This would enable (c).
Xilinx's recommended implementation is discussed in "Figure 3-2: Slave
Serial Mode Configuration Interface Example" of the
UltraScale Architecture Configuration guide.

It seems (b) and (c) are not mutually exclusive.


The microcontroller would have to be connected and programmed to drive 
the Mx pins of the FPGA to switch between the two modes - serial flash 
for (b) and slave serial for (c).



It is reasonable to use the same approach to (a) and (c) on
metlino_motherboard?


I suggest (a) and (b) on all boards. (c) involves gadgetry.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 1.2 released

2016-07-14 Thread Sébastien Bourdeauducq

Hi,

ARTIQ 1.2 is out and packages are available in our main Anaconda channel.

This is a pure bugfix release and we recommend that all 1.0 and 1.1 
users upgrade to 1.2. Thanks to everyone who submitted detailed bug reports.


Best,
Sébastien for the ARTIQ team

List of changes:
  * pipistrello: shrink TTL FIFOs
  * ir: `invoke` is a valid `delay` decomposition. (#510)
  * monkey-patch Python 3.5.2 to disable broken 
asyncio.base_events._ipaddr_info optimization (#506)

  * rtio: do not reset DDS and SPI PHYs on RTIO reset (#503)
  * compiler.testbench.perf_embedding: more fine grained reporting.
  * compiler.testbench.perf_embedding: update for core changes.
  * runtime: update ppp code for lwip 2.0.0.
  * runtime: fix serialization of object lists. (#500)
  * embedding: fix location for diagnostics on quoted values.
  * targets/kc705: redefine user SMAs as 3.3V IO. Closes #502
  * targets/kc705: fix import
  * spi: expose more documentation on chaining transfers
  * spi: do not shift when starting a xfer, closes #495
  * language/environment: fix mutate_dataset with parent. Closes #498
  * worker: increase send_timeout (Windows can be really slow)
  * dashboard: kill the Qt built-in main window closing mechanism 
(fixes exit crashes)

  * compiler.embedding: use the builtin print as RPC.
  * compiler: remove now()/at().

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] 7-series White Rabbit GTX settings

2016-07-11 Thread Sébastien Bourdeauducq

Hi,

On Monday, July 11, 2016 05:27 PM, Peter Jansweijer wrote:

Both sides of the the tx and rx fifos are clocked with the same
frequency so there are no under- or overflows.


Yes, I understand this.


For high speed devices the region where you would hit a jump point is
extremely small, you would be very (un)lucky to hit it. To my knowledge,
this has not been observed or reported up to now.


A rough estimation is ~3% of the designs would exhibit the problem 
(assuming 8ns data clock, 150ps setup+hold times, 100ps of jitter, and 
each design has a random phase relationship between the clocks on each 
side of the elastic buffer).



And yes, we would like the highest accuracy we can reasonably get.

This will also be our goal, now that WR has matured. The (restart)
jitter behaviour of the transceivers should be part of a more detailed
study on jitter contributions (an awareness that became clear due to our
fruitful email conversation).


Cheers,
Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] 7-series White Rabbit GTX settings

2016-07-07 Thread Sébastien Bourdeauducq

Hi,

@list: See below for some nitty-gritty details of fixed-latency 
transceiver implementation, if you are interested in that. It only 
impacts the gateware and is of no relevance to the hardware.


@Peter: Thanks for your replies. I expected the elastic buffer to be 
clocked at the frequency of the user data clock, and therefore the 
magnitude of the jumps should be equal to its period, i.e. several 
nanoseconds. Are you sure that you do not happen to hit a phase 
relationship in your design that is away from a jump point and therefore 
you get deterministic latency (by chance)? Or am I misunderstanding 
something? Is the elastic buffer clocked faster than the user data clock?


And yes, we would like the highest accuracy we can reasonably get.

Sébastien


On Thursday, July 07, 2016 06:12 PM, Peter Jansweijer wrote:

Hi Sebastien,

For Virtex 6 devices where the TX latency (using elastic buffer) was not
fixed. In retrospective you bring up a good point about discrete jumps
in the elastic virtex6 buffer. All other FPGAs behaved deterministic.
Indeed, jitter causes small latency variations between startups. Have a
look at slide 16 of
http://www.ohwr.org/attachments/3552/141006_WR_Workshop_8_KM3NeT.pptx
Never mind the outliers since they were caused by endpoint bugs that
were solved in later releases. You can see a red (SPEC) and blue (KM3eT
CLB) band with a spread in time for different restarts. There are
multiple causes but the jitter you mention is one of the parameters.
Oscillator stability and the servo software are other parameters.
For a high accuracy WR implementation we should determine the amount of
jitter for each of the parameters, among others these PHY elastic buffer
restart latency uncertainty. For the past specification (< 1 ns) is was
just accepted as good enough.
Are you aiming for better precision than the 1 ns specification, and is
this why you are concerned about latency variations?

Cheers,

Peter




On 7-7-2016 10:24, Sébastien Bourdeauducq wrote:

Hi,

Yes. But how do you ensure that the phase relation between the two
domains isn't near a value that causes a discrete jump in the elastic
buffer latency (and then jitter can cause latency variation between
startups)?

Sébastien


On Thursday, July 07, 2016 03:55 PM, Peter Jansweijer wrote:

Hi Sebastien,

The situation is as you describe. However the phase relation between one
and the other domain is fixed (and thus deterministic).
For Rx you use the recovered clock at both sides of the fifo.
For Tx the recovered clock is used for write and the GTX ref clock
(derived from the recovered clock bij the VCXO) is used for reading the
Tx fifo.
So you can do without the fifo's but need to be careful that both
domains have proper setup and hold times. This can be automatically done
by using the phase align ports on the GTX or GTP (see for example
gtp_phase_align_virtex6). The phase is then fixed at a safe value (and
deterministic again). The latter involves an extra state machine which
can be avoided when using the elastic buffer. For Virtex6 the elastic
buffer didn't function well so there we needed to implement the phase
alignment.

Cheers,

Peter


On 6-7-2016 14:38, Sébastien Bourdeauducq wrote:

Hi Peter,

As I understand the situation, the TX/RX buffers are shallow FIFO
ringbuffers where the write pointer is incremented at each cycle of
the write clock, and the read pointer is incremented at each cycle of
the read clock. The initial difference between the write and read
pointers depends on the phase relationship between the write and read
clocks. For some phase relationships, the jitter causes an uncertainty
of one position for the write/read pointer difference, and therefore
the buffers cannot be used if one wants reliable deterministic latency.

Sébastien


On Monday, June 20, 2016 05:28 PM, Peter Jansweijer wrote:

Hi Sebastien,

Enabling TX/RX buffers avoids a phase alignment cycle. The buffers
take
care of proper PMA/PCS clock domain crossing. The latencies are fixed.
I didn't understand your later remark about using the RX CDR clock,
filter it, and use it for TX? The Rx recovered clock is used for the
receiving side of the PHY. In combination with gtp_bitslide this gives
you fixed latency as well on the Rx path.

Cheers,

Peter


On 20-6-2016 10:30, Tomasz Wlostowski wrote:

On 17.06.2016 17:26, Sébastien Bourdeauducq wrote:

Also, how can you take the RX CDR clock, filter it, and use it
for TX
without using the QPLL?

Or is this transceiver PHY not working according to the published WR
documentation?


Hi Sebastien,

I think the best person to answer your questions would be Peter. I
haven't used WR yet on a Kintex-7...

Cheers,
Tom

Sébastien


On Friday, June 17, 2016 11:18 PM, Sébastien Bourdeauducq wrote:

Hi,

I'm looking at the WR code right now and I'm surprised that the RX
and
TX buffers of the GTX transceivers for 7-series are enabled. Aren't
they
introducing unpredictable latency that is potentially differ

Re: [ARTIQ] uTCA backplane driver choices

2016-07-05 Thread Sébastien Bourdeauducq

On Tuesday, June 28, 2016 04:00 PM, Grzegorz Kasprowicz wrote:

For Sayma board to not waste precious GTX we will add PHY with
1000Base-X. We can also route it to SATA connector and provide dedicated
SATA-SFP cables. Most of these PHYs apart from RGMII to 1000base-X offer
1000base-T, so we can also put low profile or vertical RJ45 connector
for quick access. Full size SFP won't fit:)


If it is not possible to fit a SFP, a direct transceiver connection e.g. 
SATA connector, would be the second-best option. Ethernet PHYs add (a 
bit of) complexity to the board and do not support White Rabbit-style 
time transfer. But do we have a spare GTX for such a connector anyway? 
Won't we need to keep using the backplane connector (with an SFP 
adapter) for the transceivers when the Sayma board is stand-alone?


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] script rename?

2016-07-05 Thread Sébastien Bourdeauducq

On Tuesday, July 05, 2016 09:34 PM, Stevens, Kelly E. M. wrote:

I noticed that artiq_gui.py is not in the trunk git repository
(2.0dev).  What is the new equivalent?


artiq_dashboard. See the release notes for the complete list of changes 
that break compatibility with 1.x.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] "connection reset" bug

2016-07-04 Thread Sébastien Bourdeauducq

Hi,

is anyone other than Chris having this problem:
https://github.com/m-labs/artiq/issues/456

If yes: can you post your precise network setup and would you consider 
shipping us hardware that would allow us to reproduce the problem?


If no, I will reduce the priority of this issue.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] June 2016 status report

2016-07-04 Thread Sébastien Bourdeauducq

Please see the attached PDF.


2016-June.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] clock recovery on Metlino and Kasli

2016-06-30 Thread Sébastien Bourdeauducq

On Thursday, June 30, 2016 04:49 PM, Grzegorz Kasprowicz wrote:

In case of WR it already worked quite well 6 years ago but later on this
circuit was modified several times:)
All the time little things.
So since we have more important things to do I'd leave it as it is.


Since we are not going to use White Rabbit, I propose removing them. I 
do not have a strong opinion about this since the complexity/part count 
should be small.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] clock recovery on Metlino and Kasli

2016-06-29 Thread Sébastien Bourdeauducq

Hi,

my question was whether one can get rid of all those 6 parts and replace 
them with a Si5324 circuit (instead of #1 and supporting parts) and 
internal FPGA PLL(s) (instead of #2 and supporting parts). Why did you 
design it that way - what do all those additional parts bring exactly?


Sébastien


On Thursday, June 30, 2016 12:06 AM, Grzegorz Kasprowicz wrote:

Hi
The idea is to use SPEC board design: http://www.ohwr.org/projects/spec/wiki
It's well debugged. I designed it a few years ago and it's already 4-th
revision.
The WR components are:
1. 25MHz VCXO VM53S3-25.000-2.5/-30+75 + CDCM61004RHBT PLL chip. The two
can be replaced by 125MHz VCXO.
2. 20MHz helper VCXO for DDMTD : LF VCXO026156
3. 2x DAC AD5662BRMZ-1
4. 3V LDO : LP5951MF-3.0/NOPB
5. 2.5V reference: LM336M-2.5/NOPB
6. some LRC passives
Greg




On 29 June 2016 at 15:56, j arl > wrote:

As the topic shifted from backplane logic, I've moved to this
conversation to a new thread.

On Tuesday, June 28, 2016 04:02 PM, Grzegorz Kasprowicz wrote:
> For synchronisation over fibre we can use existing White Rabbit core.
> The card requires only 2 VCXO oscillators and FPGA logic. The WR core
> consumes 50% of small Spartan 45T. It ensures 1ns timing accuracy.

>We probably won't use White Rabbit as-is, but the basic principle will
>be the same. It can be a good idea to include those VCXOs on boards
that
>may be synchronized over fiber (Kasli/Metlino), though can't we use
>instead a Si5324 for jitter cleanup and FPGA PLLs for DDMTD?

Heads up that the clock synchronization for Metlino resides on
Metlino_clk (Tongue 2) not Metlino_motherboard (Tongue 3-4). So
headers need to provide whatever io is required. Greg already has a
layout for a Metlino_clk board that has White Rabbit components on it.

Greg, do you have a short list of components required for
WhiteRabbit-type clock recovery?  -Joe



___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] uTCA backplane driver choices

2016-06-28 Thread Sébastien Bourdeauducq

On Tuesday, June 28, 2016 04:02 PM, Grzegorz Kasprowicz wrote:

For synchronisation over fibre we can use existing White Rabbit core.
The card requires only 2 VCXO oscillators and FPGA logic. The WR core
consumes 50% of small Spartan 45T. It ensures 1ns timing accuracy.


We probably won't use White Rabbit as-is, but the basic principle will 
be the same. It can be a good idea to include those VCXOs on boards that 
may be synchronized over fiber (Kasli/Metlino), though can't we use 
instead a Si5324 for jitter cleanup and FPGA PLLs for DDMTD?


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] GUI issues when running artiq_gui remotely via an ssh session

2016-06-08 Thread Sébastien Bourdeauducq

Hi,

On Tuesday, June 07, 2016 03:34 PM, Hankin, Aaron M. (Assoc) wrote:

The main attraction of the X11-forwarding-over-ssh approach is that it
feels more responsive, which helps when working remotely from home
rather than the office.


What about running the ARTIQ GUI locally and passing the connections to 
the ARTIQ master over SSH or VPN? This will result in maximum 
responsivity and minimum network bandwidth usage. There is nothing that 
fundamentally prevents ARTIQ GUI from running on OS X, though there will 
certainly be significant trouble (compiling/installing Qt, bugs, etc.)


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] GUI issues when running artiq_gui remotely via an ssh session

2016-06-07 Thread Sébastien Bourdeauducq

Hi,

Have you tried VNC? X is a flawed protocol that rarely works.

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] Setting up Pipistrello with ARTIQ

2016-05-24 Thread Sébastien Bourdeauducq

On Tuesday, May 24, 2016 12:47 PM, Ken Brown wrote:

I am having a problem with the socket connection.  After I run pppd, I
receive a modem hangup message and the connection is terminated.
It seems to connect fine based on the rcvd and sent commands logged in
verbose.


Please post the exact commands you typed, the exact error message 
received and the output of pppd in debug mode.


Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 1.0rc3

2016-04-19 Thread Sébastien Bourdeauducq

Hi,

ARTIQ 1.0rc3 is now in the main conda channel and contains a number of 
bugfixes and a faster compiler. Please upgrade.


The last hardware test necessary before 1.0 is the PDQ:
https://github.com/m-labs/artiq/issues/178

Sébastien
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-04-11 Thread Sébastien Bourdeauducq
On Tuesday, 12 April 2016 1:55:45 AM HKT Slichter, Daniel H. (Fed) wrote:
> On the DSP/"Sayma" boards, a hard SoC could have use for reducing the time
> it takes to perform feedback calculations (e.g. shifts of output signal
> frequency based on ADC readings).  

Can't this be done on the Metlino (where it *might* make more sense to have a 
MPSoC) or with gateware? What are those feedback calculations exactly?

> My comment in the previous email about "users" wanting this also pertains to
> the idea that people not using ARTIQ might want to buy these boards and use
> them for their own purposes.  For many of them, having a hard SoC on board
> may make it faster/easier for them to get the boards doing what they
> want.  Again, it's not totally necessary to have this but would enable more
> applications and a broader group of potentially interested customers.  

There is no shortage of general purpose FPGA boards for Zynq users. Do we 
really want to compete in this market?

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-04-09 Thread Sébastien Bourdeauducq
On Saturday, 9 April 2016 6:10:36 PM HKT Grzegorz Kasprowicz wrote:
> Why do you think that CPUs have negative value? You don't have to use them
> at all. 

I already explained that the MPSoC has to be dealt with and cannot be 
completely ignored. If we have two SDRAM systems, maybe we can to a reasonable 
extent, but then it does complicate the board.

> Anyway, to build MCH, we need first tongue. We can either build it or buy
> it.  Buying is the simplest option at the moment and we get gigabit switch
> and management.

> On the MCH Tongue 3, there is single port Ethernet connection that can be
> used for slow control and remote upgrade.

We do not need Ethernet (nor IPMI in fact) inside the crates. The only part of 
the system that has Ethernet is between the control PC and the root master 
(Metlino); this can go over one of its regular SFPs.

> 6. MCH Tongue 2 clock distributor(Metlino). We can use NAT clock module with
> two low jitter crossbar switches.

Why do we need a fancy clock module? Data transfer certainly does not require 
one. Are you reconsidering your position that the AMC backplane should not be 
used for DAC clocks?

Also, (5 data/clock signals)*(10 AMCs)*(2 for differential) = 100 pins, plus 
power supply. Do we really need all those "tongues"?

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-04-08 Thread Sébastien Bourdeauducq
On Friday, 8 April 2016 12:37:02 PM HKT Grzegorz Kasprowicz wrote:
> Modification of the backplane is quite difficult and expensive.

If we have a minimalistic backplane with just power, maybe IPMI, 4x 
differential pairs between MCH and each AMCs, and clock - why is that 
particularly expensive?

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-04-08 Thread Sébastien Bourdeauducq
On Friday, 8 April 2016 11:53:25 AM HKT Grzegorz Kasprowicz wrote:
> Btw,
> ZU11 FPGA costs more or less the same as 7K325 and offers almost twice more
> logic resources. The price of ZU11 is $1,376.00 at 100pcs.
> Since we will buy such quantity for our CBM project (Fair facility in GSI),
> we can use that step pricing in this case as well.

Even on Digikey in single quantities, 7K325T starts at $898.75 (and the 
cheapest one in FF which seems to be recommended for the full transceiver 
bandwidth is $1122.50) and we don't have to deal with any of this disgusting 
Zynq stuff.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-04-08 Thread Sébastien Bourdeauducq
Just to clarify this: We do need a pretty large FPGA (7K325T) on the AMC cards 
due to the resource requirements of signal processing. Then we have 16 GTX 
12.5Gbps transceivers, but if we want to make the two FMCs identical and 
capable of receiving fast 4-channel DACs with 8 lanes, then we have no more 
transceivers. So the IOSERDES option is still interesting.

Greg - How many data lanes can we easily have between the MCH and each AMC 
with a star topology backplane?

Sébastien


On Wednesday, 30 March 2016 9:33:51 PM HKT Grzegorz Kasprowicz wrote:
> Hi all
> I like that idea - in this way we can do it really affordable with lower end
> FPGAs and still compatible with MTCA.
> If we place all RF stuff on the modules, with RF connectors mounted directly
> on the front panel, this could work.
> How we would deliver the clocks? Using external SMA connectors?
> The only think I worry about is performance of the DACs.
> ZynQ US+ can run IOserdes with almost 2Gbit/s performance - one can make
> 1000base-x directly on IO pins using oversampling technique.
> Greg
> 
> -Original Message-
> From: Sébastien Bourdeauducq [mailto:s...@m-labs.hk]
> Sent: Wednesday, March 30, 2016 1:50 PM
> To: Grzegorz Kasprowicz <kaspr...@gmail.com>
> Cc: 'Slichter, Daniel H. (Fed)' <daniel.slich...@nist.gov>; 'Grzegorz
> Kasprowicz' <gkasp...@elka.pw.edu.pl>; artiq@lists.m-labs.hk
> Subject: Re: [ARTIQ] FW: initial specification of the project
> 
> On Tuesday, 29 March 2016 11:55:19 PM HKT Grzegorz Kasprowicz wrote:
> > - they can be used immediately with existing OSHW carriers like AFC/AFCK.
> 
> AFCK may be overkill. We are still working on evaluating the FPGA resource
> requirements.
> 
> > - it could be hard to fit FPGA, supply, DACs and several RF modules,
> > all on single dual width AMC, especially when shielding is required.
> > RTM relaxes these constraints
> > - on AMC+RTM you can place 8 ADC channels + 8 DAC channels + 8 RF
> > modules. In case of single AMC board it would be hard to achieve such
> > channels density.
> 
> How about this:
> * we reduce the number of channels per AMC to 4 DACs + 4 ADCs
> * we can therefore use a smaller FPGA. Communication lanes to the master
> board are relatively cheap if we put them on IOSERDES.
> * the power density and cooling requirements are also reduced.
> * for the RF daughter cards, we use a custom form factor that can use at
> least
> 2/3 of the AMC front panel
> * connectors between the DSP card and the RF daughter card are 2mm header
> and 8x SMP
> * the rest of the AMC front panel used for (optional, runtime selectable)
> clock input and some TTLs.
> 
> Sébastien


___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ 1.0rc2

2016-04-02 Thread Sébastien Bourdeauducq
Hi,

ARTIQ 1.0rc2 is out and contains fixes for most of the problems reported so 
far.

There is one API change from 1.0rc1: set_dataset in broadcast mode no longer 
returns a Notifier. Mutating datasets should be done with mutate_dataset 
instead (https://github.com/m-labs/artiq/issues/345).

Things that need particular attention for 1.0 are:
* SPI (https://github.com/m-labs/artiq/issues/277)
* PDQ https://github.com/m-labs/artiq/issues/178
* I need to know whether my timing patch is necessary for I2C (https://
github.com/m-labs/artiq/issues/304)
* I also still need patch/docs for QC2 (https://github.com/m-labs/artiq/
issues/264)

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-04-01 Thread Sébastien Bourdeauducq
On Friday, 1 April 2016 5:25:19 PM HKT Thomas Harty wrote:
> d) Something else I'm missing?

Low-frequency jitter of the PLL?

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-03-30 Thread Sébastien Bourdeauducq
On Wednesday, 30 March 2016 9:37:51 PM HKT Grzegorz Kasprowicz wrote:
> Well, you don't have to write it.
> It is already available for RTOS and linux.

We are not using RTOS or Linux.

> But it's true - it occupies MIO bank and dedicated DDR port. But this is axi
> and can be easily accessible from PL part.
> How many IOs do you need?

IO count is not the problem. The problem is you cannot switch between designs 
that use the fabric to talk directly to Ethernet and SDRAM, and designs that 
use those questionable hardened blocks.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-03-30 Thread Sébastien Bourdeauducq
On Tuesday, 29 March 2016 11:14:14 PM HKT Grzegorz Kasprowicz wrote:
> [GK] If you don't use ARM, you still get hardened SDRAM controller and GBE
> MACs.

Yes, that's what I was saying: you cannot get rid of them (i.e. use their pins 
like other IOs). So you need to use the Zynq-specific features of Vivado, 
interface your design to the hardened AXI system, use some obscure Vivado 
"wizard" to set up the SDRAM, and write a software driver for Xilinx's GbE 
MAC. All doable, but annoying.

> > * we want to avoid RTMs and instead put the DAC/ADCs on the AMC card
> > and have analog plug-ins using the FMC form factor (see my document).
> > **Are you sure you would get noise performance from such setup that
> > satisfies you?
> 
> Unless the FMC connector is particularly bad with analog signals, I think it
> should not be worse than the current hardware. [GK] All depends how you
> deliver the clock for such DAC. This is the weakest point of such solution.

The DACs will be mounted on the DSP cards (AMC directly), not the RF 
daughtercards. And the DSP cards will have an external clocking option, that 
may use semi-rigid coax if need be.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] FW: initial specification of the project

2016-03-26 Thread Sébastien Bourdeauducq
On Saturday, 26 March 2016 4:06:17 PM HKT Slichter, Daniel H. (Fed) wrote:
> The cost savings from using FMC, which might amount to $50 per AMC, are not
> worth if the crosstalk will make the cards not useful for researchers.  

It's not only about cost of the connector - the RF daughter cards will often 
need some digital signals (e.g. SPI) for control. FMC provides plenty of pins 
for that. Mixing two different types of board-to-board connectors will cause 
mechanical problems and I think we should not do that. If two types of 
connectors are needed, one of them should use cables.

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] Fwd: RE: FW: initial specification of the project

2016-03-26 Thread Sébastien Bourdeauducq
--  Forwarded Message  --

Subject: RE: FW: initial specification of the project
Date: Friday, 25 March 2016, 12:24:02 PM HKT
From: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>
To: 'Robert Jördens' <r...@m-labs.hk>, 'Sébastien Bourdeauducq' <s...@m-labs.hk>
CC: 'Joe Britton' <joe.britton@gmail.com>

Hi all.
Of course we can to for SFP+, but with QSFP we get much higher bandwidth. 
There are also popular QSFP to SFP cables capable of 10Gbit/s operation.
Best regards,
Greg

-Original Message-
From: Robert Jördens [mailto:r...@m-labs.hk] 
Sent: Thursday, March 24, 2016 11:48 AM
To: Sébastien Bourdeauducq <s...@m-labs.hk>
Cc: Grzegorz Kasprowicz <gkasp...@elka.pw.edu.pl>; Joe Britton 
<joe.britton@gmail.com>
Subject: Re: FW: initial specification of the project

Hello all,

thanks for the documents. On top of Sébastien's questions and remarks:
Can we go for plain SFP(+) and not QSFP? Then even 4 channels might fit onto a 
single-width card.

Robert.


-
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] ARTIQ hardware proposal

2016-03-19 Thread Sébastien Bourdeauducq
On Friday, 18 March 2016 4:27:53 PM HKT Ben Keitch wrote:
> This gives me a 404:
> 
> http://ssl.serverraum.org/lists-archive/artiq/attachments/20160318/ce9d656c/
> attachment.pdf

This links works fine here. Anyway, here is the slightly updated version. 
Change log: https://github.com/m-labs/artiq-hardware

Sébastien


artiq_hardware.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


[ARTIQ] ARTIQ hardware proposal

2016-03-18 Thread Sébastien Bourdeauducq
Hi,

I've been collecting ideas about the new hardware we've been talking about for 
a while. Please see the attached document and let me know what you think.

Sébastien


artiq_hardware.pdf
Description: Adobe PDF document
___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


Re: [ARTIQ] installing ARTIQ

2016-03-15 Thread Sébastien Bourdeauducq
On Wednesday, 16 March 2016 3:55:53 AM HKT Jonathan Mizrahi wrote:
> Ah, adding the dev channel allowed me to install it. The installation
> instructions say that the dev channel is only necessary to use the
> development version of ARTIQ, so I had not added it.

Well, there aren't any non-development versions of ARTIQ right now, though we 
should release the first one in a few weeks.
 
> Where is the required FPGA support code, if it's not in the OpenOCD
> releases?

It is in its Git repository, from which they should make a release at some 
point:
http://openocd.org/repos/
https://github.com/ntfreak/openocd

Sébastien

___
ARTIQ mailing list
https://ssl.serverraum.org/lists/listinfo/artiq


  1   2   >