Am Dienstag, den 22.05.2018, 13:24 +0200 schrieb Michael Biebl: > Am 22.05.2018 um 12:50 schrieb Benjamin Drung: > > Am Dienstag, den 22.05.2018, 12:36 +0200 schrieb Michael Biebl: > > > Am 22.05.2018 um 12:27 schrieb Benjamin Drung: > > > > What do you suggest to do instead? > > > > > > For that I know too little how rdma works, so I can't really > > > answer > > > that. > > > > Mellanox ConnectX architecture is: mlx4_core is the lower level > > PCI driver which register on the PCI id, and protocol specific > > drivers > > are depended on it: mlx4_en - for Ethernet and mlx4_ib for > > Infiniband. > > NIC could have multiple ports which can change their type > > dynamically. > > We use the request_module() call to load the relevant protocol > > driver > > when needed: on loading time or at port type change event. The > > ib_ipoib > > kernel modules provides IP over Infiniband support. > > > > The order is as following: > > 1) The kernel loads mlx4_core on boot (due to device detection) > > 2) mlx4_core loads mlx4_ib > > 3) udev triggers rdma-load-modules@infiniband.service > > 4) rdma-load-modules@infiniband.service loads ib_ipoib (can be > > configured in /etc/rdma) > > Afaics there is a race condition here. How do you guarantee that > ib_ipoib is loaded before systemd-udev-settle.service is started?
rdma-load-modules@infiniband.service declares to run before network- pre.target (which runs before networking.service). So it is sufficient that rdma-load-modules@infiniband.service is loaded before networking.service (not systemd-udev-settle.service) is started. > > 5) networking.service brings up the IPoIB devices > > Not sure if ifupdown is the best fit for such a configuration, at > least not networking.service. Why not? These IPoIB devices behave like normal network devices. > Maybe allow-hotplug would be a better fit or > another networking configuration system which is more dynamic and > does listen to uevents? I tried to use allow-hotplug, but it does not work at all (see initial bug report for example configuration). -- Benjamin Drung System Developer Debian & Ubuntu Developer ProfitBricks GmbH Greifswalder Str. 207 10405 Berlin Email: benjamin.dr...@profitbricks.com URL: https://www.profitbricks.de Sitz der Gesellschaft: Berlin Registergericht: Amtsgericht Charlottenburg, HRB 125506 B Geschäftsführer: Achim Weiss, Matthias Steinberg, Christoph Steffens
signature.asc
Description: This is a digitally signed message part
