Re: [systemd-devel] About stable network interface names

[Greg KH]

On Sat, Jun 10, 2017 at 09:08:17AM +0300, Andrei Borzenkov wrote:
> 09.06.2017 23:42, Martin Wilck пишет:
> > On Tue, 2017-06-06 at 21:40 +0300, Andrei Borzenkov wrote:
> >>
> >> Can device and function really change? My understanding is that
> >> device
> >> part is determined by bus physical wiring and function by PCI card
> >> design; this leaves bus as volatile run-time enumeration value.
> > 
> > For PCIe, that's only true for the "function" part.
> > https://superuser.com/questions/1060808/how-is-the-device-determined-in
> > -pci-enumeration-bus-device-function
> > 
> 
> I do not see anything there that would imply device designation is
> random. You have PCIe switch port instead of IDSEL wiring but port is
> most likely hardwired and does not change. Actually this article says
> the same
> 
> --><--
> Since each device has its own independent set of wires, the device IDs
> are essentially all hard-coded
> --><--

"essentially" does not mean "will never change".  Some BIOSes will
renumber them, some will not, as it's not a PCI requirement it can, and
will, happen.

Yes, for lots of people this will be fine and nothing will ever change,
but you can not guarantee it will never happen for all types of
machines, sorry.

That's just the nature of dynamic busses :)

good luck!

greg k-h
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Andrei Borzenkov]

09.06.2017 23:42, Martin Wilck пишет:
> On Tue, 2017-06-06 at 21:40 +0300, Andrei Borzenkov wrote:
>>
>> Can device and function really change? My understanding is that
>> device
>> part is determined by bus physical wiring and function by PCI card
>> design; this leaves bus as volatile run-time enumeration value.
> 
> For PCIe, that's only true for the "function" part.
> https://superuser.com/questions/1060808/how-is-the-device-determined-in
> -pci-enumeration-bus-device-function
> 

I do not see anything there that would imply device designation is
random. You have PCIe switch port instead of IDSEL wiring but port is
most likely hardwired and does not change. Actually this article says
the same

--><--
Since each device has its own independent set of wires, the device IDs
are essentially all hard-coded
--><--


> The systemd docs are a bit misleading for PCIe, as they talk about
> "physical/geographical location" for the common enp$Xs$Yf$Z scheme,
> which is actually just the BDF. The interface on my laptop is called
> enp0s31f6 although the laptop doesn't have "slot 31". (1)
> 

The reason for this scheme is to generate unique names during boot. All
this buzz about "predictability" etc is just usual marketing stuff to
better sell it.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Martin Wilck]

On Tue, 2017-06-06 at 21:40 +0300, Andrei Borzenkov wrote:
> 
> Can device and function really change? My understanding is that
> device
> part is determined by bus physical wiring and function by PCI card
> design; this leaves bus as volatile run-time enumeration value.

For PCIe, that's only true for the "function" part.
https://superuser.com/questions/1060808/how-is-the-device-determined-in
-pci-enumeration-bus-device-function

The systemd docs are a bit misleading for PCIe, as they talk about
"physical/geographical location" for the common enp$Xs$Yf$Z scheme,
which is actually just the BDF. The interface on my laptop is called
enp0s31f6 although the laptop doesn't have "slot 31". (1)

Martin

(1) Well, that's actually because the manufacturer saved the money to
implement the DMI BIOS correctly: there is even a DMI type 41 entry for
onboard LAN, but the PCI device number (sic!) is wrong.

-- 
Dr. Martin Wilck , Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Andrei Borzenkov]

06.06.2017 16:20, Martin Wilck пишет:
> As others have remarked already, PCI bus-device-function is subject to
> change.
> 

Can device and function really change? My understanding is that device
part is determined by bus physical wiring and function by PCI card
design; this leaves bus as volatile run-time enumeration value.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Martin Wilck]

On Mon, 2017-05-29 at 02:35 +0200, Cesare Leonardi wrote:

> I ask because I've done several tests, with different motherboards, 
> adding and removing PCI-express cards and that expectation was not 
> satisfied in many cases.
> 
> For example, in one of those tests I initially had this setup:
> Integrated NIC: enp9s0
> PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
> PCIE2 (x16): empty
> PCIE3 (x1): dual port ethernet card [enp7s0, enp8s0]
> 
> Then i inserted a SATA controller in the PCIE2 slot and three NICs
> got 
> renamed:
> Integrated NIC: enp10s0
> PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
> PCIE2 (x16): empty
> PCIE3 (x1): dual port ethernet card [enp8s0, enp9s0]
> 
> Why?
> Didn't this interface naming scheme supposed to avoid this kind of
> renaming?
>  From what i've experimented network names are guaranteed to be
> stable 
> across reboots *and* if you doesn't add or remove hardware.

As others have remarked already, PCI bus-device-function is subject to
change.

Yet this highlights a problem with the current "predictable" network
device naming scheme. "biosdevname" uses information from various
sources, including ACPI _DSM method and SMBIOS type 41 and type 9. The
systemd device naming scheme (or the kernel code in pci-label.c, for
that matter) evaluates only _DSM and SMBIOS type 41, and not SMBIOS
type 9. The latter is necessary for mapping system PCI slot numbers to
bus-device-function tuples. Obviously, the physical slot a controller
is connected to is less likely to change than the bus-device-function
number, so exposing it might make a lot of sense.

All of this requires support on the BIOS/Firmware side - without that,
none of the "predictable" schemes work correctly.

Regards,
Martin

-- 
Dr. Martin Wilck , Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Greg KH]

On Mon, May 29, 2017 at 11:34:13AM +0200, Cesare Leonardi wrote:
> On 29/05/2017 07:10, Greg KH wrote:
> > Anyway, PCI can, and will sometimes, renumber it's devices on booting
> > again, that's a known issue.  It is rare, but as you have found out,
> > will happen.  So anything depending on PCI numbers will change.  Nothing
> > we can really do about that.
> 
> Do you mean that it could rarely happen on boot also without doing any
> change to the hardware?

Yes it can, I used to have a machine that renumbered the PCI buses every
5th boot or something, was fun for testing PCI changes on :)

> So, to avoid surprises, in case of multiple NICs it's highly recommendable
> anyway to hook interface naming to MAC address, isn't it?

Yes, or something else that you know will be stable in the system.

good luck!

greg k-h
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Reindl Harald]

Am 29.05.2017 um 11:40 schrieb Lennart Poettering:

Well, different naming strategies have different advantages and
disadvantages. If you use the MAC address, then replacing hardware
becomes harder, and you can't cover hardware that doesn't have fixed
MAC addresses (or VMs)

than KVM or whatever you mean with VMs needs to be fixed

on our VMware cluster all nods are Fedora installed 2008 and all are 
configured with MAC addresses in the ifcfg-ethX files, where moved 
between different vCenter versions and hosts and the MAC is stable

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Lennart Poettering]

On Mon, 29.05.17 11:34, Cesare Leonardi (celeo...@gmail.com) wrote:

> On 29/05/2017 07:10, Greg KH wrote:
> > > For example, in one of those tests I initially had this setup:
> > > Integrated NIC: enp9s0
> > > PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
> > > PCIE2 (x16): empty
> > > PCIE3 (x1): dual port ethernet card [enp7s0, enp8s0]
> > > 
> > > Then i inserted a SATA controller in the PCIE2 slot and three NICs got
> > > renamed:
> > > Integrated NIC: enp10s0
> > > PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
> > > PCIE2 (x16): empty
> > > PCIE3 (x1): dual port ethernet card [enp8s0, enp9s0]
> > 
> > Do you mean to show that PCIE2 is still empty here?
> 
> No, sorry, cut and paste error. In the last case PCIE2 was occupied by the
> SATA controller.
> 
> > Anyway, PCI can, and will sometimes, renumber it's devices on booting
> > again, that's a known issue.  It is rare, but as you have found out,
> > will happen.  So anything depending on PCI numbers will change.  Nothing
> > we can really do about that.
> 
> Do you mean that it could rarely happen on boot also without doing any
> change to the hardware?

Well, the firmware can do whatever it wants at any time, and this is
really up to the firmware. Ideally firmware would keep things strictly
stable, to make this useful, but you know how firmware is...

> So, to avoid surprises, in case of multiple NICs it's highly recommendable
> anyway to hook interface naming to MAC address, isn't it?

Well, different naming strategies have different advantages and
disadvantages. If you use the MAC address, then replacing hardware
becomes harder, and you can't cover hardware that doesn't have fixed
MAC addresses (or VMs).

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Cesare Leonardi]

On 29/05/2017 07:10, Greg KH wrote:

For example, in one of those tests I initially had this setup:
Integrated NIC: enp9s0
PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
PCIE2 (x16): empty
PCIE3 (x1): dual port ethernet card [enp7s0, enp8s0]

Then i inserted a SATA controller in the PCIE2 slot and three NICs got
renamed:
Integrated NIC: enp10s0
PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
PCIE2 (x16): empty
PCIE3 (x1): dual port ethernet card [enp8s0, enp9s0]


Do you mean to show that PCIE2 is still empty here?


No, sorry, cut and paste error. In the last case PCIE2 was occupied by 
the SATA controller.



Anyway, PCI can, and will sometimes, renumber it's devices on booting
again, that's a known issue.  It is rare, but as you have found out,
will happen.  So anything depending on PCI numbers will change.  Nothing
we can really do about that.


Do you mean that it could rarely happen on boot also without doing any 
change to the hardware?


So, to avoid surprises, in case of multiple NICs it's highly 
recommendable anyway to hook interface naming to MAC address, isn't it?


Thank you for the clarifications, Greg.

Cesare.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] About stable network interface names

[Greg KH]

On Mon, May 29, 2017 at 02:35:12AM +0200, Cesare Leonardi wrote:
> I ask because I've done several tests, with different motherboards, adding
> and removing PCI-express cards and that expectation was not satisfied in
> many cases.
> 
> For example, in one of those tests I initially had this setup:
> Integrated NIC: enp9s0
> PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
> PCIE2 (x16): empty
> PCIE3 (x1): dual port ethernet card [enp7s0, enp8s0]
> 
> Then i inserted a SATA controller in the PCIE2 slot and three NICs got
> renamed:
> Integrated NIC: enp10s0
> PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
> PCIE2 (x16): empty
> PCIE3 (x1): dual port ethernet card [enp8s0, enp9s0]

Do you mean to show that PCIE2 is still empty here?

Anyway, PCI can, and will sometimes, renumber it's devices on booting
again, that's a known issue.  It is rare, but as you have found out,
will happen.  So anything depending on PCI numbers will change.  Nothing
we can really do about that.

thanks,

greg k-h
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] About stable network interface names

[Cesare Leonardi]

Hello, I've some dubts related to predictable network interface names.

If I understand correctly the reference document about this topic is:
https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/

Take this paragraph from that document:
-
We believe it is a good default choice to generalize the scheme 
pioneered by "biosdevname". Assigning fixed names based on 
firmware/topology/location information has the big advantage that the 
names are fully automatic, fully predictable, that they stay fixed even 
if hardware is added or removed (i.e. no reenumeration takes place) and 
that broken hardware can be replaced seamlessly.

-

And this one:
-
Stable interface names even when hardware is added or removed, i.e. no 
re-enumeration takes place (to the level the firmware permits this).

-

They say that no re-enumeration take place if hardware is added or 
removed, with the last quote that puts a depends on firmware.


I ask because I've done several tests, with different motherboards, 
adding and removing PCI-express cards and that expectation was not 
satisfied in many cases.


For example, in one of those tests I initially had this setup:
Integrated NIC: enp9s0
PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
PCIE2 (x16): empty
PCIE3 (x1): dual port ethernet card [enp7s0, enp8s0]

Then i inserted a SATA controller in the PCIE2 slot and three NICs got 
renamed:

Integrated NIC: enp10s0
PCIE1 (x1): dual port ethernet card [enp3s0, enp4s0]
PCIE2 (x16): empty
PCIE3 (x1): dual port ethernet card [enp8s0, enp9s0]

Why?
Didn't this interface naming scheme supposed to avoid this kind of renaming?
From what i've experimented network names are guaranteed to be stable 
across reboots *and* if you doesn't add or remove hardware.


Please, can you clarify on this?

Cesare.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel