Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
Hi, After this change missed F37, it's now testable in F38: https://src.fedoraproject.org/rpms/systemd/pull-request/100 systemd-253~rc2-3.fc38 Thanks Dusty, zbyszek! Thomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
Hi, On 6/27/22 13:18, Zbigniew Jędrzejewski-Szmek wrote: > On Mon, Jun 27, 2022 at 11:20:13AM +0200, Hans de Goede wrote: >> Hi, >> >> On 6/27/22 10:10, Zbigniew Jędrzejewski-Szmek wrote: >>> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote: This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == The `systemd-udev` package installs `"/usr/lib/systemd/network/99-default.link"`, which sets `Link.MACAddressPolicy=persistent`. This proposal is to change it to set `Link.MACAddressPolicy=none` to stop changing the MAC address. This is particularly important for bridge and bond devices. This change can either only apply to bridge/bond devices, or to various software devices. That is to be discussed. >>> >>> Hi! >>> >>> I already participated in the upstream discussion, so what I write >>> here will be a restatement to some extent, but with a look from the >>> side of Fedora specifically. >>> >>> The proposal has two variants: 1. just changing the policy to >>> MACAddressPolicy=none >>> or 2. limiting the change to bridge and bond devices. >>> >>> Re variant 1: MACAddressPolicy=persistent applies to all devices that don't >>> have >>> a hardware address. The proposal as written (blanket MACAddressPolicy=none) >>> would change behaviour for all kinds of devices, incl. e.g. software >>> devices like veths, and cheap hardware devices without a fixed MAC. >>> The proposal doesn't provide any justification for this (except for >>> simplicity of implementation) and this variant seems pretty bad and >>> I'm strongly opposed. >> >> >> >> I did not know systemd/udev can save/restore MAC addresses for devices >> where there is no MAC address. I have looking into a related issue on my >> todo list. There are some cheap x86 devices with wifi which don't have >> a nvram (eeprom) chip instead they load nvram from /lib/firmware. On most >> devices there is enough otp inside the wifi chip that they do have a unique >> MAC inside the wifi chip's otp memory so everything works well. >> >> But on some devices that otp is either not there or not programmed, so >> they take their MAC from the nvram in /lib/firmware which means that >> all devices from the same model end up using the same MAC (which is >> defined in the nvram in /lib/firmware). >> >> Am I reading the above correctly that udev/systemd already is able >> to deal with this and I just need to make it so that the device has >> no MAC set at all and then the first time systemd sees this it will >> generate a random MAC and use that on future boots ? > > Yes. (With the following clarification: the MAC address is generated > from as some hash of machine-id + device name. So "first time" and > "any time" are the same — no state is ever saved, you're supposed to > get the same result for a specific device on a specific system.) > >> Any docs on this / code you can point me to how systemd determines >> it needs to do this ? > > udev looks at /sys/class/net/hub0/addr_assign_type. > But the docs are not great :( The description is split between two man pages: > - > https://www.freedesktop.org/software/systemd/man/systemd.link.html#MACAddressPolicy= > - > https://www.freedesktop.org/software/systemd/man/systemd.net-naming-scheme.html Thanks this was useful. I've submitted a kernel patch upstream now which fixes the described issue by using a random MAC + setting the addr_assign_type: https://lore.kernel.org/linux-wireless/20220708133712.102179-2-hdego...@redhat.com/ Regards, Hans > >> >> >> The same also applies to some bluetooth devices: >> >> https://github.com/bluez/bluez/issues/107 >> >> I wonder if similar functionality for bluetooth would >> be welcome in systemd ? > > I think so. From what I can see, we don't do much setup for bluetooth > devices, so I'm not sure if udevd is a better place for this than > e.g. bluez. But if yes, it's nice to do things uniformly, i.e. in this > case treat bluebooth interfaces more like other network interfaces… > The implementation should be simple, since it'd be a question of > extending existing code to cover one more case. > > Zbyszek > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 7/6/22 04:37, Lennart Poettering wrote: > On Di, 05.07.22 22:35, Dusty Mabe (du...@dustymabe.com) wrote: > >> On 6/25/22 15:06, Vipul Siddharth wrote: >>> This document represents a proposed Change. As part of the Changes >>> process, proposals are publicly announced in order to receive >>> community feedback. This proposal will only be implemented if approved >>> by the Fedora Engineering Steering Committee. >>> >>> >>> == Summary == >>> >>> The `systemd-udev` package installs >>> `"/usr/lib/systemd/network/99-default.link"`, >>> which sets `Link.MACAddressPolicy=persistent`. This proposal is to >>> change it to set `Link.MACAddressPolicy=none` to stop changing the MAC >>> address. >>> This is particularly important for bridge and bond devices. >>> >>> This change can either only apply to bridge/bond devices, or to >>> various software devices. That is to be discussed. >> >> Based on the feedback here on the list we have modified the scope of this >> proposal >> to now be limited to changing the MACAddressPolicy=none for bond/bridge/team >> devices only. >> >> New summary: >> >> ``` >> The systemd-udev package installs >> "/usr/lib/systemd/network/99-default.link", which sets >> Link.MACAddressPolicy=persistent for all software NIC devices. This proposal >> is to add to the policy so that we use Link.MACAddressPolicy=none for >> bond/bridge/team devices. >> ``` >> >> https://fedoraproject.org/wiki/Changes/MAC_Address_Policy_none > > So, with changing this back you'll also break all those setups where > it is assumed the bridge MAC just works and is stable and independent > from the devices added to it and the order in which they are added in. One thing we do need to think about here is the "upgrade" case. We could consider leaving existing (upgrading) systems alone. > > What makes you so sure, that changing this *again* is so much better > than just leaving it the way it is now? This change isn't precisely > new, and changing stuff forth and back comes at quite a price. Indeed it does come at a price, which is why we're discussing the merits here and also why we scaled back the proposal to just cover the use case where it matters the most. For example, this kernel documentation is a good place where a user is now confused because the kernel behaves one way for bonds, but systemd delivers configuration to make it behave differently: https://wiki.linuxfoundation.org/networking/bonding#where_does_a_bonding_device_get_its_mac_address_from If you look at the longer term landscape (i.e. years from now) the change is worth it if it's the right change to make. I think the fact that RHEL9 didn't pick up the original change is an indication of it's value in enterprise environments where bond/bridge/team devices are more common. > > Generally, if we change behaviour like this, then the pros must > seriously outweight the cons. Now you might argue that when the > original change was made that wasn't the case, that's a valid opinion, > though one I disagree with. But that has no effect on today, on the > question whether changing it again is worth it. I am pretty sure that > there are *also* numerous scripts that benefit from the predictability > and stability you get with the status quo, and which you'll break if > you revert to the old state – again. There is a lot that happens upstream systemd that we should more carefully consider in Fedora. For example this probably should have been proposed to Fedora as a change then (by systemd maintainer in Fedora I suspect) and the merits examined at that point in time. > > I am not convinced we should flip flop on this all the time. Yes, it's > unfortunate that people tripped over this, but you are not really > making it better if you then break it *again*, given the benefit is > unclear, and the major software creating bridges/bonds/… doesn't care > anyway (i.e. NM, networkd, …). This definitely affects NetworkManager and people do care.. See https://github.com/coreos/fedora-coreos-tracker/issues/919 https://github.com/systemd/systemd/issues/15208 > > I for one do not see where you'd get the crystal ball to look into to > know that by changing this *again* you'll make bazillions of people > happy, and only very few people sad, because you break their stuff. It > might very well be that you'll make more people sad because you break > their stuff again, than were happy before. The discussion here is one way to gather feedback. There is also the collective expertise of the members of FESCO, who make decisions like this all the time. > > Also, let's not forget that allowing users to set the policy is a good > thing, we should let them. Given that the original change was already > made a lot of software that cannot work with such changes has already > been updated to override the default policy. That's a good thing, > since the overall system becomes more robust and people can more > safely change the policies locally, with less breakage to
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Di, 05.07.22 22:35, Dusty Mabe (du...@dustymabe.com) wrote: > On 6/25/22 15:06, Vipul Siddharth wrote: > > This document represents a proposed Change. As part of the Changes > > process, proposals are publicly announced in order to receive > > community feedback. This proposal will only be implemented if approved > > by the Fedora Engineering Steering Committee. > > > > > > == Summary == > > > > The `systemd-udev` package installs > > `"/usr/lib/systemd/network/99-default.link"`, > > which sets `Link.MACAddressPolicy=persistent`. This proposal is to > > change it to set `Link.MACAddressPolicy=none` to stop changing the MAC > > address. > > This is particularly important for bridge and bond devices. > > > > This change can either only apply to bridge/bond devices, or to > > various software devices. That is to be discussed. > > Based on the feedback here on the list we have modified the scope of this > proposal > to now be limited to changing the MACAddressPolicy=none for bond/bridge/team > devices only. > > New summary: > > ``` > The systemd-udev package installs "/usr/lib/systemd/network/99-default.link", > which sets Link.MACAddressPolicy=persistent for all software NIC devices. > This proposal is to add to the policy so that we use > Link.MACAddressPolicy=none for bond/bridge/team devices. > ``` > > https://fedoraproject.org/wiki/Changes/MAC_Address_Policy_none So, with changing this back you'll also break all those setups where it is assumed the bridge MAC just works and is stable and independent from the devices added to it and the order in which they are added in. What makes you so sure, that changing this *again* is so much better than just leaving it the way it is now? This change isn't precisely new, and changing stuff forth and back comes at quite a price. Generally, if we change behaviour like this, then the pros must seriously outweight the cons. Now you might argue that when the original change was made that wasn't the case, that's a valid opinion, though one I disagree with. But that has no effect on today, on the question whether changing it again is worth it. I am pretty sure that there are *also* numerous scripts that benefit from the predictability and stability you get with the status quo, and which you'll break if you revert to the old state – again. I am not convinced we should flip flop on this all the time. Yes, it's unfortunate that people tripped over this, but you are not really making it better if you then break it *again*, given the benefit is unclear, and the major software creating bridges/bonds/… doesn't care anyway (i.e. NM, networkd, …). I for one do not see where you'd get the crystal ball to look into to know that by changing this *again* you'll make bazillions of people happy, and only very few people sad, because you break their stuff. It might very well be that you'll make more people sad because you break their stuff again, than were happy before. Also, let's not forget that allowing users to set the policy is a good thing, we should let them. Given that the original change was already made a lot of software that cannot work with such changes has already been updated to override the default policy. That's a good thing, since the overall system becomes more robust and people can more safely change the policies locally, with less breakage to expect. if you now revert to the status quo ante, then you basically also say: fuck it, we don't care that software is fixed to work with changed policies, let's keep things brittle that you cannot change policies anymore effectively because we are sticking our head in the sand and don't care that they are fixed. So, I am not convinced. I can accept that this wasn't handled particularly nicely originally. My proposal for addressing this is via documentation, i.e update the udev and iproute docs to explicitly point to the issue and how to deal with it, with an example drop-in to make it easy to deal with it. Lennart -- Lennart Poettering, Berlin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 6/25/22 15:06, Vipul Siddharth wrote: > This document represents a proposed Change. As part of the Changes > process, proposals are publicly announced in order to receive > community feedback. This proposal will only be implemented if approved > by the Fedora Engineering Steering Committee. > > > == Summary == > > The `systemd-udev` package installs > `"/usr/lib/systemd/network/99-default.link"`, > which sets `Link.MACAddressPolicy=persistent`. This proposal is to > change it to set `Link.MACAddressPolicy=none` to stop changing the MAC > address. > This is particularly important for bridge and bond devices. > > This change can either only apply to bridge/bond devices, or to > various software devices. That is to be discussed. Based on the feedback here on the list we have modified the scope of this proposal to now be limited to changing the MACAddressPolicy=none for bond/bridge/team devices only. New summary: ``` The systemd-udev package installs "/usr/lib/systemd/network/99-default.link", which sets Link.MACAddressPolicy=persistent for all software NIC devices. This proposal is to add to the policy so that we use Link.MACAddressPolicy=none for bond/bridge/team devices. ``` https://fedoraproject.org/wiki/Changes/MAC_Address_Policy_none ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 6/27/22 22:35, Neal Gompa wrote: > On Sat, Jun 25, 2022 at 3:06 PM Vipul Siddharth > wrote: >> >> This document represents a proposed Change. As part of the Changes >> process, proposals are publicly announced in order to receive >> community feedback. This proposal will only be implemented if approved >> by the Fedora Engineering Steering Committee. >> >> >> == Summary == >> >> The `systemd-udev` package installs >> `"/usr/lib/systemd/network/99-default.link"`, >> which sets `Link.MACAddressPolicy=persistent`. This proposal is to >> change it to set `Link.MACAddressPolicy=none` to stop changing the MAC >> address. >> This is particularly important for bridge and bond devices. >> >> This change can either only apply to bridge/bond devices, or to >> various software devices. That is to be discussed. >> >> == Owner == >> >> * Name: [[User:thaller|Thomas Haller]] (NetworkManager) >> * Email: >> * Name: [[User:dustymabe| Dusty Mabe]] (Fedora CoreOS) >> * Email: >> >> >> == Detailed Description == >> >> On Fedora, udev by default changes the MAC address of a wide range of >> software devices. >> This was introduced by systemd 242 in early 2019 (Fedora 31), when >> `MACAddressPolicy=` was >> extended to affect more types of devices. >> >> With `MACAddressPolicy=persistent` udev's aim is to provide a stable >> MAC address, otherwise >> the kernel will assign a random one. However, that can cause problems: >> >> Firstly, software devices are always created by some tool that has >> plans for the device. The tool may not >> expect that udev is going to change the MAC address and races against >> that. The best solution >> for the tool is to set the MAC address when creating an interface. >> This will prevent >> udev from changing the MAC address according to the MACAddressPolicy. >> Otherwise, the tool should wait for udev to initialize the device to >> avoid the race. In theory, a >> tool is always advised to wait for udev to initialize the device. >> However, if it were not for MACAddressPolicy, >> in common scenarios udev doesn't do anything relevant for software >> devices to make that necessary. >> >> Secondly, for interface types bridge and bond, an unset MAC address >> has a special meaning >> to the kernel and the MAC address of the first port is used. If udev >> changes the MAC >> address, that no longer works. >> Now the generated MAC address is not directly discoverable as it is >> based on `/etc/machine-id` >> ([https://www.man7.org/linux/man-pages/man5/machine-id.5.html >> machine-id(5)]), among >> other data. Even if there were a tool to easily calculate the MAC >> address, it could be cumbersome >> to use it without logging into the machine first. The MAC address can >> directly affect the >> assigned IP address, for example when using DHCP. When booting a new >> virtual machine, the user might >> know the MAC address of the (virtual) "physical" interfaces. When >> bonding/bridging those >> interfaces, the bond/bridge would get one of the well known MAC >> addresses. `MACAddressPolicy=persistent` >> interferes with that. >> >> The goal of persistent policy is to provide a stable MAC address. Note >> that if the >> tool or user who created the interface would want a certain MAC address, they >> have all the means to set it already. That applies regardless whether >> the tool is >> iproute2, NetworkManager, systemd-networkd. Neither NetworkManager nor >> systemd-networkd >> rely on udev's MACAddressPolicy for setting the MAC address. This >> behavior is mostly >> useful for plain `ip link add`, but it's unclear which real world user >> wants this behavior. >> >> Of course, the user is welcome to configure the MAC address in any way >> they want. Including, >> dropping a link file that sets `MACAddressPolicy=persistent`. The >> problem is once udev sets a MAC address, >> it cannot be unset. Which makes this problematic to do by default. >> >> While Fedora inherited this behavior from upstream systemd, RHEL-9 >> does not follow this behavior >> ([https://gitlab.com/redhat/centos-stream/rpms/systemd/-/blob/c8953519504bf2e694bfbc2b02a456c1056f252e/0028-udev-net-setup-link-change-the-default-MACAddressPol.patch#L43 >> centos9], >> [https://bugzilla.redhat.com/show_bug.cgi?id=1921094 rh#1921094]). For >> RHEL-8, this doesn't >> apply because the systemd there is too old to change the MAC address >> of most software devices. >> >> This could be either implemented by patching >> `/usr/lib/systemd/network/99-default.link` >> to have a different policy, or by dropping a link file with higher >> priority. In the latter >> case, that override could be shipped either by udev or even by >> NetworkManager. >> >> Another option is to change the scope of this proposal to only change to >> `MACAddressPolicy=none` for the device types where this causes the most >> issues >> (bridge/bond/team). >> >> >> == Feedback == >> >> This was also discussed on upstream systemd mailing list >>
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
Once upon a time, Neal Gompa said: > I'd be interested in seeing the justification for the revert in RHEL > 9. From my perspective, it doesn't make sense to disable this because > it makes bonds/teams/etc. device dependent. This can be particularly > bad if you've got a bond and then you swap out the hardware over time. > The MAC address would change on the bond, which could break any layer > 2 rules in place on the network. The flip side of that is that converting a stand-alone interface to a LAG changes MAC address on Linux, while it doesn't on most other things like routers and switches. Replacing a NIC is a more uncommon event IMHO, which already would be disruptive to things that care about the MAC in a non-LAG setup. I don't care strongly either way myself; I've started copying the MAC manually when I set up a LAG, but it would be nice to drop that step. -- Chris Adams ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Sat, Jun 25, 2022 at 3:06 PM Vipul Siddharth wrote: > > This document represents a proposed Change. As part of the Changes > process, proposals are publicly announced in order to receive > community feedback. This proposal will only be implemented if approved > by the Fedora Engineering Steering Committee. > > > == Summary == > > The `systemd-udev` package installs > `"/usr/lib/systemd/network/99-default.link"`, > which sets `Link.MACAddressPolicy=persistent`. This proposal is to > change it to set `Link.MACAddressPolicy=none` to stop changing the MAC > address. > This is particularly important for bridge and bond devices. > > This change can either only apply to bridge/bond devices, or to > various software devices. That is to be discussed. > > == Owner == > > * Name: [[User:thaller|Thomas Haller]] (NetworkManager) > * Email: > * Name: [[User:dustymabe| Dusty Mabe]] (Fedora CoreOS) > * Email: > > > == Detailed Description == > > On Fedora, udev by default changes the MAC address of a wide range of > software devices. > This was introduced by systemd 242 in early 2019 (Fedora 31), when > `MACAddressPolicy=` was > extended to affect more types of devices. > > With `MACAddressPolicy=persistent` udev's aim is to provide a stable > MAC address, otherwise > the kernel will assign a random one. However, that can cause problems: > > Firstly, software devices are always created by some tool that has > plans for the device. The tool may not > expect that udev is going to change the MAC address and races against > that. The best solution > for the tool is to set the MAC address when creating an interface. > This will prevent > udev from changing the MAC address according to the MACAddressPolicy. > Otherwise, the tool should wait for udev to initialize the device to > avoid the race. In theory, a > tool is always advised to wait for udev to initialize the device. > However, if it were not for MACAddressPolicy, > in common scenarios udev doesn't do anything relevant for software > devices to make that necessary. > > Secondly, for interface types bridge and bond, an unset MAC address > has a special meaning > to the kernel and the MAC address of the first port is used. If udev > changes the MAC > address, that no longer works. > Now the generated MAC address is not directly discoverable as it is > based on `/etc/machine-id` > ([https://www.man7.org/linux/man-pages/man5/machine-id.5.html > machine-id(5)]), among > other data. Even if there were a tool to easily calculate the MAC > address, it could be cumbersome > to use it without logging into the machine first. The MAC address can > directly affect the > assigned IP address, for example when using DHCP. When booting a new > virtual machine, the user might > know the MAC address of the (virtual) "physical" interfaces. When > bonding/bridging those > interfaces, the bond/bridge would get one of the well known MAC > addresses. `MACAddressPolicy=persistent` > interferes with that. > > The goal of persistent policy is to provide a stable MAC address. Note > that if the > tool or user who created the interface would want a certain MAC address, they > have all the means to set it already. That applies regardless whether > the tool is > iproute2, NetworkManager, systemd-networkd. Neither NetworkManager nor > systemd-networkd > rely on udev's MACAddressPolicy for setting the MAC address. This > behavior is mostly > useful for plain `ip link add`, but it's unclear which real world user > wants this behavior. > > Of course, the user is welcome to configure the MAC address in any way > they want. Including, > dropping a link file that sets `MACAddressPolicy=persistent`. The > problem is once udev sets a MAC address, > it cannot be unset. Which makes this problematic to do by default. > > While Fedora inherited this behavior from upstream systemd, RHEL-9 > does not follow this behavior > ([https://gitlab.com/redhat/centos-stream/rpms/systemd/-/blob/c8953519504bf2e694bfbc2b02a456c1056f252e/0028-udev-net-setup-link-change-the-default-MACAddressPol.patch#L43 > centos9], > [https://bugzilla.redhat.com/show_bug.cgi?id=1921094 rh#1921094]). For > RHEL-8, this doesn't > apply because the systemd there is too old to change the MAC address > of most software devices. > > This could be either implemented by patching > `/usr/lib/systemd/network/99-default.link` > to have a different policy, or by dropping a link file with higher > priority. In the latter > case, that override could be shipped either by udev or even by NetworkManager. > > Another option is to change the scope of this proposal to only change to > `MACAddressPolicy=none` for the device types where this causes the most issues > (bridge/bond/team). > > > == Feedback == > > This was also discussed on upstream systemd mailing list > [https://lists.freedesktop.org/archives/systemd-devel/2022-May/047893.html > here]. > The upstream systemd maintainers' opinion is that the current udev > behavior is desirable, but accepts that
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
> The `systemd-udev` package installs > `"/usr/lib/systemd/network/99-default.link"`, > which sets `Link.MACAddressPolicy=persistent`. This proposal is to > change it to set `Link.MACAddressPolicy=none` to stop changing the MAC > address. This is particularly important for bridge and bond devices. > > This change can either only apply to bridge/bond devices, or to > various software devices. That is to be discussed. IMHO, the default should be "none" for all devices. I do not see why systemd spoofs MAC addresses by default. Kevin Kofler ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 6/27/22 13:03, Thomas Haller wrote:
> On Mon, 2022-06-27 at 10:10 +0200, Zbigniew Jędrzejewski-Szmek wrote:
>> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
>>> This document represents a proposed Change. As part of the Changes
>>> process, proposals are publicly announced in order to receive
>>> community feedback. This proposal will only be implemented if
>>> approved
>>> by the Fedora Engineering Steering Committee.
>>>
>>>
>>> == Summary ==
>>>
>>> The `systemd-udev` package installs
>>> `"/usr/lib/systemd/network/99-default.link"`,
>>> which sets `Link.MACAddressPolicy=persistent`. This proposal is to
>>> change it to set `Link.MACAddressPolicy=none` to stop changing the
>>> MAC address.
>>> This is particularly important for bridge and bond devices.
>>>
>>> This change can either only apply to bridge/bond devices, or to
>>> various software devices. That is to be discussed.
>>
>> Hi!
>>
>> I already participated in the upstream discussion, so what I write
>> here will be a restatement to some extent, but with a look from the
>> side of Fedora specifically.
>>
>> The proposal has two variants: 1. just changing the policy to
>> MACAddressPolicy=none
>> or 2. limiting the change to bridge and bond devices.
>>
>> Re variant 1: MACAddressPolicy=persistent applies to all devices that
>> don't have
>> a hardware address. The proposal as written (blanket
>> MACAddressPolicy=none)
>> would change behaviour for all kinds of devices, incl. e.g. software
>> devices like veths, and cheap hardware devices without a fixed MAC.
>
> the proposal was only about software devices.
>
> The difference is that software devices are explicitly added by some
> tool (if we ignore module parameters like "max_bonds" -- which on
> fedora/systemd is already 0 by default).
>
> A hardware device gets automatically added when loading the kernel.
> More importantly, udev already does renaming, so any user is well
> advised to wait for udev to settles. At which point, they can also wait
> for the MAC address policy.
>
>
>> The proposal doesn't provide any justification for this (except for
>> simplicity of implementation) and this variant seems pretty bad and
>> I'm strongly opposed.
>
> The first point: the race. And that udev touches interfaces created by
> somebody, without being explicitly told to do so.
>
> Yes, udev always does something to the device, like attaching
> attributes such as ID_NET_NAMING_SCHEME. But aside the MAC address
> policy, it doesn't do anything relevant where a race would matter.
>
>>
>> Re variant 2: the proposal limited to brige/bond devices seems much
>> more
>> reasonable. In particular, the case described below of a server
>> (virtualized
>> or not) in a big datacenter is the one case where the benefits of
>> MACAddressPolicy=none are clearly visible. I still don't think it's
>> worth
>> changing the default, but here the cost:benefit ratio is much closer.
>>
>>> == Benefit to Fedora ==
>>>
>>> Pros:
>>>
>>> - Consistent behavior with RHEL8 and RHEL9.
>>>
>>> - Avoid race of udev and the tool that creates the interface.
>>
>> The race will happen if the creation is done in a specific way. But
>> at
>> the same time, even the Change proposal describes how to avoid the
>> race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the
>> situation
>> can be summarized as "we have a bunch of 'big' tools that create
>> devices like NetworkManager or systemd-networkd, which already know
>> or
>> can be easily fixed to avoid the race, and a manual tool which can be
>> invoked
>> in a way that avoids the race". Instead of changing the default in
>> udev we
>> could educate people how to invoke it better.
>
> I don't think that big projects matter much. They can learn this. For
> example, docker had a problem with this ([1]), which presumably got
> fixed long ago.
>
> [1]
> https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/15#note_162509
>
> The problem are smaller projects and user scripts/tools.
>
> For example, NetworkManager-ci contains hundreds of scripts for
> testing, and -- while we should know better -- we don't wait for udev
> after creating a veth interface for testing. That is no problem for us,
> we just set MACAddressPolicy=none. But it makes me wonder, just how
> many "naive" scripts are out there that don't get this right.
At least for block devices, waiting for udev to finish can actually
be a non-trivial overhead, especially when udev has been explicitly
told to *not* do anything with the devices in question. It has a
measurable impact on how long LVM commands take to run, and LVM is
known to be a significant contributor to the amount of time Qubes OS
needs to start a VM.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 6/27/22 13:03, Thomas Haller wrote: > On Mon, 2022-06-27 at 10:10 +0200, Zbigniew Jędrzejewski-Szmek wrote: > >> - For "big" users (the datacenter case), changing the policy make >> sense, >> but at the same time, those folks can just insert a policy >> override, >> they're most likely using some ansible/puppet/cheffy thingy. >> - RHEL is more of the "big" case, Fedora is more the "small" case. >> Sometimes RHEL and Fedora have different defaults, sometimes RHEL >> takes years to follow what Fedora does. > > I personally care mostly about RHEL here and to find out what to do. So > if we agree that RHEL is allowed to have a different downstream > behavior than Fedora, I'd be satisfied! RHEL and Fedora *can* deviate, but I think it's valuable for them to have alignment whenever possible. i.e. if they deviate the benefit should outweigh the cost. Optimally we find a default that works best in most cases that we can share. > > >> >> So overall, I think this proposal would make most sense when limited >> to specific types of hardware (bridge and bond?), and also to >> specific >> spins: it's a better fit for CoreOS than Workstation… > > Makes sense. > > I think CoreOS would be tempted to set MACAddressPolicy=none. Thus, a > major Fedora user would go against the distro default. That raises a > question about why Workstation behaves different. But if CoreOS (or > other spins) are allowed to do that, I think Dusty would also be > satisfied. While I agree it's nice to have the flexibility to deviate on just certain variants, I'd much prefer to have the policy set Fedora wide if we can. Every delta is a cost and our goal in Fedora CoreOS is to try to not deviate from Fedora unless it really really benefits the users. The more we deviate over time the more maintenance burden we accumulate. If this change got rejected at the Fedora level I don't know if Fedora CoreOS would pick it up individually. From one perspective it wouldn't make sense: deviating from the rest of Fedora, but from another perspective it would make sense: deviating from our downstream of RHCOS (based on RHEL). Again, I'd really prefer to find a compromise here. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, 2022-06-27 at 17:26 +0100, Tom Hughes wrote: > On 27/06/2022 17:09, Tom Hughes via devel wrote: > > On 27/06/2022 17:05, Thomas Haller wrote: > > > > > On Mon, 2022-06-27 at 13:09 +0100, Tom Hughes via devel wrote: > > > > > > > > Twice now I have had to go and reconfigure my networks after a > > > > Fedora > > > > upgrade has changed the MAC assignment policy. > > > > > > Interesting. Are you sure it was twice? I thought it changed > > > "only" > > > once in F31 (2019). > > > > No it has just changed again in F36 at least for bridges, back > > to what it was before the previous change I believe. > > I've had a look through the git log for my DNS and taking one > machine as an example: > > initially - bridge had MAC of 00:b0:c2:02:4c:f3 from member > Aug 2015 (F22) - bridge changed to 1a:81:14:46:3c:c7 > Jan 2020 (F31) - bridge changed to fe:e3:75:bd:6a:8c > May 2022 (F36) - bridge changed back to 1a:81:14:46:3c:c7 > > Only the first one corresponds to a member, so I think F22 is > where persistent addresses were first introduced. I'm not sure > what happened in F31 than then got reverted in F36. > > Tom > Hi Tom, Thanks for checking. I don't know why that would have happened. It also depends which tools you are using for creating the bridge. I agree, this is something to avoid! And maybe a the best reason to not change anything on Fedora 37. Thomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, 2022-06-27 at 13:34 +0200, Zbigniew Jędrzejewski-Szmek wrote: > > I'm not "blaming" the tools, I completely understand that they were > written a long time ago. But in fact the issue is fairly generic: any > software which interacts with devices that udev also touches MUST > wait > for udev to be done with the device. It's not just the MAC address > policy, but also other rules that users may configure locally, sysctl > configuration, etc. Without synchronization, one runs into races and > errors from the tools when they try to configure things in parallel. Yes, any software is well advised to wait for udev. But if it were not for MACAddressPolicy=, default Fedora would not ship much of relevant to make that necessary in practice. For example, NetworkManager's udev rules set some attributes. But those are mainly for NetworkManager itself (and NetworkManager knows to wait). If you have a tool that creates a software device, then there is no problem to not wait for those. Also, "other rules that the user confiugres locally", are entirely user's choice. Of course, the tools that this user runs, need to interoperate with the user's environment. > > > > IMHO The more compelling reason is compliance with MAC address > > filtering > > applied to VMs. > > FWIW, conditionalizing the policy on ConditionVirtualization=!vm > would > be trivial to add… Maybe having ConditionKernelCommandLine= could allow convenient customization for users and could be set by CoreOS (or suggested/documented). Thomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, 2022-06-27 at 10:10 +0200, Zbigniew Jędrzejewski-Szmek wrote:
> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
> > This document represents a proposed Change. As part of the Changes
> > process, proposals are publicly announced in order to receive
> > community feedback. This proposal will only be implemented if
> > approved
> > by the Fedora Engineering Steering Committee.
> >
> >
> > == Summary ==
> >
> > The `systemd-udev` package installs
> > `"/usr/lib/systemd/network/99-default.link"`,
> > which sets `Link.MACAddressPolicy=persistent`. This proposal is to
> > change it to set `Link.MACAddressPolicy=none` to stop changing the
> > MAC address.
> > This is particularly important for bridge and bond devices.
> >
> > This change can either only apply to bridge/bond devices, or to
> > various software devices. That is to be discussed.
>
> Hi!
>
> I already participated in the upstream discussion, so what I write
> here will be a restatement to some extent, but with a look from the
> side of Fedora specifically.
>
> The proposal has two variants: 1. just changing the policy to
> MACAddressPolicy=none
> or 2. limiting the change to bridge and bond devices.
>
> Re variant 1: MACAddressPolicy=persistent applies to all devices that
> don't have
> a hardware address. The proposal as written (blanket
> MACAddressPolicy=none)
> would change behaviour for all kinds of devices, incl. e.g. software
> devices like veths, and cheap hardware devices without a fixed MAC.
the proposal was only about software devices.
The difference is that software devices are explicitly added by some
tool (if we ignore module parameters like "max_bonds" -- which on
fedora/systemd is already 0 by default).
A hardware device gets automatically added when loading the kernel.
More importantly, udev already does renaming, so any user is well
advised to wait for udev to settles. At which point, they can also wait
for the MAC address policy.
> The proposal doesn't provide any justification for this (except for
> simplicity of implementation) and this variant seems pretty bad and
> I'm strongly opposed.
The first point: the race. And that udev touches interfaces created by
somebody, without being explicitly told to do so.
Yes, udev always does something to the device, like attaching
attributes such as ID_NET_NAMING_SCHEME. But aside the MAC address
policy, it doesn't do anything relevant where a race would matter.
>
> Re variant 2: the proposal limited to brige/bond devices seems much
> more
> reasonable. In particular, the case described below of a server
> (virtualized
> or not) in a big datacenter is the one case where the benefits of
> MACAddressPolicy=none are clearly visible. I still don't think it's
> worth
> changing the default, but here the cost:benefit ratio is much closer.
>
> > == Benefit to Fedora ==
> >
> > Pros:
> >
> > - Consistent behavior with RHEL8 and RHEL9.
> >
> > - Avoid race of udev and the tool that creates the interface.
>
> The race will happen if the creation is done in a specific way. But
> at
> the same time, even the Change proposal describes how to avoid the
> race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the
> situation
> can be summarized as "we have a bunch of 'big' tools that create
> devices like NetworkManager or systemd-networkd, which already know
> or
> can be easily fixed to avoid the race, and a manual tool which can be
> invoked
> in a way that avoids the race". Instead of changing the default in
> udev we
> could educate people how to invoke it better.
I don't think that big projects matter much. They can learn this. For
example, docker had a problem with this ([1]), which presumably got
fixed long ago.
[1]
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/15#note_162509
The problem are smaller projects and user scripts/tools.
For example, NetworkManager-ci contains hundreds of scripts for
testing, and -- while we should know better -- we don't wait for udev
after creating a veth interface for testing. That is no problem for us,
we just set MACAddressPolicy=none. But it makes me wonder, just how
many "naive" scripts are out there that don't get this right.
You say, "the race will happen if the creation is done in a specific
way". The problem is that the specific way is the most straight forward
one.
Also, to educate people how to use iproute2 (?) better, seems not
clearly best. Is there really a problem on the user's side when they
are creating an interface and don't expect udev to interfere?
> > - Bridge and bond interfaces can get the MAC addresses from their
> > first port.
> >
> > In the case of `MACAddressPolicy=none` for a bond (or bridge) the
> > bond will
> > get a MAC related to one of its physical NIC devices. In the case
> > of
> > provisioning
> > new systems (especially in a large datacenter) information about
> > the server
> > can be used to configure the network environment (DHCP,
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, Jun 27, 2022 at 06:08:13PM +0200, Thomas Haller wrote: > On Mon, 2022-06-27 at 10:10 +0200, Zbigniew Jędrzejewski-Szmek wrote: > > > > > > - Deviate from upstream systemd. > > > > It is also important to mention that Fedora will "deviate" from > > itself > > (it's former self). We would be changing a default in place since > > ~2013 [1]. > > > > [1] https://github.com/systemd/systemd/commit/16b9b87aee > > > Hi, > > Is that the case? I think in practice, it only changed in F31/2019 with > systemd 242: > https://github.com/systemd/systemd/blob/eaee50fee2d0baec9116ae76945f18fef4b8ec2f/NEWS#L4835 > > Granted, 2019 is also a long time... 2013 — MACAddressPolicy=persistent mostly for hardware devices 2019 — MACAddressPolicy=persistent for bridge and bond and some other stuff Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 27/06/2022 17:09, Tom Hughes via devel wrote: On 27/06/2022 17:05, Thomas Haller wrote: On Mon, 2022-06-27 at 13:09 +0100, Tom Hughes via devel wrote: Twice now I have had to go and reconfigure my networks after a Fedora upgrade has changed the MAC assignment policy. Interesting. Are you sure it was twice? I thought it changed "only" once in F31 (2019). No it has just changed again in F36 at least for bridges, back to what it was before the previous change I believe. I've had a look through the git log for my DNS and taking one machine as an example: initially - bridge had MAC of 00:b0:c2:02:4c:f3 from member Aug 2015 (F22) - bridge changed to 1a:81:14:46:3c:c7 Jan 2020 (F31) - bridge changed to fe:e3:75:bd:6a:8c May 2022 (F36) - bridge changed back to 1a:81:14:46:3c:c7 Only the first one corresponds to a member, so I think F22 is where persistent addresses were first introduced. I'm not sure what happened in F31 than then got reverted in F36. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 27/06/2022 17:05, Thomas Haller wrote: On Mon, 2022-06-27 at 13:09 +0100, Tom Hughes via devel wrote: Twice now I have had to go and reconfigure my networks after a Fedora upgrade has changed the MAC assignment policy. Interesting. Are you sure it was twice? I thought it changed "only" once in F31 (2019). No it has just changed again in F36 at least for bridges, back to what it was before the previous change I believe. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, 2022-06-27 at 10:10 +0200, Zbigniew Jędrzejewski-Szmek wrote: > > > > - Deviate from upstream systemd. > > It is also important to mention that Fedora will "deviate" from > itself > (it's former self). We would be changing a default in place since > ~2013 [1]. > > [1] https://github.com/systemd/systemd/commit/16b9b87aee Hi, Is that the case? I think in practice, it only changed in F31/2019 with systemd 242: https://github.com/systemd/systemd/blob/eaee50fee2d0baec9116ae76945f18fef4b8ec2f/NEWS#L4835 Granted, 2019 is also a long time... Thomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
Hi, On Mon, 2022-06-27 at 13:09 +0100, Tom Hughes via devel wrote: > > Twice now I have had to go and reconfigure my networks after a Fedora > upgrade has changed the MAC assignment policy. Interesting. Are you sure it was twice? I thought it changed "only" once in F31 (2019). Thomas ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 6/27/22 07:34, Zbigniew Jędrzejewski-Szmek wrote:
> On Mon, Jun 27, 2022 at 09:40:53AM +0100, Daniel P. Berrangé wrote:
>> On Mon, Jun 27, 2022 at 10:10:31AM +0200, Zbigniew Jędrzejewski-Szmek wrote:
>>> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
The `systemd-udev` package installs
`"/usr/lib/systemd/network/99-default.link"`,
which sets `Link.MACAddressPolicy=persistent`. This proposal is to
change it to set `Link.MACAddressPolicy=none` to stop changing the MAC
address.
This is particularly important for bridge and bond devices.
This change can either only apply to bridge/bond devices, or to
various software devices. That is to be discussed.
>>>
>>> Hi!
>>>
>>> I already participated in the upstream discussion, so what I write
>>> here will be a restatement to some extent, but with a look from the
>>> side of Fedora specifically.
>>>
>>> The proposal has two variants: 1. just changing the policy to
>>> MACAddressPolicy=none
>>> or 2. limiting the change to bridge and bond devices.
>>>
>>> Re variant 1: MACAddressPolicy=persistent applies to all devices that don't
>>> have
>>> a hardware address. The proposal as written (blanket MACAddressPolicy=none)
>>> would change behaviour for all kinds of devices, incl. e.g. software
>>> devices like veths, and cheap hardware devices without a fixed MAC.
>>> The proposal doesn't provide any justification for this (except for
>>> simplicity of implementation) and this variant seems pretty bad and
>>> I'm strongly opposed.
>>>
>>> Re variant 2: the proposal limited to brige/bond devices seems much more
>>> reasonable. In particular, the case described below of a server (virtualized
>>> or not) in a big datacenter is the one case where the benefits of
>>> MACAddressPolicy=none are clearly visible. I still don't think it's worth
>>> changing the default, but here the cost:benefit ratio is much closer.
>>>
== Benefit to Fedora ==
Pros:
- Consistent behavior with RHEL8 and RHEL9.
- Avoid race of udev and the tool that creates the interface.
>>>
>>> The race will happen if the creation is done in a specific way. But at
>>> the same time, even the Change proposal describes how to avoid the
>>> race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the situation
>>> can be summarized as "we have a bunch of 'big' tools that create
>>> devices like NetworkManager or systemd-networkd, which already know or
>>> can be easily fixed to avoid the race, and a manual tool which can be
>>> invoked
>>> in a way that avoids the race". Instead of changing the default in udev we
>>> could educate people how to invoke it better.
>>
>> This comes across as blaming every networking tool out there for
>> having their previously correct & working behaviour be broken by
>> a systemd change imposing new requirements on them.
>
> I was a bit sloppy in my phrasing. NetworkManager and systemd-networkd
> already do the right thing. 'ip link' is the odd one out. Various other
> tools (e.g. netplan or Debian's network scripts) don't implement this
> internally,
> but call e.g. NetworkManager or 'ip'. So if we changed 'ip', this would go
> a long way to solving the problem (I'm may be wrong on some details here,
> please correct me if necessary.)
>
> I'm not "blaming" the tools, I completely understand that they were
> written a long time ago. But in fact the issue is fairly generic: any
> software which interacts with devices that udev also touches MUST wait
> for udev to be done with the device. It's not just the MAC address
> policy, but also other rules that users may configure locally, sysctl
> configuration, etc. Without synchronization, one runs into races and
> errors from the tools when they try to configure things in parallel.
>
>> Are there any notable tools which actually follow the usage pattern
>> described, or are we in effect having to fix *everything*, including
>> an uncountable amount of documentation, blogs, examples, most of
>> which will prove impossible to fix in practice.
>>
- Bridge and bond interfaces can get the MAC addresses from their first
port.
In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond will
get a MAC related to one of its physical NIC devices. In the case of
provisioning
new systems (especially in a large datacenter) information about the server
can be used to configure the network environment (DHCP, Firewall, etc)
before
the server is ever even powered on. This does mean that you may have to
update
your network environment configuration if you replace a NIC (con),
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On 6/27/22 04:10, Zbigniew Jędrzejewski-Szmek wrote: > > > Re variant 2: the proposal limited to brige/bond devices seems much more > reasonable. In particular, the case described below of a server (virtualized > or not) in a big datacenter is the one case where the benefits of > MACAddressPolicy=none are clearly visible. I still don't think it's worth > changing the default, but here the cost:benefit ratio is much closer. I think that's reasonable. The proposal owners are willing to make this change to the proposal so that it only applies to bond/bridge/team devices if that is more acceptable. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
As I said before I don't care that much what the policy is but I do care very much that Fedora keeps changing it. Twice now I have had to go and reconfigure my networks after a Fedora upgrade has changed the MAC assignment policy. My vote is therefore to leave it as it is so that I don't have to go and change everything again! Basically I don't care what the MAC of a machine is but I care very much if it changes because I have to update DHCP so that it can continue to issue the correct IPs and DNS for IPv6 addresses which change when the MAC changes. Tom On 25/06/2022 20:06, Vipul Siddharth wrote: This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == The `systemd-udev` package installs `"/usr/lib/systemd/network/99-default.link"`, which sets `Link.MACAddressPolicy=persistent`. This proposal is to change it to set `Link.MACAddressPolicy=none` to stop changing the MAC address. This is particularly important for bridge and bond devices. This change can either only apply to bridge/bond devices, or to various software devices. That is to be discussed. == Owner == * Name: [[User:thaller|Thomas Haller]] (NetworkManager) * Email: * Name: [[User:dustymabe| Dusty Mabe]] (Fedora CoreOS) * Email: == Detailed Description == On Fedora, udev by default changes the MAC address of a wide range of software devices. This was introduced by systemd 242 in early 2019 (Fedora 31), when `MACAddressPolicy=` was extended to affect more types of devices. With `MACAddressPolicy=persistent` udev's aim is to provide a stable MAC address, otherwise the kernel will assign a random one. However, that can cause problems: Firstly, software devices are always created by some tool that has plans for the device. The tool may not expect that udev is going to change the MAC address and races against that. The best solution for the tool is to set the MAC address when creating an interface. This will prevent udev from changing the MAC address according to the MACAddressPolicy. Otherwise, the tool should wait for udev to initialize the device to avoid the race. In theory, a tool is always advised to wait for udev to initialize the device. However, if it were not for MACAddressPolicy, in common scenarios udev doesn't do anything relevant for software devices to make that necessary. Secondly, for interface types bridge and bond, an unset MAC address has a special meaning to the kernel and the MAC address of the first port is used. If udev changes the MAC address, that no longer works. Now the generated MAC address is not directly discoverable as it is based on `/etc/machine-id` ([https://www.man7.org/linux/man-pages/man5/machine-id.5.html machine-id(5)]), among other data. Even if there were a tool to easily calculate the MAC address, it could be cumbersome to use it without logging into the machine first. The MAC address can directly affect the assigned IP address, for example when using DHCP. When booting a new virtual machine, the user might know the MAC address of the (virtual) "physical" interfaces. When bonding/bridging those interfaces, the bond/bridge would get one of the well known MAC addresses. `MACAddressPolicy=persistent` interferes with that. The goal of persistent policy is to provide a stable MAC address. Note that if the tool or user who created the interface would want a certain MAC address, they have all the means to set it already. That applies regardless whether the tool is iproute2, NetworkManager, systemd-networkd. Neither NetworkManager nor systemd-networkd rely on udev's MACAddressPolicy for setting the MAC address. This behavior is mostly useful for plain `ip link add`, but it's unclear which real world user wants this behavior. Of course, the user is welcome to configure the MAC address in any way they want. Including, dropping a link file that sets `MACAddressPolicy=persistent`. The problem is once udev sets a MAC address, it cannot be unset. Which makes this problematic to do by default. While Fedora inherited this behavior from upstream systemd, RHEL-9 does not follow this behavior ([https://gitlab.com/redhat/centos-stream/rpms/systemd/-/blob/c8953519504bf2e694bfbc2b02a456c1056f252e/0028-udev-net-setup-link-change-the-default-MACAddressPol.patch#L43 centos9], [https://bugzilla.redhat.com/show_bug.cgi?id=1921094 rh#1921094]). For RHEL-8, this doesn't apply because the systemd there is too old to change the MAC address of most software devices. This could be either implemented by patching `/usr/lib/systemd/network/99-default.link` to have a different policy, or by dropping a link file with higher priority. In the latter case, that override could be shipped either by udev or even by NetworkManager. Another option is to change the scope of this proposal to only change to `MACAddressPolicy=none`
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, Jun 27, 2022 at 09:40:53AM +0100, Daniel P. Berrangé wrote:
> On Mon, Jun 27, 2022 at 10:10:31AM +0200, Zbigniew Jędrzejewski-Szmek wrote:
> > On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
> > > This document represents a proposed Change. As part of the Changes
> > > process, proposals are publicly announced in order to receive
> > > community feedback. This proposal will only be implemented if approved
> > > by the Fedora Engineering Steering Committee.
> > >
> > >
> > > == Summary ==
> > >
> > > The `systemd-udev` package installs
> > > `"/usr/lib/systemd/network/99-default.link"`,
> > > which sets `Link.MACAddressPolicy=persistent`. This proposal is to
> > > change it to set `Link.MACAddressPolicy=none` to stop changing the MAC
> > > address.
> > > This is particularly important for bridge and bond devices.
> > >
> > > This change can either only apply to bridge/bond devices, or to
> > > various software devices. That is to be discussed.
> >
> > Hi!
> >
> > I already participated in the upstream discussion, so what I write
> > here will be a restatement to some extent, but with a look from the
> > side of Fedora specifically.
> >
> > The proposal has two variants: 1. just changing the policy to
> > MACAddressPolicy=none
> > or 2. limiting the change to bridge and bond devices.
> >
> > Re variant 1: MACAddressPolicy=persistent applies to all devices that don't
> > have
> > a hardware address. The proposal as written (blanket MACAddressPolicy=none)
> > would change behaviour for all kinds of devices, incl. e.g. software
> > devices like veths, and cheap hardware devices without a fixed MAC.
> > The proposal doesn't provide any justification for this (except for
> > simplicity of implementation) and this variant seems pretty bad and
> > I'm strongly opposed.
> >
> > Re variant 2: the proposal limited to brige/bond devices seems much more
> > reasonable. In particular, the case described below of a server (virtualized
> > or not) in a big datacenter is the one case where the benefits of
> > MACAddressPolicy=none are clearly visible. I still don't think it's worth
> > changing the default, but here the cost:benefit ratio is much closer.
> >
> > > == Benefit to Fedora ==
> > >
> > > Pros:
> > >
> > > - Consistent behavior with RHEL8 and RHEL9.
> > >
> > > - Avoid race of udev and the tool that creates the interface.
> >
> > The race will happen if the creation is done in a specific way. But at
> > the same time, even the Change proposal describes how to avoid the
> > race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the situation
> > can be summarized as "we have a bunch of 'big' tools that create
> > devices like NetworkManager or systemd-networkd, which already know or
> > can be easily fixed to avoid the race, and a manual tool which can be
> > invoked
> > in a way that avoids the race". Instead of changing the default in udev we
> > could educate people how to invoke it better.
>
> This comes across as blaming every networking tool out there for
> having their previously correct & working behaviour be broken by
> a systemd change imposing new requirements on them.
I was a bit sloppy in my phrasing. NetworkManager and systemd-networkd
already do the right thing. 'ip link' is the odd one out. Various other
tools (e.g. netplan or Debian's network scripts) don't implement this
internally,
but call e.g. NetworkManager or 'ip'. So if we changed 'ip', this would go
a long way to solving the problem (I'm may be wrong on some details here,
please correct me if necessary.)
I'm not "blaming" the tools, I completely understand that they were
written a long time ago. But in fact the issue is fairly generic: any
software which interacts with devices that udev also touches MUST wait
for udev to be done with the device. It's not just the MAC address
policy, but also other rules that users may configure locally, sysctl
configuration, etc. Without synchronization, one runs into races and
errors from the tools when they try to configure things in parallel.
> Are there any notable tools which actually follow the usage pattern
> described, or are we in effect having to fix *everything*, including
> an uncountable amount of documentation, blogs, examples, most of
> which will prove impossible to fix in practice.
>
> > > - Bridge and bond interfaces can get the MAC addresses from their first
> > > port.
> > >
> > > In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond
> > > will
> > > get a MAC related to one of its physical NIC devices. In the case of
> > > provisioning
> > > new systems (especially in a large datacenter) information about the
> > > server
> > > can be used to configure the network environment (DHCP, Firewall, etc)
> > > before
> > > the server is ever even powered on. This does mean that you may have to
> > > update
> > > your network environment configuration if you replace a NIC (con), but
> > > that you
> > > won't
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, Jun 27, 2022 at 11:20:13AM +0200, Hans de Goede wrote: > Hi, > > On 6/27/22 10:10, Zbigniew Jędrzejewski-Szmek wrote: > > On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote: > >> This document represents a proposed Change. As part of the Changes > >> process, proposals are publicly announced in order to receive > >> community feedback. This proposal will only be implemented if approved > >> by the Fedora Engineering Steering Committee. > >> > >> > >> == Summary == > >> > >> The `systemd-udev` package installs > >> `"/usr/lib/systemd/network/99-default.link"`, > >> which sets `Link.MACAddressPolicy=persistent`. This proposal is to > >> change it to set `Link.MACAddressPolicy=none` to stop changing the MAC > >> address. > >> This is particularly important for bridge and bond devices. > >> > >> This change can either only apply to bridge/bond devices, or to > >> various software devices. That is to be discussed. > > > > Hi! > > > > I already participated in the upstream discussion, so what I write > > here will be a restatement to some extent, but with a look from the > > side of Fedora specifically. > > > > The proposal has two variants: 1. just changing the policy to > > MACAddressPolicy=none > > or 2. limiting the change to bridge and bond devices. > > > > Re variant 1: MACAddressPolicy=persistent applies to all devices that don't > > have > > a hardware address. The proposal as written (blanket MACAddressPolicy=none) > > would change behaviour for all kinds of devices, incl. e.g. software > > devices like veths, and cheap hardware devices without a fixed MAC. > > The proposal doesn't provide any justification for this (except for > > simplicity of implementation) and this variant seems pretty bad and > > I'm strongly opposed. > > > > I did not know systemd/udev can save/restore MAC addresses for devices > where there is no MAC address. I have looking into a related issue on my > todo list. There are some cheap x86 devices with wifi which don't have > a nvram (eeprom) chip instead they load nvram from /lib/firmware. On most > devices there is enough otp inside the wifi chip that they do have a unique > MAC inside the wifi chip's otp memory so everything works well. > > But on some devices that otp is either not there or not programmed, so > they take their MAC from the nvram in /lib/firmware which means that > all devices from the same model end up using the same MAC (which is > defined in the nvram in /lib/firmware). > > Am I reading the above correctly that udev/systemd already is able > to deal with this and I just need to make it so that the device has > no MAC set at all and then the first time systemd sees this it will > generate a random MAC and use that on future boots ? Yes. (With the following clarification: the MAC address is generated from as some hash of machine-id + device name. So "first time" and "any time" are the same — no state is ever saved, you're supposed to get the same result for a specific device on a specific system.) > Any docs on this / code you can point me to how systemd determines > it needs to do this ? udev looks at /sys/class/net/hub0/addr_assign_type. But the docs are not great :( The description is split between two man pages: - https://www.freedesktop.org/software/systemd/man/systemd.link.html#MACAddressPolicy= - https://www.freedesktop.org/software/systemd/man/systemd.net-naming-scheme.html > > > The same also applies to some bluetooth devices: > > https://github.com/bluez/bluez/issues/107 > > I wonder if similar functionality for bluetooth would > be welcome in systemd ? I think so. From what I can see, we don't do much setup for bluetooth devices, so I'm not sure if udevd is a better place for this than e.g. bluez. But if yes, it's nice to do things uniformly, i.e. in this case treat bluebooth interfaces more like other network interfaces… The implementation should be simple, since it'd be a question of extending existing code to cover one more case. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
Hi,
On 6/27/22 10:10, Zbigniew Jędrzejewski-Szmek wrote:
> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
>> This document represents a proposed Change. As part of the Changes
>> process, proposals are publicly announced in order to receive
>> community feedback. This proposal will only be implemented if approved
>> by the Fedora Engineering Steering Committee.
>>
>>
>> == Summary ==
>>
>> The `systemd-udev` package installs
>> `"/usr/lib/systemd/network/99-default.link"`,
>> which sets `Link.MACAddressPolicy=persistent`. This proposal is to
>> change it to set `Link.MACAddressPolicy=none` to stop changing the MAC
>> address.
>> This is particularly important for bridge and bond devices.
>>
>> This change can either only apply to bridge/bond devices, or to
>> various software devices. That is to be discussed.
>
> Hi!
>
> I already participated in the upstream discussion, so what I write
> here will be a restatement to some extent, but with a look from the
> side of Fedora specifically.
>
> The proposal has two variants: 1. just changing the policy to
> MACAddressPolicy=none
> or 2. limiting the change to bridge and bond devices.
>
> Re variant 1: MACAddressPolicy=persistent applies to all devices that don't
> have
> a hardware address. The proposal as written (blanket MACAddressPolicy=none)
> would change behaviour for all kinds of devices, incl. e.g. software
> devices like veths, and cheap hardware devices without a fixed MAC.
> The proposal doesn't provide any justification for this (except for
> simplicity of implementation) and this variant seems pretty bad and
> I'm strongly opposed.
I did not know systemd/udev can save/restore MAC addresses for devices
where there is no MAC address. I have looking into a related issue on my
todo list. There are some cheap x86 devices with wifi which don't have
a nvram (eeprom) chip instead they load nvram from /lib/firmware. On most
devices there is enough otp inside the wifi chip that they do have a unique
MAC inside the wifi chip's otp memory so everything works well.
But on some devices that otp is either not there or not programmed, so
they take their MAC from the nvram in /lib/firmware which means that
all devices from the same model end up using the same MAC (which is
defined in the nvram in /lib/firmware).
Am I reading the above correctly that udev/systemd already is able
to deal with this and I just need to make it so that the device has
no MAC set at all and then the first time systemd sees this it will
generate a random MAC and use that on future boots ?
Any docs on this / code you can point me to how systemd determines
it needs to do this ?
The same also applies to some bluetooth devices:
https://github.com/bluez/bluez/issues/107
I wonder if similar functionality for bluetooth would
be welcome in systemd ?
Regards,
Hans
>
> Re variant 2: the proposal limited to brige/bond devices seems much more
> reasonable. In particular, the case described below of a server (virtualized
> or not) in a big datacenter is the one case where the benefits of
> MACAddressPolicy=none are clearly visible. I still don't think it's worth
> changing the default, but here the cost:benefit ratio is much closer.
>
>> == Benefit to Fedora ==
>>
>> Pros:
>>
>> - Consistent behavior with RHEL8 and RHEL9.
>>
>> - Avoid race of udev and the tool that creates the interface.
>
> The race will happen if the creation is done in a specific way. But at
> the same time, even the Change proposal describes how to avoid the
> race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the situation
> can be summarized as "we have a bunch of 'big' tools that create
> devices like NetworkManager or systemd-networkd, which already know or
> can be easily fixed to avoid the race, and a manual tool which can be invoked
> in a way that avoids the race". Instead of changing the default in udev we
> could educate people how to invoke it better.
>
>> - Bridge and bond interfaces can get the MAC addresses from their first port.
>>
>> In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond will
>> get a MAC related to one of its physical NIC devices. In the case of
>> provisioning
>> new systems (especially in a large datacenter) information about the server
>> can be used to configure the network environment (DHCP, Firewall, etc) before
>> the server is ever even powered on. This does mean that you may have to
>> update
>> your network environment configuration if you replace a NIC (con), but that
>> you
>> won't have to update your network environment configuration if you re-install
>> your server (pro), which is what you'd have to do now with
>> `MACAddressPolicy=persistent`.
>
> Yep. This is *the* case.
>
>> Cons:
>>
>> - Deviate from upstream systemd.
>
> It is also important to mention that Fedora will "deviate" from itself
> (it's former self). We would be changing a default in place since ~2013 [1].
>
> [1]
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Mon, Jun 27, 2022 at 10:10:31AM +0200, Zbigniew Jędrzejewski-Szmek wrote:
> On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
> > This document represents a proposed Change. As part of the Changes
> > process, proposals are publicly announced in order to receive
> > community feedback. This proposal will only be implemented if approved
> > by the Fedora Engineering Steering Committee.
> >
> >
> > == Summary ==
> >
> > The `systemd-udev` package installs
> > `"/usr/lib/systemd/network/99-default.link"`,
> > which sets `Link.MACAddressPolicy=persistent`. This proposal is to
> > change it to set `Link.MACAddressPolicy=none` to stop changing the MAC
> > address.
> > This is particularly important for bridge and bond devices.
> >
> > This change can either only apply to bridge/bond devices, or to
> > various software devices. That is to be discussed.
>
> Hi!
>
> I already participated in the upstream discussion, so what I write
> here will be a restatement to some extent, but with a look from the
> side of Fedora specifically.
>
> The proposal has two variants: 1. just changing the policy to
> MACAddressPolicy=none
> or 2. limiting the change to bridge and bond devices.
>
> Re variant 1: MACAddressPolicy=persistent applies to all devices that don't
> have
> a hardware address. The proposal as written (blanket MACAddressPolicy=none)
> would change behaviour for all kinds of devices, incl. e.g. software
> devices like veths, and cheap hardware devices without a fixed MAC.
> The proposal doesn't provide any justification for this (except for
> simplicity of implementation) and this variant seems pretty bad and
> I'm strongly opposed.
>
> Re variant 2: the proposal limited to brige/bond devices seems much more
> reasonable. In particular, the case described below of a server (virtualized
> or not) in a big datacenter is the one case where the benefits of
> MACAddressPolicy=none are clearly visible. I still don't think it's worth
> changing the default, but here the cost:benefit ratio is much closer.
>
> > == Benefit to Fedora ==
> >
> > Pros:
> >
> > - Consistent behavior with RHEL8 and RHEL9.
> >
> > - Avoid race of udev and the tool that creates the interface.
>
> The race will happen if the creation is done in a specific way. But at
> the same time, even the Change proposal describes how to avoid the
> race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the situation
> can be summarized as "we have a bunch of 'big' tools that create
> devices like NetworkManager or systemd-networkd, which already know or
> can be easily fixed to avoid the race, and a manual tool which can be invoked
> in a way that avoids the race". Instead of changing the default in udev we
> could educate people how to invoke it better.
This comes across as blaming every networking tool out there for
having their previously correct & working behaviour be broken by
a systemd change imposing new requirements on them.
Are there any notable tools which actually follow the usage pattern
described, or are we in effect having to fix *everything*, including
an uncountable amount of documentation, blogs, examples, most of
which will prove impossible to fix in practice.
> > - Bridge and bond interfaces can get the MAC addresses from their first
> > port.
> >
> > In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond will
> > get a MAC related to one of its physical NIC devices. In the case of
> > provisioning
> > new systems (especially in a large datacenter) information about the server
> > can be used to configure the network environment (DHCP, Firewall, etc)
> > before
> > the server is ever even powered on. This does mean that you may have to
> > update
> > your network environment configuration if you replace a NIC (con), but that
> > you
> > won't have to update your network environment configuration if you
> > re-install
> > your server (pro), which is what you'd have to do now with
> > `MACAddressPolicy=persistent`.
>
> Yep. This is *the* case.
Having the bridge/bond get the MAC addr of tht first physical NIC is pretty
compelling from an automation POV, especially when the virtualziation host is
doing traffic filtering for the VM. When a mgmt app assigns a MAC & IP pair to
a guest, it is not uncommon to apply firewall rules providing anti-spoofing
protection for MAC/IP, such that the VM cannot send traffic with other MAC/IP
addresses. Libvirt provides this functionality with its 'clean-traffic'
network filter, and other virt/cloud mgmt apps do similar.
> > Cons:
> >
> > - Deviate from upstream systemd.
This is disappointing, as a great benefit of systemd is the level of
consistency it brought to distro behaviour, and I think this change
would be useful to all distros (in the context of bridge/bond devices)
> It is also important to mention that Fedora will "deviate" from itself
> (it's former self). We would be changing a default in place since ~2013 [1].
>
> [1]
Re: F37 Change Proposal: MAC Address Policy none (System-Wide Change)
On Sun, Jun 26, 2022 at 12:36:14AM +0530, Vipul Siddharth wrote:
> This document represents a proposed Change. As part of the Changes
> process, proposals are publicly announced in order to receive
> community feedback. This proposal will only be implemented if approved
> by the Fedora Engineering Steering Committee.
>
>
> == Summary ==
>
> The `systemd-udev` package installs
> `"/usr/lib/systemd/network/99-default.link"`,
> which sets `Link.MACAddressPolicy=persistent`. This proposal is to
> change it to set `Link.MACAddressPolicy=none` to stop changing the MAC
> address.
> This is particularly important for bridge and bond devices.
>
> This change can either only apply to bridge/bond devices, or to
> various software devices. That is to be discussed.
Hi!
I already participated in the upstream discussion, so what I write
here will be a restatement to some extent, but with a look from the
side of Fedora specifically.
The proposal has two variants: 1. just changing the policy to
MACAddressPolicy=none
or 2. limiting the change to bridge and bond devices.
Re variant 1: MACAddressPolicy=persistent applies to all devices that don't have
a hardware address. The proposal as written (blanket MACAddressPolicy=none)
would change behaviour for all kinds of devices, incl. e.g. software
devices like veths, and cheap hardware devices without a fixed MAC.
The proposal doesn't provide any justification for this (except for
simplicity of implementation) and this variant seems pretty bad and
I'm strongly opposed.
Re variant 2: the proposal limited to brige/bond devices seems much more
reasonable. In particular, the case described below of a server (virtualized
or not) in a big datacenter is the one case where the benefits of
MACAddressPolicy=none are clearly visible. I still don't think it's worth
changing the default, but here the cost:benefit ratio is much closer.
> == Benefit to Fedora ==
>
> Pros:
>
> - Consistent behavior with RHEL8 and RHEL9.
>
> - Avoid race of udev and the tool that creates the interface.
The race will happen if the creation is done in a specific way. But at
the same time, even the Change proposal describes how to avoid the
race ('ip link add address aa:aa:aa:aa:aa:aa type bridge'). So the situation
can be summarized as "we have a bunch of 'big' tools that create
devices like NetworkManager or systemd-networkd, which already know or
can be easily fixed to avoid the race, and a manual tool which can be invoked
in a way that avoids the race". Instead of changing the default in udev we
could educate people how to invoke it better.
> - Bridge and bond interfaces can get the MAC addresses from their first port.
>
> In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond will
> get a MAC related to one of its physical NIC devices. In the case of
> provisioning
> new systems (especially in a large datacenter) information about the server
> can be used to configure the network environment (DHCP, Firewall, etc) before
> the server is ever even powered on. This does mean that you may have to update
> your network environment configuration if you replace a NIC (con), but that
> you
> won't have to update your network environment configuration if you re-install
> your server (pro), which is what you'd have to do now with
> `MACAddressPolicy=persistent`.
Yep. This is *the* case.
> Cons:
>
> - Deviate from upstream systemd.
It is also important to mention that Fedora will "deviate" from itself
(it's former self). We would be changing a default in place since ~2013 [1].
[1] https://github.com/systemd/systemd/commit/16b9b87aee
> It is desirable that RHEL and Fedora behaves similar. A possible outcome
> could be the current behavior stays and RHEL 10 would change behavior. On the
> other hand, different distributions (or even Fedora spins) have different
> uses and needs. Deviating might be fine. In the same vein, there is also
> a desire to stay close to upstream systemd behavior. But the uses of systemd
> project go beyond Fedora/RHEL, so deviating here may also be fine.
So:
- Variant 1 is not good, variant 2 makes more sense.
- The motivating case for v.2 is the "big datacenter" case and race
when 'ip' is invoked.
- We could improve the tools: 'ip link' could be taught to wait until
udev has stopped processing the device, users can be taught to use
better invocations.
- For "small" users (individual machine admins who just install some
server and configure it after turning it on and/or swap network
cards between machines), having stable MAC addresses that doesn't
depend on the order of device discovery or removal of a single
network card seems better.
- For "big" users (the datacenter case), changing the policy make sense,
but at the same time, those folks can just insert a policy override,
they're most likely using some ansible/puppet/cheffy thingy.
- RHEL is more of the "big" case, Fedora is more the "small" case.
Sometimes RHEL and Fedora have
F37 Change Proposal: MAC Address Policy none (System-Wide Change)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == The `systemd-udev` package installs `"/usr/lib/systemd/network/99-default.link"`, which sets `Link.MACAddressPolicy=persistent`. This proposal is to change it to set `Link.MACAddressPolicy=none` to stop changing the MAC address. This is particularly important for bridge and bond devices. This change can either only apply to bridge/bond devices, or to various software devices. That is to be discussed. == Owner == * Name: [[User:thaller|Thomas Haller]] (NetworkManager) * Email: * Name: [[User:dustymabe| Dusty Mabe]] (Fedora CoreOS) * Email: == Detailed Description == On Fedora, udev by default changes the MAC address of a wide range of software devices. This was introduced by systemd 242 in early 2019 (Fedora 31), when `MACAddressPolicy=` was extended to affect more types of devices. With `MACAddressPolicy=persistent` udev's aim is to provide a stable MAC address, otherwise the kernel will assign a random one. However, that can cause problems: Firstly, software devices are always created by some tool that has plans for the device. The tool may not expect that udev is going to change the MAC address and races against that. The best solution for the tool is to set the MAC address when creating an interface. This will prevent udev from changing the MAC address according to the MACAddressPolicy. Otherwise, the tool should wait for udev to initialize the device to avoid the race. In theory, a tool is always advised to wait for udev to initialize the device. However, if it were not for MACAddressPolicy, in common scenarios udev doesn't do anything relevant for software devices to make that necessary. Secondly, for interface types bridge and bond, an unset MAC address has a special meaning to the kernel and the MAC address of the first port is used. If udev changes the MAC address, that no longer works. Now the generated MAC address is not directly discoverable as it is based on `/etc/machine-id` ([https://www.man7.org/linux/man-pages/man5/machine-id.5.html machine-id(5)]), among other data. Even if there were a tool to easily calculate the MAC address, it could be cumbersome to use it without logging into the machine first. The MAC address can directly affect the assigned IP address, for example when using DHCP. When booting a new virtual machine, the user might know the MAC address of the (virtual) "physical" interfaces. When bonding/bridging those interfaces, the bond/bridge would get one of the well known MAC addresses. `MACAddressPolicy=persistent` interferes with that. The goal of persistent policy is to provide a stable MAC address. Note that if the tool or user who created the interface would want a certain MAC address, they have all the means to set it already. That applies regardless whether the tool is iproute2, NetworkManager, systemd-networkd. Neither NetworkManager nor systemd-networkd rely on udev's MACAddressPolicy for setting the MAC address. This behavior is mostly useful for plain `ip link add`, but it's unclear which real world user wants this behavior. Of course, the user is welcome to configure the MAC address in any way they want. Including, dropping a link file that sets `MACAddressPolicy=persistent`. The problem is once udev sets a MAC address, it cannot be unset. Which makes this problematic to do by default. While Fedora inherited this behavior from upstream systemd, RHEL-9 does not follow this behavior ([https://gitlab.com/redhat/centos-stream/rpms/systemd/-/blob/c8953519504bf2e694bfbc2b02a456c1056f252e/0028-udev-net-setup-link-change-the-default-MACAddressPol.patch#L43 centos9], [https://bugzilla.redhat.com/show_bug.cgi?id=1921094 rh#1921094]). For RHEL-8, this doesn't apply because the systemd there is too old to change the MAC address of most software devices. This could be either implemented by patching `/usr/lib/systemd/network/99-default.link` to have a different policy, or by dropping a link file with higher priority. In the latter case, that override could be shipped either by udev or even by NetworkManager. Another option is to change the scope of this proposal to only change to `MACAddressPolicy=none` for the device types where this causes the most issues (bridge/bond/team). == Feedback == This was also discussed on upstream systemd mailing list [https://lists.freedesktop.org/archives/systemd-devel/2022-May/047893.html here]. The upstream systemd maintainers' opinion is that the current udev behavior is desirable, but accepts that distributions (or network stacks such as NetworkManager) may choose to change the default depending on their needs ([https://lists.freedesktop.org/archives/systemd-devel/2022-May/047943.html reference]). The goal here is to find out what the Fedora community thinks is the
F37 Change Proposal: MAC Address Policy none (System-Wide Change)
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == The `systemd-udev` package installs `"/usr/lib/systemd/network/99-default.link"`, which sets `Link.MACAddressPolicy=persistent`. This proposal is to change it to set `Link.MACAddressPolicy=none` to stop changing the MAC address. This is particularly important for bridge and bond devices. This change can either only apply to bridge/bond devices, or to various software devices. That is to be discussed. == Owner == * Name: [[User:thaller|Thomas Haller]] (NetworkManager) * Email: * Name: [[User:dustymabe| Dusty Mabe]] (Fedora CoreOS) * Email: == Detailed Description == On Fedora, udev by default changes the MAC address of a wide range of software devices. This was introduced by systemd 242 in early 2019 (Fedora 31), when `MACAddressPolicy=` was extended to affect more types of devices. With `MACAddressPolicy=persistent` udev's aim is to provide a stable MAC address, otherwise the kernel will assign a random one. However, that can cause problems: Firstly, software devices are always created by some tool that has plans for the device. The tool may not expect that udev is going to change the MAC address and races against that. The best solution for the tool is to set the MAC address when creating an interface. This will prevent udev from changing the MAC address according to the MACAddressPolicy. Otherwise, the tool should wait for udev to initialize the device to avoid the race. In theory, a tool is always advised to wait for udev to initialize the device. However, if it were not for MACAddressPolicy, in common scenarios udev doesn't do anything relevant for software devices to make that necessary. Secondly, for interface types bridge and bond, an unset MAC address has a special meaning to the kernel and the MAC address of the first port is used. If udev changes the MAC address, that no longer works. Now the generated MAC address is not directly discoverable as it is based on `/etc/machine-id` ([https://www.man7.org/linux/man-pages/man5/machine-id.5.html machine-id(5)]), among other data. Even if there were a tool to easily calculate the MAC address, it could be cumbersome to use it without logging into the machine first. The MAC address can directly affect the assigned IP address, for example when using DHCP. When booting a new virtual machine, the user might know the MAC address of the (virtual) "physical" interfaces. When bonding/bridging those interfaces, the bond/bridge would get one of the well known MAC addresses. `MACAddressPolicy=persistent` interferes with that. The goal of persistent policy is to provide a stable MAC address. Note that if the tool or user who created the interface would want a certain MAC address, they have all the means to set it already. That applies regardless whether the tool is iproute2, NetworkManager, systemd-networkd. Neither NetworkManager nor systemd-networkd rely on udev's MACAddressPolicy for setting the MAC address. This behavior is mostly useful for plain `ip link add`, but it's unclear which real world user wants this behavior. Of course, the user is welcome to configure the MAC address in any way they want. Including, dropping a link file that sets `MACAddressPolicy=persistent`. The problem is once udev sets a MAC address, it cannot be unset. Which makes this problematic to do by default. While Fedora inherited this behavior from upstream systemd, RHEL-9 does not follow this behavior ([https://gitlab.com/redhat/centos-stream/rpms/systemd/-/blob/c8953519504bf2e694bfbc2b02a456c1056f252e/0028-udev-net-setup-link-change-the-default-MACAddressPol.patch#L43 centos9], [https://bugzilla.redhat.com/show_bug.cgi?id=1921094 rh#1921094]). For RHEL-8, this doesn't apply because the systemd there is too old to change the MAC address of most software devices. This could be either implemented by patching `/usr/lib/systemd/network/99-default.link` to have a different policy, or by dropping a link file with higher priority. In the latter case, that override could be shipped either by udev or even by NetworkManager. Another option is to change the scope of this proposal to only change to `MACAddressPolicy=none` for the device types where this causes the most issues (bridge/bond/team). == Feedback == This was also discussed on upstream systemd mailing list [https://lists.freedesktop.org/archives/systemd-devel/2022-May/047893.html here]. The upstream systemd maintainers' opinion is that the current udev behavior is desirable, but accepts that distributions (or network stacks such as NetworkManager) may choose to change the default depending on their needs ([https://lists.freedesktop.org/archives/systemd-devel/2022-May/047943.html reference]). The goal here is to find out what the Fedora community thinks is the
