On 11/18/21 23:25, Brian Hutchinson wrote:
> Hi Alvin,
>
>
> On Thu, Nov 18, 2021 at 4:20 PM Alvin Šipraga <a...@bang-olufsen.dk
> <mailto:a...@bang-olufsen.dk>> wrote:
>
> Hi Brian,
>
> On 11/18/21 01:20, Brian Hutchinson wrote:
> > Yet another update, I was able to get it working .. but feel
> like it is
> > a hack so comments welcome ... see below:
> >
>
> <snip>
>
> >
> > I tried and tried to get eth0 to come up before the bond was brought
> > up. I had everything named in lexical order but didn't appear to
> > matter. I added a eth0.network file and in it specified
> > |ActivationPolicy=|always-up and other things but could not get
> eth0 to
> > come up.
>
> What kernel version are you using? Since Linux 5.11 we have the
>
>
> I'm using linux-fslc-imx 5.10.69
>
> following two changes ([1] and [2]) which should automatically bring
> up/down the master (eth0) whenever user ports (lan1, lan2) are brought
> up/down. Please confirm whether or not you are using 5.11 or later.
>
>
> I don't think that will work either. eth0 has to be up and stay up or
> DSA driver won't work at all. eth0 has to be up or the slaves can't be
> added to the bond.
This use-case is also addressed in the newer kernels (>=5.11), see the
below commit. It is not only when a user port is brought up (as I
summarized it), but rather when it is opened.
commit 9d5ef190e5615a7b63af89f88c4106a5bc127974
Author: Vladimir Oltean <vladimir.olt...@nxp.com>
Date: Fri Feb 5 15:37:10 2021 +0200
net: dsa: automatically bring up DSA master when opening user port
DSA wants the master interface to be open before the user port is
due to
historical reasons. The promiscuity of interfaces that are down used to
have issues, as referenced Lennert Buytenhek in commit df02c6ff2e39
("dsa: fix master interface allmulti/promisc handling").
The bugfix mentioned there, commit b6c40d68ff64 ("net: only invoke
dev->change_rx_flags when device is UP"), was basically a "don't do
that" approach to working around the promiscuity while down issue.
Further work done by Vlad Yasevich in commit d2615bf45069 ("net: core:
Always propagate flag changes to interfaces") has resolved the
underlying issue, and it is strictly up to the DSA and 8021q drivers
now, it is no longer mandated by the networking core that the master
interface must be up when changing its promiscuity.
From DSA's point of view, deciding to error out in dsa_slave_open
because the master isn't up is
(a) a bad user experience and
(b) knocking at an open door.
Even if there still was an issue with promiscuity while down, DSA could
still just open the master and avoid it.
Doing it this way has the additional benefit that user space can now
remove DSA-specific workarounds, like systemd-networkd with
BindCarrier:
https://github.com/systemd/systemd/issues/7478
And we can finally remove one of the 2 bullets in the "Common pitfalls
using DSA setups" chapter.
Tested with two cascaded DSA switches:
$ ip link set sw0p2 up
fsl_enetc 0000:00:00.2 eno2: configuring for fixed/internal link mode
fsl_enetc 0000:00:00.2 eno2: Link is Up - 1Gbps/Full - flow control
rx/tx
mscc_felix 0000:00:00.5 swp0: configuring for fixed/sgmii link mode
mscc_felix 0000:00:00.5 swp0: Link is Up - 1Gbps/Full - flow
control off
8021q: adding VLAN 0 to HW filter on device swp0
sja1105 spi2.0 sw0p2: configuring for phy/rgmii-id link mode
IPv6: ADDRCONF(NETDEV_CHANGE): eno2: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): swp0: link becomes ready
Signed-off-by: Vladimir Oltean <vladimir.olt...@nxp.com>
Reviewed-by: Andrew Lunn <and...@lunn.ch>
Reviewed-by: Florian Fainelli <f.faine...@gmail.com>
Signed-off-by: Jakub Kicinski <k...@kernel.org>
>
> If you look back up the thread where I'm doing the commands manually
> (I'll re-copy below) ... the first thing is to bring eth0 up. If that
> doesn't happen, nothing else works.
I forgot about the bond in my reply. Does it work if you do
BindCarrier=eth0 from the bond's .network file? You might be out of luck
depending on how networkd implements BindCarrier=, but hopefully
something like that works. Remember also that you need a .network file
covering eth0, even though it's just a conduit and you don't want to
give it an IP. You can do that by omitting the [Network] section and
just having a [Match] on eth0.
>
> These are the manual steps that work. This is what I'm trying to
> automate the "systemd way". I read that "bond0" is somehow reserved so
> my systemd.netdev and .network files use bond1.
>
> #!/bin/bash
>
> # Create a redundant bond between ksz9567 DSA lan1 and lan2 interfaces
>
> # Load bonding kernel module
> modprobe bonding
>
> # Bring up CPU interface (cpu to switch port 7 - the RGMII link)
> ip link set eth0 up
>
> # Create a bond
> echo +bond0 > /sys/class/net/bonding_masters
>
> # Set mode to active-backup (redundancy failover)
> echo active-backup > /sys/class/net/bond0/bonding/mode
>
> # Set time it takes (in ms) for slave to move when a link goes down
> echo 1000 > /sys/class/net/bond0/bonding/miimon
>
> # Add slaves to bond
>
> echo +lan1 > /sys/class/net/bond0/bonding/slaves
> echo +lan2 > /sys/class/net/bond0/bonding/slaves
>
> # Set IP and netmask of the bond
> ip addr add 192.168.0.4/24
> <http://192.168.0.4/24
> dev bond0
>
> # And bring bond up. Pings and network connectivity should work now
> ip link set bond0 up
>
> # For a board that doesn't have Ethernet switch hardware strapped to
> enable at boot .. enable it now
> i2cset -f -y 0 0x5f 0x03 0x00 0x01 i
>
>
> If you are using an older kernel, and systemd is version 243 or later,
>
>
> I checked and I'm using 244.5
>
> you can use the BindCarrier= setting (see [1]) in the [Network] section
> of your .network file(s) for lan1 and lan2. Something like this:
>
> [Match]
> Name=lan*
>
> [Network]
> BindCarrier=eth0
>
>
> I added the BindCarrier=eth0 in both of my DSA lan1 and lan2 .network
> files. I got the same result as before ... dmesg reports lan1 and lan2
> slaves are brought up before eth0 ... and so they fail. The only thing
> so far that works is to bring eth0 up with a network-pre.target
> service. And I'm doing the i2c command to enable my switch in
> a|After=network-online.target service.
> |
>
> [ 4.367802] bond1: (slave lan2): Opening slave failed
> [ 4.467561] bond1: (slave lan1): Opening slave failed
> [ 4.640914] fec 30be0000.ethernet eth0: Link is Up - 1Gbps/Full -
> flow control off
> [ 5.104983] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
>
> Regards,
>
> Brian
