Re: Xilinx axienet 1000BaseX support
On Tue, Apr 28, 2020 at 05:51:58PM -0600, Robert Hancock wrote: > On 2020-04-28 5:01 p.m., Russell King - ARM Linux admin wrote: > > On Tue, Apr 28, 2020 at 03:59:45PM -0600, Robert Hancock wrote: > > > On 2020-04-22 1:51 a.m., Russell King - ARM Linux admin wrote: > > > > On Tue, Apr 21, 2020 at 07:45:47PM -0600, Robert Hancock wrote: > > > > > Hi Andre/Russell, > > > > > > > > > > Just wondering where things got to with the changes for SGMII on > > > > > Xilinx > > > > > axienet that you were discussing (below)? I am looking into our > > > > > Xilinx setup > > > > > using 1000BaseX SFP and trying to get it working "properly" with newer > > > > > kernels. My understanding is that the requirements for 1000BaseX and > > > > > SGMII > > > > > are somewhat similar. I gathered that SGMII was working somewhat > > > > > already, > > > > > but that not all link modes had been tested. However, it appears > > > > > 1000BaseX > > > > > is not yet working in the stock kernel. > > > > > > > > > > The way I had this working before with a 4.19-based kernel was > > > > > basically a > > > > > hack to phylink to allow the Xilinx PCS/PMA PHY to be configured > > > > > sufficiently as a PHY for it to work, and mostly ignored the link > > > > > status of > > > > > the SFP PHY itself, even though we were using in-band signalling mode > > > > > with > > > > > an SFP module. That was using this patch: > > > > > > > > > > https://patchwork.ozlabs.org/project/netdev/patch/1559330285-30246-5-git-send-email-hanc...@sedsystems.ca/ > > > > > > > > > > Of course, that's basically just a hack which I suspect mostly worked > > > > > by > > > > > luck. I see that there are some helpers that were added to phylink to > > > > > allow > > > > > setting PHY advertisements and reading PHY status from clause 22 PHY > > > > > devices, so I'm guessing that is the way to go in this case? > > > > > Something like: > > > > > > > > > > axienet_mac_config: if using in-band mode, use > > > > > phylink_mii_c22_pcs_set_advertisement to configure the Xilinx PHY. > > > > > > > > > > axienet_mac_pcs_get_state: use phylink_mii_c22_pcs_get_state to get > > > > > the MAC > > > > > PCS state from the Xilinx PHY > > > > > > > > > > axienet_mac_an_restart: if using in-band mode, use > > > > > phylink_mii_c22_pcs_an_restart to restart autonegotiation on Xilinx > > > > > PHY > > > > > > > > > > To use those c22 functions, we need to find the mdio_device that's > > > > > referenced by the phy-handle in the device tree - I guess we can just > > > > > use > > > > > some of the guts of of_phy_find_device to do that? > > > > > > > > Please see the code for DPAA2 - it's changed slightly since I sent a > > > > copy to the netdev mailing list, and it still isn't clear whether this > > > > is the final approach (DPAA2 has some fun stuff such as several > > > > different PHYs at address 0.) NXP basically didn't like the approach > > > > I had in the patches I sent to netdev, we had a call, they presented > > > > an alternative appraoch, I implemented it, then they decided my > > > > original approach was the better solution for their situation. > > > > > > > > See http://git.armlinux.org.uk/cgit/linux-arm.git/log/?h=cex7 > > > > > > > > specifically the patches from: > > > > > > > > "dpaa2-mac: add 1000BASE-X/SGMII PCS support" > > > > > > > > through to: > > > > > > > > "net: phylink: add interface to configure clause 22 PCS PHY" > > > > > > > > You may also need some of the patches further down in the net-queue > > > > branch: > > > > > > > > "net: phylink: avoid mac_config calls" > > > > > > > > through to: > > > > > > > > "net: phylink: rejig link state tracking" > > > > > > I've been playing with this a bit on a 5.4 kernel with some of these > > > patches > > > backported. However, I'm running into something that my previous hacks for > > > this basically dealt with as a side effect: when phylink_start is called, > > > sfp_upstream_start gets called, an SFP module is detected, > > > phylink_connect_phy gets called, but then it hits this condition and bails > > > out, because we are using INBAND mode with 1000BaseX: > > > > > > if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED || > > > (pl->cfg_link_an_mode == MLO_AN_INBAND && > > >phy_interface_mode_is_8023z(interface > > > return -EINVAL; > > > > I'm expecting SGMII mode to be used when there's an external PHY as > > that gives greatest flexibility (as it allows 10 and 100Mbps speeds > > as well.) From what I remember, these blocks support SGMII, so it > > should just be a matter of adding that. > > They do support SGMII, but unfortunately it's not a runtime configurable > parameter, it's a synthesis-level parameter on the FPGA IP core so you have > to pick one or the other for any given build. We want to be able to support > various fiber module types as well, and my understanding is that at least > some of those only
Re: Xilinx axienet 1000BaseX support
On 2020-04-28 5:01 p.m., Russell King - ARM Linux admin wrote: On Tue, Apr 28, 2020 at 03:59:45PM -0600, Robert Hancock wrote: On 2020-04-22 1:51 a.m., Russell King - ARM Linux admin wrote: On Tue, Apr 21, 2020 at 07:45:47PM -0600, Robert Hancock wrote: Hi Andre/Russell, Just wondering where things got to with the changes for SGMII on Xilinx axienet that you were discussing (below)? I am looking into our Xilinx setup using 1000BaseX SFP and trying to get it working "properly" with newer kernels. My understanding is that the requirements for 1000BaseX and SGMII are somewhat similar. I gathered that SGMII was working somewhat already, but that not all link modes had been tested. However, it appears 1000BaseX is not yet working in the stock kernel. The way I had this working before with a 4.19-based kernel was basically a hack to phylink to allow the Xilinx PCS/PMA PHY to be configured sufficiently as a PHY for it to work, and mostly ignored the link status of the SFP PHY itself, even though we were using in-band signalling mode with an SFP module. That was using this patch: https://patchwork.ozlabs.org/project/netdev/patch/1559330285-30246-5-git-send-email-hanc...@sedsystems.ca/ Of course, that's basically just a hack which I suspect mostly worked by luck. I see that there are some helpers that were added to phylink to allow setting PHY advertisements and reading PHY status from clause 22 PHY devices, so I'm guessing that is the way to go in this case? Something like: axienet_mac_config: if using in-band mode, use phylink_mii_c22_pcs_set_advertisement to configure the Xilinx PHY. axienet_mac_pcs_get_state: use phylink_mii_c22_pcs_get_state to get the MAC PCS state from the Xilinx PHY axienet_mac_an_restart: if using in-band mode, use phylink_mii_c22_pcs_an_restart to restart autonegotiation on Xilinx PHY To use those c22 functions, we need to find the mdio_device that's referenced by the phy-handle in the device tree - I guess we can just use some of the guts of of_phy_find_device to do that? Please see the code for DPAA2 - it's changed slightly since I sent a copy to the netdev mailing list, and it still isn't clear whether this is the final approach (DPAA2 has some fun stuff such as several different PHYs at address 0.) NXP basically didn't like the approach I had in the patches I sent to netdev, we had a call, they presented an alternative appraoch, I implemented it, then they decided my original approach was the better solution for their situation. See http://git.armlinux.org.uk/cgit/linux-arm.git/log/?h=cex7 specifically the patches from: "dpaa2-mac: add 1000BASE-X/SGMII PCS support" through to: "net: phylink: add interface to configure clause 22 PCS PHY" You may also need some of the patches further down in the net-queue branch: "net: phylink: avoid mac_config calls" through to: "net: phylink: rejig link state tracking" I've been playing with this a bit on a 5.4 kernel with some of these patches backported. However, I'm running into something that my previous hacks for this basically dealt with as a side effect: when phylink_start is called, sfp_upstream_start gets called, an SFP module is detected, phylink_connect_phy gets called, but then it hits this condition and bails out, because we are using INBAND mode with 1000BaseX: if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED || (pl->cfg_link_an_mode == MLO_AN_INBAND && phy_interface_mode_is_8023z(interface return -EINVAL; I'm expecting SGMII mode to be used when there's an external PHY as that gives greatest flexibility (as it allows 10 and 100Mbps speeds as well.) From what I remember, these blocks support SGMII, so it should just be a matter of adding that. They do support SGMII, but unfortunately it's not a runtime configurable parameter, it's a synthesis-level parameter on the FPGA IP core so you have to pick one or the other for any given build. We want to be able to support various fiber module types as well, and my understanding is that at least some of those only do 1000BaseX, so that ends up being the standard in common that we are using. I guess I'm not sure how this is supposed to work when the PHY on the SFP module gets detected, i.e. if there's supposed to be another code path that this is supposed to go down, or this is something that just hasn't been fully implemented yet? Copper PHYs work fine - using SGMII mode everywhere so far. The problem is, if you want to use them as 1000BASE-X, you generally have to ensure that the PHY is appropriately programmed for 1000BASE-X negotiation, and the copper side advertisement only indicates 1G support. Not all copper PHYs have the PHY accessible for such programming, and in that case it becomes an exercise of "read the SFP documentation before buying"! The other complication is... there's nothing in the module EEPROM that really says whether they are 1000BASE-X or
Re: Xilinx axienet 1000BaseX support
On Tue, Apr 28, 2020 at 03:59:45PM -0600, Robert Hancock wrote: > On 2020-04-22 1:51 a.m., Russell King - ARM Linux admin wrote: > > On Tue, Apr 21, 2020 at 07:45:47PM -0600, Robert Hancock wrote: > > > Hi Andre/Russell, > > > > > > Just wondering where things got to with the changes for SGMII on Xilinx > > > axienet that you were discussing (below)? I am looking into our Xilinx > > > setup > > > using 1000BaseX SFP and trying to get it working "properly" with newer > > > kernels. My understanding is that the requirements for 1000BaseX and SGMII > > > are somewhat similar. I gathered that SGMII was working somewhat already, > > > but that not all link modes had been tested. However, it appears 1000BaseX > > > is not yet working in the stock kernel. > > > > > > The way I had this working before with a 4.19-based kernel was basically a > > > hack to phylink to allow the Xilinx PCS/PMA PHY to be configured > > > sufficiently as a PHY for it to work, and mostly ignored the link status > > > of > > > the SFP PHY itself, even though we were using in-band signalling mode with > > > an SFP module. That was using this patch: > > > > > > https://patchwork.ozlabs.org/project/netdev/patch/1559330285-30246-5-git-send-email-hanc...@sedsystems.ca/ > > > > > > Of course, that's basically just a hack which I suspect mostly worked by > > > luck. I see that there are some helpers that were added to phylink to > > > allow > > > setting PHY advertisements and reading PHY status from clause 22 PHY > > > devices, so I'm guessing that is the way to go in this case? Something > > > like: > > > > > > axienet_mac_config: if using in-band mode, use > > > phylink_mii_c22_pcs_set_advertisement to configure the Xilinx PHY. > > > > > > axienet_mac_pcs_get_state: use phylink_mii_c22_pcs_get_state to get the > > > MAC > > > PCS state from the Xilinx PHY > > > > > > axienet_mac_an_restart: if using in-band mode, use > > > phylink_mii_c22_pcs_an_restart to restart autonegotiation on Xilinx PHY > > > > > > To use those c22 functions, we need to find the mdio_device that's > > > referenced by the phy-handle in the device tree - I guess we can just use > > > some of the guts of of_phy_find_device to do that? > > > > Please see the code for DPAA2 - it's changed slightly since I sent a > > copy to the netdev mailing list, and it still isn't clear whether this > > is the final approach (DPAA2 has some fun stuff such as several > > different PHYs at address 0.) NXP basically didn't like the approach > > I had in the patches I sent to netdev, we had a call, they presented > > an alternative appraoch, I implemented it, then they decided my > > original approach was the better solution for their situation. > > > > See http://git.armlinux.org.uk/cgit/linux-arm.git/log/?h=cex7 > > > > specifically the patches from: > > > >"dpaa2-mac: add 1000BASE-X/SGMII PCS support" > > > > through to: > > > >"net: phylink: add interface to configure clause 22 PCS PHY" > > > > You may also need some of the patches further down in the net-queue > > branch: > > > >"net: phylink: avoid mac_config calls" > > > > through to: > > > >"net: phylink: rejig link state tracking" > > I've been playing with this a bit on a 5.4 kernel with some of these patches > backported. However, I'm running into something that my previous hacks for > this basically dealt with as a side effect: when phylink_start is called, > sfp_upstream_start gets called, an SFP module is detected, > phylink_connect_phy gets called, but then it hits this condition and bails > out, because we are using INBAND mode with 1000BaseX: > > if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED || > (pl->cfg_link_an_mode == MLO_AN_INBAND && >phy_interface_mode_is_8023z(interface > return -EINVAL; I'm expecting SGMII mode to be used when there's an external PHY as that gives greatest flexibility (as it allows 10 and 100Mbps speeds as well.) From what I remember, these blocks support SGMII, so it should just be a matter of adding that. > I guess I'm not sure how this is supposed to work when the PHY on the SFP > module gets detected, i.e. if there's supposed to be another code path that > this is supposed to go down, or this is something that just hasn't been > fully implemented yet? Copper PHYs work fine - using SGMII mode everywhere so far. The problem is, if you want to use them as 1000BASE-X, you generally have to ensure that the PHY is appropriately programmed for 1000BASE-X negotiation, and the copper side advertisement only indicates 1G support. Not all copper PHYs have the PHY accessible for such programming, and in that case it becomes an exercise of "read the SFP documentation before buying"! The other complication is... there's nothing in the module EEPROM that really says whether they are 1000BASE-X or SGMII. What saves us thus far is that most copper SFPs use the Marvell 88E chip, which is I2C
Re: Xilinx axienet 1000BaseX support
On 2020-04-22 1:51 a.m., Russell King - ARM Linux admin wrote: On Tue, Apr 21, 2020 at 07:45:47PM -0600, Robert Hancock wrote: Hi Andre/Russell, Just wondering where things got to with the changes for SGMII on Xilinx axienet that you were discussing (below)? I am looking into our Xilinx setup using 1000BaseX SFP and trying to get it working "properly" with newer kernels. My understanding is that the requirements for 1000BaseX and SGMII are somewhat similar. I gathered that SGMII was working somewhat already, but that not all link modes had been tested. However, it appears 1000BaseX is not yet working in the stock kernel. The way I had this working before with a 4.19-based kernel was basically a hack to phylink to allow the Xilinx PCS/PMA PHY to be configured sufficiently as a PHY for it to work, and mostly ignored the link status of the SFP PHY itself, even though we were using in-band signalling mode with an SFP module. That was using this patch: https://patchwork.ozlabs.org/project/netdev/patch/1559330285-30246-5-git-send-email-hanc...@sedsystems.ca/ Of course, that's basically just a hack which I suspect mostly worked by luck. I see that there are some helpers that were added to phylink to allow setting PHY advertisements and reading PHY status from clause 22 PHY devices, so I'm guessing that is the way to go in this case? Something like: axienet_mac_config: if using in-band mode, use phylink_mii_c22_pcs_set_advertisement to configure the Xilinx PHY. axienet_mac_pcs_get_state: use phylink_mii_c22_pcs_get_state to get the MAC PCS state from the Xilinx PHY axienet_mac_an_restart: if using in-band mode, use phylink_mii_c22_pcs_an_restart to restart autonegotiation on Xilinx PHY To use those c22 functions, we need to find the mdio_device that's referenced by the phy-handle in the device tree - I guess we can just use some of the guts of of_phy_find_device to do that? Please see the code for DPAA2 - it's changed slightly since I sent a copy to the netdev mailing list, and it still isn't clear whether this is the final approach (DPAA2 has some fun stuff such as several different PHYs at address 0.) NXP basically didn't like the approach I had in the patches I sent to netdev, we had a call, they presented an alternative appraoch, I implemented it, then they decided my original approach was the better solution for their situation. See http://git.armlinux.org.uk/cgit/linux-arm.git/log/?h=cex7 specifically the patches from: "dpaa2-mac: add 1000BASE-X/SGMII PCS support" through to: "net: phylink: add interface to configure clause 22 PCS PHY" You may also need some of the patches further down in the net-queue branch: "net: phylink: avoid mac_config calls" through to: "net: phylink: rejig link state tracking" I've been playing with this a bit on a 5.4 kernel with some of these patches backported. However, I'm running into something that my previous hacks for this basically dealt with as a side effect: when phylink_start is called, sfp_upstream_start gets called, an SFP module is detected, phylink_connect_phy gets called, but then it hits this condition and bails out, because we are using INBAND mode with 1000BaseX: if (WARN_ON(pl->cfg_link_an_mode == MLO_AN_FIXED || (pl->cfg_link_an_mode == MLO_AN_INBAND && phy_interface_mode_is_8023z(interface return -EINVAL; That same code is still in the latest version in the arm-linux cex7 branch, except now in phylink_attach_phy, and from what I can see would behave similarly. I guess I'm not sure how this is supposed to work when the PHY on the SFP module gets detected, i.e. if there's supposed to be another code path that this is supposed to go down, or this is something that just hasn't been fully implemented yet? -- Robert Hancock Senior Hardware Designer SED Systems, a division of Calian Ltd. Email: hanc...@sedsystems.ca