Bug#981186: linux: Enable CMN-600 interconnect on arm64
Hi Vincent, Wookey, On Wed, Mar 17, 2021 at 10:02:25PM +0100, Vincent Blut wrote: > Le 2021-03-17 19:24, Wookey a écrit : > > On 2021-03-17 19:43 +0100, Vincent Blut wrote: > > > Le 2021-03-17 15:49, Wookey a écrit : > > > > On 2021-03-17 14:52 +0100, Vincent Blut wrote: > > > > > Le 2021-01-27 12:57, Wookey a écrit : > > > > > > Version: Please enable ARM CMN-600 power management on arm64 > > > > > > > > > > > > This requires CONFIG_ARM_CMN=y > > > > > > > > > > Does it really have to be built-in instead of being provided as a > > > > > module? Last I > > > > > checked, Fedora and Ubuntu provide it as a module. > > > > > > > > No it should really be a module. Perf is driven from userspace so you > > > > never need to use it before modules can be loaded. > > > > > > Agreed. > > > > > > I see that > > > > CONFIG_THUNDERX2_PMU=y > > > > CONFIG_ARM_SMMU_V3_PMU=y > > > > are also set as builtins. That's probably wrong too. > > > > > > It seems your arm64 kernel config deviates from the one we provide in > > > Debian. > > > CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU > > > is > > > not set, at least in linux 5.10.19-1. > > > > Hmm. I was looking at the (built, with CONFIG_ARM_CMN=y) sources for > > 5.10.9-1 and the (unbuilt) sources for 5.10.19-1. So yes, slightly > > different and the built version is not up to date any more. > > > > If we already have CONFIG_THUNDERX2_PMU=m already then that's great > > (Ah yes - that's the upstream default). Adding > > CONFIG_ARM_SMMU_V3_PMU=m would be good too. Adding it as a module > > should be pretty harmless then at least it's available? I'll set off a > > build now to check it works. > > Enabling ARM_SMMU_V3_PMU as a module should be harmless, indeed. > > > > > […] > > > > > > > I also checked the state of the other perf configs with the arm kernel > > > > team > > > > and got feedback that we have all the ones that should sensibly be set > > > > set once > > > > CONFIG_ARM_CMN=m > > > > and > > > > CONFIG_THUNDERX2_PMU=m > > > > is added > > > > > > This means updating the arm64 kernel config to only include ARM_CMN as a > > > module. > > > To me it is acceptable for Bullseye as this seems uncontroversial, but > > > note that > > > I can't speak for the kernel team. > > > > Will you ask them, or should I? > > I can send merge requests to enable ARM_CMN and ARM_SMMU_V3_PMU if you wish. > > > It seems like prodding someone would be good as this was filed back on 27th > > jan and there have been uploads since, so I guess no-one has noticed till > > now. > > I have been contributing for some time to help the kernel team, but I must > admit > I didn't notice this one (and probably many others). So just to confirm, were you be able to test with those two changes? Regards, Salvatore
Bug#981186: linux: Enable CMN-600 interconnect on arm64
Le 2021-03-17 19:24, Wookey a écrit : > On 2021-03-17 19:43 +0100, Vincent Blut wrote: > > Le 2021-03-17 15:49, Wookey a écrit : > > > On 2021-03-17 14:52 +0100, Vincent Blut wrote: > > > > Le 2021-01-27 12:57, Wookey a écrit : > > > > > Version: Please enable ARM CMN-600 power management on arm64 > > > > > > > > > > This requires CONFIG_ARM_CMN=y > > > > > > > > Does it really have to be built-in instead of being provided as a > > > > module? Last I > > > > checked, Fedora and Ubuntu provide it as a module. > > > > > > No it should really be a module. Perf is driven from userspace so you > > > never need to use it before modules can be loaded. > > > > Agreed. > > > > I see that > > > CONFIG_THUNDERX2_PMU=y > > > CONFIG_ARM_SMMU_V3_PMU=y > > > are also set as builtins. That's probably wrong too. > > > > It seems your arm64 kernel config deviates from the one we provide in > > Debian. > > CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU is > > not set, at least in linux 5.10.19-1. > > Hmm. I was looking at the (built, with CONFIG_ARM_CMN=y) sources for > 5.10.9-1 and the (unbuilt) sources for 5.10.19-1. So yes, slightly > different and the built version is not up to date any more. > > If we already have CONFIG_THUNDERX2_PMU=m already then that's great > (Ah yes - that's the upstream default). Adding > CONFIG_ARM_SMMU_V3_PMU=m would be good too. Adding it as a module > should be pretty harmless then at least it's available? I'll set off a > build now to check it works. Enabling ARM_SMMU_V3_PMU as a module should be harmless, indeed. > > > […] > > > > > I also checked the state of the other perf configs with the arm kernel > > > team > > > and got feedback that we have all the ones that should sensibly be set > > > set once > > > CONFIG_ARM_CMN=m > > > and > > > CONFIG_THUNDERX2_PMU=m > > > is added > > > > This means updating the arm64 kernel config to only include ARM_CMN as a > > module. > > To me it is acceptable for Bullseye as this seems uncontroversial, but note > > that > > I can't speak for the kernel team. > > Will you ask them, or should I? I can send merge requests to enable ARM_CMN and ARM_SMMU_V3_PMU if you wish. > It seems like prodding someone would be good as this was filed back on 27th > jan and there have been uploads since, so I guess no-one has noticed till now. I have been contributing for some time to help the kernel team, but I must admit I didn't notice this one (and probably many others). > > > Upstream enables > > > CONFIG_FSL_IMX8_DDR_PMU=m > > > by default too. IMX8 hardware is available so we should probably turn > > > this on too > > > Contrary to Ubuntu, we do not provide support for the i.MX8M SoC family, > > so enabling this option in the arm64 kernel config is not an option, right? > > Ah OK. I didn't realise IMX8 was not enabled in the debian kernel (A > subject for a different bug). In that case, no this is not > appropriate. I wanted to work on this a few months ago, but sadly I was unable to obtain a i.MX8 SBC. > Wookey > -- > Principal hats: Linaro, Debian, Wookware, ARM > http://wookware.org/ Cheers, Vincent signature.asc Description: PGP signature
Bug#981186: linux: Enable CMN-600 interconnect on arm64
On 2021-03-17 19:43 +0100, Vincent Blut wrote: > Le 2021-03-17 15:49, Wookey a écrit : > > On 2021-03-17 14:52 +0100, Vincent Blut wrote: > > > Le 2021-01-27 12:57, Wookey a écrit : > > > > Version: Please enable ARM CMN-600 power management on arm64 > > > > > > > > This requires CONFIG_ARM_CMN=y > > > > > > Does it really have to be built-in instead of being provided as a module? > > > Last I > > > checked, Fedora and Ubuntu provide it as a module. > > > > No it should really be a module. Perf is driven from userspace so you > > never need to use it before modules can be loaded. > > Agreed. > > I see that > > CONFIG_THUNDERX2_PMU=y > > CONFIG_ARM_SMMU_V3_PMU=y > > are also set as builtins. That's probably wrong too. > > It seems your arm64 kernel config deviates from the one we provide in Debian. > CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU is > not set, at least in linux 5.10.19-1. Hmm. I was looking at the (built, with CONFIG_ARM_CMN=y) sources for 5.10.9-1 and the (unbuilt) sources for 5.10.19-1. So yes, slightly different and the built version is not up to date any more. If we already have CONFIG_THUNDERX2_PMU=m already then that's great (Ah yes - that's the upstream default). Adding CONFIG_ARM_SMMU_V3_PMU=m would be good too. Adding it as a module should be pretty harmless then at least it's available? I'll set off a build now to check it works. > > […] > > > I also checked the state of the other perf configs with the arm kernel team > > and got feedback that we have all the ones that should sensibly be set set > > once > > CONFIG_ARM_CMN=m > > and > > CONFIG_THUNDERX2_PMU=m > > is added > > This means updating the arm64 kernel config to only include ARM_CMN as a > module. > To me it is acceptable for Bullseye as this seems uncontroversial, but note > that > I can't speak for the kernel team. Will you ask them, or should I? It seems like prodding someone would be good as this was filed back on 27th jan and there have been uploads since, so I guess no-one has noticed till now. > > Upstream enables > > CONFIG_FSL_IMX8_DDR_PMU=m > > by default too. IMX8 hardware is available so we should probably turn this > > on too > > Contrary to Ubuntu, we do not provide support for the i.MX8M SoC family, > so enabling this option in the arm64 kernel config is not an option, right? Ah OK. I didn't realise IMX8 was not enabled in the debian kernel (A subject for a different bug). In that case, no this is not appropriate. Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#981186: linux: Enable CMN-600 interconnect on arm64
Le 2021-03-17 15:49, Wookey a écrit : > On 2021-03-17 14:52 +0100, Vincent Blut wrote: > > Control: tags -1 moreinfo > > > > Hi Wookey, > > > > Le 2021-01-27 12:57, Wookey a écrit : > > > Source: linux > > > Version: Please enable ARM CMN-600 power management on arm64 > > > Severity: normal > > > Tags: patch > > > > > > Current arm hardware such as graviton2 (AWS arm64 hardware) has > > > 'Coherent Mesh Network' interconnect (between components in a > > > soc). It's important that support for this is built in the kernel so > > > it can be used. > > > > > > This requires CONFIG_ARM_CMN=y > > > > Does it really have to be built-in instead of being provided as a module? > > Last I > > checked, Fedora and Ubuntu provide it as a module. > > No it should really be a module. Perf is driven from userspace so you > never need to use it before modules can be loaded. Agreed. > I just did it like this because the other settings here are set as > built-ins too and this seemed less disruptive. (If we make it a module > we'll need to make sure it gets included in the right module package - > I'm not sure if that need tweaking somewhere else in the build system) > > I see that > CONFIG_THUNDERX2_PMU=y > CONFIG_ARM_SMMU_V3_PMU=y > are also set as builtins. That's probably wrong too. It seems your arm64 kernel config deviates from the one we provide in Debian. CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU is not set, at least in linux 5.10.19-1. > […] > I also checked the state of the other perf configs with the arm kernel team > and got feedback that we have all the ones that should sensibly be set set > once > CONFIG_ARM_CMN=m > and > CONFIG_THUNDERX2_PMU=m > is added This means updating the arm64 kernel config to only include ARM_CMN as a module. To me it is acceptable for Bullseye as this seems uncontroversial, but note that I can't speak for the kernel team. > Upstream enables > CONFIG_FSL_IMX8_DDR_PMU=m > by default too. IMX8 hardware is available so we should probably turn this on > too Contrary to Ubuntu, we do not provide support for the i.MX8M SoC family, so enabling this option in the arm64 kernel config is not an option, right? signature.asc Description: PGP signature
Bug#981186: linux: Enable CMN-600 interconnect on arm64
On 2021-03-17 14:52 +0100, Vincent Blut wrote: > Control: tags -1 moreinfo > > Hi Wookey, > > Le 2021-01-27 12:57, Wookey a écrit : > > Source: linux > > Version: Please enable ARM CMN-600 power management on arm64 > > Severity: normal > > Tags: patch > > > > Current arm hardware such as graviton2 (AWS arm64 hardware) has > > 'Coherent Mesh Network' interconnect (between components in a > > soc). It's important that support for this is built in the kernel so > > it can be used. > > > > This requires CONFIG_ARM_CMN=y > > Does it really have to be built-in instead of being provided as a module? > Last I > checked, Fedora and Ubuntu provide it as a module. No it should really be a module. Perf is driven from userspace so you never need to use it before modules can be loaded. I just did it like this because the other settings here are set as built-ins too and this seemed less disruptive. (If we make it a module we'll need to make sure it gets included in the right module package - I'm not sure if that need tweaking somewhere else in the build system) I see that CONFIG_THUNDERX2_PMU=y CONFIG_ARM_SMMU_V3_PMU=y are also set as builtins. That's probably wrong too. This should be a module (whic is the upstream default - I'm not sure why it's coming out as a built-in in the debian build): CONFIG_THUNDERX2_PMU=m I'm not sure about CONFIG_ARM_SMMU_V3_PMU=y as it's an architectureal feature. Best to leave it as a built-in for now. I also checked the state of the other perf configs with the arm kernel team and got feedback that we have all the ones that should sensibly be set set once CONFIG_ARM_CMN=m and CONFIG_THUNDERX2_PMU=m is added Upstream enables CONFIG_FSL_IMX8_DDR_PMU=m by default too. IMX8 hardware is available so we should probably turn this on too Do you want me to knock up a patch for this or is that enough info? Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/ signature.asc Description: PGP signature
Bug#981186: linux: Enable CMN-600 interconnect on arm64
Control: tags -1 moreinfo Hi Wookey, Le 2021-01-27 12:57, Wookey a écrit : > Source: linux > Version: Please enable ARM CMN-600 power management on arm64 > Severity: normal > Tags: patch > > Current arm hardware such as graviton2 (AWS arm64 hardware) has > 'Coherent Mesh Network' interconnect (between components in a > soc). It's important that support for this is built in the kernel so > it can be used. > > This requires CONFIG_ARM_CMN=y Does it really have to be built-in instead of being provided as a module? Last I checked, Fedora and Ubuntu provide it as a module. > This explains what the feature is: > https://www.arm.com/products/silicon-ip-system/corelink-interconnect/cmn-600 > Graviton2: > https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd > > patch attached: > --- debian/config/arm64/config~ 2021-01-27 04:34:51.359552398 + > +++ debian/config/arm64/config2021-01-27 04:53:11.922998842 + > @@ -942,6 +942,7 @@ > CONFIG_ARM_CCI400_PMU=y > CONFIG_ARM_CCI5xx_PMU=y > CONFIG_ARM_CCN=y > +CONFIG_ARM_CMN=y > CONFIG_QCOM_L2_PMU=y > CONFIG_QCOM_L3_PMU=y > CONFIG_XGENE_PMU=y > --- debian/config/arm64/config~ 2021-01-27 04:34:51.359552398 + > +++ debian/config/arm64/config2021-01-27 04:53:11.922998842 + > @@ -942,6 +942,7 @@ > CONFIG_ARM_CCI400_PMU=y > CONFIG_ARM_CCI5xx_PMU=y > CONFIG_ARM_CCN=y > +CONFIG_ARM_CMN=y > CONFIG_QCOM_L2_PMU=y > CONFIG_QCOM_L3_PMU=y > CONFIG_XGENE_PMU=y Cheers, Vincent signature.asc Description: PGP signature
Bug#981186: linux: Enable CMN-600 interconnect on arm64
There is hardware publicly available to buy now that uses the CMN-600 interconnect so we really should turn it on for this stable release if at all possible. The Ampere ALTRA:https://store.avantek.co.uk/arm-servers.html Wookey -- Principal hats: Linaro, Debian, Wookware, ARM http://wookware.org/
Bug#981186: linux: Enable CMN-600 interconnect on arm64
On Wed, Jan 27, 2021 at 12:57:07PM +, Wookey wrote: > Current arm hardware such as graviton2 (AWS arm64 hardware) has > 'Coherent Mesh Network' interconnect (between components in a > soc). It's important that support for this is built in the kernel so > it can be used. > > This requires CONFIG_ARM_CMN=y To be precise, this driver is needed for perf event monitoring of this interconnect. The interconnect itself is always in use. On Amazon EC2, these PMU events are only exposed on the bare-metal instances (e.g. m6g.metal), not the VMs. We should still enable support for this driver, in any case. noah
Bug#981186: linux: Enable CMN-600 interconnect on arm64
Source: linux Version: Please enable ARM CMN-600 power management on arm64 Severity: normal Tags: patch Current arm hardware such as graviton2 (AWS arm64 hardware) has 'Coherent Mesh Network' interconnect (between components in a soc). It's important that support for this is built in the kernel so it can be used. This requires CONFIG_ARM_CMN=y This explains what the feature is: https://www.arm.com/products/silicon-ip-system/corelink-interconnect/cmn-600 Graviton2: https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd patch attached: --- debian/config/arm64/config~ 2021-01-27 04:34:51.359552398 + +++ debian/config/arm64/config 2021-01-27 04:53:11.922998842 + @@ -942,6 +942,7 @@ CONFIG_ARM_CCI400_PMU=y CONFIG_ARM_CCI5xx_PMU=y CONFIG_ARM_CCN=y +CONFIG_ARM_CMN=y CONFIG_QCOM_L2_PMU=y CONFIG_QCOM_L3_PMU=y CONFIG_XGENE_PMU=y --- debian/config/arm64/config~ 2021-01-27 04:34:51.359552398 + +++ debian/config/arm64/config 2021-01-27 04:53:11.922998842 + @@ -942,6 +942,7 @@ CONFIG_ARM_CCI400_PMU=y CONFIG_ARM_CCI5xx_PMU=y CONFIG_ARM_CCN=y +CONFIG_ARM_CMN=y CONFIG_QCOM_L2_PMU=y CONFIG_QCOM_L3_PMU=y CONFIG_XGENE_PMU=y
