Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On 12/04/2023 23:34, Rob Clark wrote: On Wed, Apr 12, 2023 at 1:19 PM Dmitry Baryshkov wrote: On Wed, 12 Apr 2023 at 23:09, Rob Clark wrote: On Wed, Apr 12, 2023 at 5:47 AM Rodrigo Vivi wrote: On Wed, Apr 12, 2023 at 10:11:32AM +0200, Daniel Vetter wrote: On Wed, Apr 12, 2023 at 01:36:52AM +0300, Dmitry Baryshkov wrote: On 11/04/2023 21:28, Rob Clark wrote: On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov wrote: On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: From: Rob Clark Similar motivation to other similar recent attempt[1]. But with an attempt to have some shared code for this. As well as documentation. It is probably a bit UMA-centric, I guess devices with VRAM might want some placement stats as well. But this seems like a reasonable start. Basic gputop support: https://patchwork.freedesktop.org/series/116236/ And already nvtop support: https://github.com/Syllo/nvtop/pull/204 On a related topic, I'm wondering if it would make sense to report some more global things (temp, freq, etc) via fdinfo? Some of this, tools like nvtop could get by trawling sysfs or other driver specific ways. But maybe it makes sense to have these sort of things reported in a standardized way (even though they aren't really per-drm_file) I think that's a bit much layering violation, we'd essentially have to reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to be in :-) I guess this is true for temp (where there are thermal zones with potentially multiple temp sensors.. but I'm still digging my way thru the thermal_cooling_device stuff) It is slightly ugly. All thermal zones and cooling devices are virtual devices (so, even no connection to the particular tsens device). One can either enumerate them by checking /sys/class/thermal/thermal_zoneN/type or enumerate them through /sys/class/hwmon. For cooling devices again the only enumeration is through /sys/class/thermal/cooling_deviceN/type. Probably it should be possible to push cooling devices and thermal zones under corresponding providers. However I do not know if there is a good way to correlate cooling device (ideally a part of GPU) to the thermal_zone (which in our case is provided by tsens / temp_alarm rather than GPU itself). But what about freq? I think, esp for cases where some "fw thing" is controlling the freq we end up needing to use gpu counters to measure the freq. For the freq it is slightly easier: /sys/class/devfreq/*, devices are registered under proper parent (IOW, GPU). So one can read /sys/class/devfreq/3d0.gpu/cur_freq or /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. However because of the components usage, there is no link from /sys/class/drm/card0 (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. Getting all these items together in a platform-independent way would be definitely an important but complex topic. But I don't believe any of the pci gpu's use devfreq ;-) And also, you can't expect the CPU to actually know the freq when fw is the one controlling freq. We can, currently, have a reasonable approximation from devfreq but that stops if IFPC is implemented. And other GPUs have even less direct control. So freq is a thing that I don't think we should try to get from "common frameworks" I think it might be useful to add another passive devfreq governor type for external frequencies. This way we can use the same interface to export non-CPU-controlled frequencies. Yeah this sounds like a decent idea to me too. It might also solve the fun of various pci devices having very non-standard freq controls in sysfs (looking at least at i915 here ...) I also like the idea of having some common infrastructure for the GPU freq. hwmon have a good infrastructure, but they are more focused on individual monitoring devices and not very welcomed to embedded monitoring and control. I still want to check the opportunity to see if at least some freq control could be aligned there. Another thing that complicates that is that there are multiple frequency domains and controls with multipliers in Intel GPU that are not very standard or easy to integrate. On a quick glace this devfreq seems neat because it aligns with the cpufreq and governors. But again it would be hard to align with the multiple domains and controls. But it deserves a look. I will take a look to both fronts for Xe: hwmon and devfreq. Right now on Xe we have a lot less controls than i915, but I can imagine soon there will be requirements to make that to grow and I fear that we end up just like i915. So I will take a look before that happens. So it looks like i915 (dgpu only) and nouveau already use hwmon.. so maybe this is a good way to
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Wed, Apr 12, 2023 at 1:19 PM Dmitry Baryshkov wrote: > > On Wed, 12 Apr 2023 at 23:09, Rob Clark wrote: > > > > On Wed, Apr 12, 2023 at 5:47 AM Rodrigo Vivi wrote: > > > > > > On Wed, Apr 12, 2023 at 10:11:32AM +0200, Daniel Vetter wrote: > > > > On Wed, Apr 12, 2023 at 01:36:52AM +0300, Dmitry Baryshkov wrote: > > > > > On 11/04/2023 21:28, Rob Clark wrote: > > > > > > On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov > > > > > > wrote: > > > > > > > > > > > > > > On Tue, 11 Apr 2023 at 20:13, Rob Clark > > > > > > > wrote: > > > > > > > > > > > > > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > > > > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > From: Rob Clark > > > > > > > > > > > > > > > > > > > > > > Similar motivation to other similar recent attempt[1]. > > > > > > > > > > > But with an > > > > > > > > > > > attempt to have some shared code for this. As well as > > > > > > > > > > > documentation. > > > > > > > > > > > > > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with > > > > > > > > > > > VRAM might want > > > > > > > > > > > some placement stats as well. But this seems like a > > > > > > > > > > > reasonable start. > > > > > > > > > > > > > > > > > > > > > > Basic gputop support: > > > > > > > > > > > https://patchwork.freedesktop.org/series/116236/ > > > > > > > > > > > And already nvtop support: > > > > > > > > > > > https://github.com/Syllo/nvtop/pull/204 > > > > > > > > > > > > > > > > > > > > On a related topic, I'm wondering if it would make sense to > > > > > > > > > > report > > > > > > > > > > some more global things (temp, freq, etc) via fdinfo? Some > > > > > > > > > > of this, > > > > > > > > > > tools like nvtop could get by trawling sysfs or other > > > > > > > > > > driver specific > > > > > > > > > > ways. But maybe it makes sense to have these sort of > > > > > > > > > > things reported > > > > > > > > > > in a standardized way (even though they aren't really > > > > > > > > > > per-drm_file) > > > > > > > > > > > > > > > > > > I think that's a bit much layering violation, we'd > > > > > > > > > essentially have to > > > > > > > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a > > > > > > > > > business I want to > > > > > > > > > be in :-) > > > > > > > > > > > > > > > > I guess this is true for temp (where there are thermal zones > > > > > > > > with > > > > > > > > potentially multiple temp sensors.. but I'm still digging my > > > > > > > > way thru > > > > > > > > the thermal_cooling_device stuff) > > > > > > > > > > > > > > It is slightly ugly. All thermal zones and cooling devices are > > > > > > > virtual > > > > > > > devices (so, even no connection to the particular tsens device). > > > > > > > One > > > > > > > can either enumerate them by checking > > > > > > > /sys/class/thermal/thermal_zoneN/type or enumerate them through > > > > > > > /sys/class/hwmon. For cooling devices again the only enumeration > > > > > > > is > > > > > > > through /sys/class/thermal/cooling_deviceN/type. > > > > > > > > > > > > > > Probably it should be possible to push cooling devices and thermal > > > > > > > zones under corresponding providers. However I do not know if > > > > > > > there is > > > > > > > a good way to correlate cooling device (ideally a part of GPU) to > > > > > > > the > > > > > > > thermal_zone (which in our case is provided by tsens / temp_alarm > > > > > > > rather than GPU itself). > > > > > > > > > > > > > > > > > > > > > > > But what about freq? I think, esp for cases where some "fw > > > > > > > > thing" is > > > > > > > > controlling the freq we end up needing to use gpu counters to > > > > > > > > measure > > > > > > > > the freq. > > > > > > > > > > > > > > For the freq it is slightly easier: /sys/class/devfreq/*, devices > > > > > > > are > > > > > > > registered under proper parent (IOW, GPU). So one can read > > > > > > > /sys/class/devfreq/3d0.gpu/cur_freq or > > > > > > > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > > > > > > > > > > > > > However because of the components usage, there is no link from > > > > > > > /sys/class/drm/card0 > > > > > > > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > > > > > > > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. > > > > > > > > > > > > > > Getting all these items together in a platform-independent way > > > > > > > would > > > > > > > be definitely an important but complex topic. > > > > > > > > > > > > But I don't believe any of the pci gpu's use devfreq ;-) > > > > > > > > > > > > And also, you can't expect the CPU to actually know the freq when fw > > > > > > is the one controlling freq. We can, currently, have a reasonable > > > > > >
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Wed, Apr 12, 2023 at 4:10 PM Rob Clark wrote: > > On Wed, Apr 12, 2023 at 5:47 AM Rodrigo Vivi wrote: > > > > On Wed, Apr 12, 2023 at 10:11:32AM +0200, Daniel Vetter wrote: > > > On Wed, Apr 12, 2023 at 01:36:52AM +0300, Dmitry Baryshkov wrote: > > > > On 11/04/2023 21:28, Rob Clark wrote: > > > > > On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov > > > > > wrote: > > > > > > > > > > > > On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > > > > > > > > > > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter > > > > > > > wrote: > > > > > > > > > > > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > > > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > From: Rob Clark > > > > > > > > > > > > > > > > > > > > Similar motivation to other similar recent attempt[1]. But > > > > > > > > > > with an > > > > > > > > > > attempt to have some shared code for this. As well as > > > > > > > > > > documentation. > > > > > > > > > > > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM > > > > > > > > > > might want > > > > > > > > > > some placement stats as well. But this seems like a > > > > > > > > > > reasonable start. > > > > > > > > > > > > > > > > > > > > Basic gputop support: > > > > > > > > > > https://patchwork.freedesktop.org/series/116236/ > > > > > > > > > > And already nvtop support: > > > > > > > > > > https://github.com/Syllo/nvtop/pull/204 > > > > > > > > > > > > > > > > > > On a related topic, I'm wondering if it would make sense to > > > > > > > > > report > > > > > > > > > some more global things (temp, freq, etc) via fdinfo? Some > > > > > > > > > of this, > > > > > > > > > tools like nvtop could get by trawling sysfs or other driver > > > > > > > > > specific > > > > > > > > > ways. But maybe it makes sense to have these sort of things > > > > > > > > > reported > > > > > > > > > in a standardized way (even though they aren't really > > > > > > > > > per-drm_file) > > > > > > > > > > > > > > > > I think that's a bit much layering violation, we'd essentially > > > > > > > > have to > > > > > > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business > > > > > > > > I want to > > > > > > > > be in :-) > > > > > > > > > > > > > > I guess this is true for temp (where there are thermal zones with > > > > > > > potentially multiple temp sensors.. but I'm still digging my way > > > > > > > thru > > > > > > > the thermal_cooling_device stuff) > > > > > > > > > > > > It is slightly ugly. All thermal zones and cooling devices are > > > > > > virtual > > > > > > devices (so, even no connection to the particular tsens device). One > > > > > > can either enumerate them by checking > > > > > > /sys/class/thermal/thermal_zoneN/type or enumerate them through > > > > > > /sys/class/hwmon. For cooling devices again the only enumeration is > > > > > > through /sys/class/thermal/cooling_deviceN/type. > > > > > > > > > > > > Probably it should be possible to push cooling devices and thermal > > > > > > zones under corresponding providers. However I do not know if there > > > > > > is > > > > > > a good way to correlate cooling device (ideally a part of GPU) to > > > > > > the > > > > > > thermal_zone (which in our case is provided by tsens / temp_alarm > > > > > > rather than GPU itself). > > > > > > > > > > > > > > > > > > > > But what about freq? I think, esp for cases where some "fw > > > > > > > thing" is > > > > > > > controlling the freq we end up needing to use gpu counters to > > > > > > > measure > > > > > > > the freq. > > > > > > > > > > > > For the freq it is slightly easier: /sys/class/devfreq/*, devices > > > > > > are > > > > > > registered under proper parent (IOW, GPU). So one can read > > > > > > /sys/class/devfreq/3d0.gpu/cur_freq or > > > > > > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > > > > > > > > > > > However because of the components usage, there is no link from > > > > > > /sys/class/drm/card0 > > > > > > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > > > > > > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. > > > > > > > > > > > > Getting all these items together in a platform-independent way would > > > > > > be definitely an important but complex topic. > > > > > > > > > > But I don't believe any of the pci gpu's use devfreq ;-) > > > > > > > > > > And also, you can't expect the CPU to actually know the freq when fw > > > > > is the one controlling freq. We can, currently, have a reasonable > > > > > approximation from devfreq but that stops if IFPC is implemented. And > > > > > other GPUs have even less direct control. So freq is a thing that I > > > > > don't think we should try to get from "common frameworks" > > > > > > > > I think it might be useful to add another passive devfreq governor type > > > > for > > > > external
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Wed, 12 Apr 2023 at 23:09, Rob Clark wrote: > > On Wed, Apr 12, 2023 at 5:47 AM Rodrigo Vivi wrote: > > > > On Wed, Apr 12, 2023 at 10:11:32AM +0200, Daniel Vetter wrote: > > > On Wed, Apr 12, 2023 at 01:36:52AM +0300, Dmitry Baryshkov wrote: > > > > On 11/04/2023 21:28, Rob Clark wrote: > > > > > On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov > > > > > wrote: > > > > > > > > > > > > On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > > > > > > > > > > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter > > > > > > > wrote: > > > > > > > > > > > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > > > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > From: Rob Clark > > > > > > > > > > > > > > > > > > > > Similar motivation to other similar recent attempt[1]. But > > > > > > > > > > with an > > > > > > > > > > attempt to have some shared code for this. As well as > > > > > > > > > > documentation. > > > > > > > > > > > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM > > > > > > > > > > might want > > > > > > > > > > some placement stats as well. But this seems like a > > > > > > > > > > reasonable start. > > > > > > > > > > > > > > > > > > > > Basic gputop support: > > > > > > > > > > https://patchwork.freedesktop.org/series/116236/ > > > > > > > > > > And already nvtop support: > > > > > > > > > > https://github.com/Syllo/nvtop/pull/204 > > > > > > > > > > > > > > > > > > On a related topic, I'm wondering if it would make sense to > > > > > > > > > report > > > > > > > > > some more global things (temp, freq, etc) via fdinfo? Some > > > > > > > > > of this, > > > > > > > > > tools like nvtop could get by trawling sysfs or other driver > > > > > > > > > specific > > > > > > > > > ways. But maybe it makes sense to have these sort of things > > > > > > > > > reported > > > > > > > > > in a standardized way (even though they aren't really > > > > > > > > > per-drm_file) > > > > > > > > > > > > > > > > I think that's a bit much layering violation, we'd essentially > > > > > > > > have to > > > > > > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business > > > > > > > > I want to > > > > > > > > be in :-) > > > > > > > > > > > > > > I guess this is true for temp (where there are thermal zones with > > > > > > > potentially multiple temp sensors.. but I'm still digging my way > > > > > > > thru > > > > > > > the thermal_cooling_device stuff) > > > > > > > > > > > > It is slightly ugly. All thermal zones and cooling devices are > > > > > > virtual > > > > > > devices (so, even no connection to the particular tsens device). One > > > > > > can either enumerate them by checking > > > > > > /sys/class/thermal/thermal_zoneN/type or enumerate them through > > > > > > /sys/class/hwmon. For cooling devices again the only enumeration is > > > > > > through /sys/class/thermal/cooling_deviceN/type. > > > > > > > > > > > > Probably it should be possible to push cooling devices and thermal > > > > > > zones under corresponding providers. However I do not know if there > > > > > > is > > > > > > a good way to correlate cooling device (ideally a part of GPU) to > > > > > > the > > > > > > thermal_zone (which in our case is provided by tsens / temp_alarm > > > > > > rather than GPU itself). > > > > > > > > > > > > > > > > > > > > But what about freq? I think, esp for cases where some "fw > > > > > > > thing" is > > > > > > > controlling the freq we end up needing to use gpu counters to > > > > > > > measure > > > > > > > the freq. > > > > > > > > > > > > For the freq it is slightly easier: /sys/class/devfreq/*, devices > > > > > > are > > > > > > registered under proper parent (IOW, GPU). So one can read > > > > > > /sys/class/devfreq/3d0.gpu/cur_freq or > > > > > > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > > > > > > > > > > > However because of the components usage, there is no link from > > > > > > /sys/class/drm/card0 > > > > > > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > > > > > > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. > > > > > > > > > > > > Getting all these items together in a platform-independent way would > > > > > > be definitely an important but complex topic. > > > > > > > > > > But I don't believe any of the pci gpu's use devfreq ;-) > > > > > > > > > > And also, you can't expect the CPU to actually know the freq when fw > > > > > is the one controlling freq. We can, currently, have a reasonable > > > > > approximation from devfreq but that stops if IFPC is implemented. And > > > > > other GPUs have even less direct control. So freq is a thing that I > > > > > don't think we should try to get from "common frameworks" > > > > > > > > I think it might be useful to add another passive devfreq governor type > > > > for > > > > external
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Wed, Apr 12, 2023 at 5:47 AM Rodrigo Vivi wrote: > > On Wed, Apr 12, 2023 at 10:11:32AM +0200, Daniel Vetter wrote: > > On Wed, Apr 12, 2023 at 01:36:52AM +0300, Dmitry Baryshkov wrote: > > > On 11/04/2023 21:28, Rob Clark wrote: > > > > On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov > > > > wrote: > > > > > > > > > > On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > > > > > > > > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter > > > > > > wrote: > > > > > > > > > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > From: Rob Clark > > > > > > > > > > > > > > > > > > Similar motivation to other similar recent attempt[1]. But > > > > > > > > > with an > > > > > > > > > attempt to have some shared code for this. As well as > > > > > > > > > documentation. > > > > > > > > > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM > > > > > > > > > might want > > > > > > > > > some placement stats as well. But this seems like a > > > > > > > > > reasonable start. > > > > > > > > > > > > > > > > > > Basic gputop support: > > > > > > > > > https://patchwork.freedesktop.org/series/116236/ > > > > > > > > > And already nvtop support: > > > > > > > > > https://github.com/Syllo/nvtop/pull/204 > > > > > > > > > > > > > > > > On a related topic, I'm wondering if it would make sense to > > > > > > > > report > > > > > > > > some more global things (temp, freq, etc) via fdinfo? Some of > > > > > > > > this, > > > > > > > > tools like nvtop could get by trawling sysfs or other driver > > > > > > > > specific > > > > > > > > ways. But maybe it makes sense to have these sort of things > > > > > > > > reported > > > > > > > > in a standardized way (even though they aren't really > > > > > > > > per-drm_file) > > > > > > > > > > > > > > I think that's a bit much layering violation, we'd essentially > > > > > > > have to > > > > > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business I > > > > > > > want to > > > > > > > be in :-) > > > > > > > > > > > > I guess this is true for temp (where there are thermal zones with > > > > > > potentially multiple temp sensors.. but I'm still digging my way > > > > > > thru > > > > > > the thermal_cooling_device stuff) > > > > > > > > > > It is slightly ugly. All thermal zones and cooling devices are virtual > > > > > devices (so, even no connection to the particular tsens device). One > > > > > can either enumerate them by checking > > > > > /sys/class/thermal/thermal_zoneN/type or enumerate them through > > > > > /sys/class/hwmon. For cooling devices again the only enumeration is > > > > > through /sys/class/thermal/cooling_deviceN/type. > > > > > > > > > > Probably it should be possible to push cooling devices and thermal > > > > > zones under corresponding providers. However I do not know if there is > > > > > a good way to correlate cooling device (ideally a part of GPU) to the > > > > > thermal_zone (which in our case is provided by tsens / temp_alarm > > > > > rather than GPU itself). > > > > > > > > > > > > > > > > > But what about freq? I think, esp for cases where some "fw thing" > > > > > > is > > > > > > controlling the freq we end up needing to use gpu counters to > > > > > > measure > > > > > > the freq. > > > > > > > > > > For the freq it is slightly easier: /sys/class/devfreq/*, devices are > > > > > registered under proper parent (IOW, GPU). So one can read > > > > > /sys/class/devfreq/3d0.gpu/cur_freq or > > > > > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > > > > > > > > > However because of the components usage, there is no link from > > > > > /sys/class/drm/card0 > > > > > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > > > > > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. > > > > > > > > > > Getting all these items together in a platform-independent way would > > > > > be definitely an important but complex topic. > > > > > > > > But I don't believe any of the pci gpu's use devfreq ;-) > > > > > > > > And also, you can't expect the CPU to actually know the freq when fw > > > > is the one controlling freq. We can, currently, have a reasonable > > > > approximation from devfreq but that stops if IFPC is implemented. And > > > > other GPUs have even less direct control. So freq is a thing that I > > > > don't think we should try to get from "common frameworks" > > > > > > I think it might be useful to add another passive devfreq governor type > > > for > > > external frequencies. This way we can use the same interface to export > > > non-CPU-controlled frequencies. > > > > Yeah this sounds like a decent idea to me too. It might also solve the fun > > of various pci devices having very non-standard freq controls in sysfs > > (looking at least at i915 here ...) > > I also
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Wed, Apr 12, 2023 at 10:11:32AM +0200, Daniel Vetter wrote: > On Wed, Apr 12, 2023 at 01:36:52AM +0300, Dmitry Baryshkov wrote: > > On 11/04/2023 21:28, Rob Clark wrote: > > > On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov > > > wrote: > > > > > > > > On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > > > > > > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: > > > > > > > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark > > > > > > > wrote: > > > > > > > > > > > > > > > > From: Rob Clark > > > > > > > > > > > > > > > > Similar motivation to other similar recent attempt[1]. But > > > > > > > > with an > > > > > > > > attempt to have some shared code for this. As well as > > > > > > > > documentation. > > > > > > > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM > > > > > > > > might want > > > > > > > > some placement stats as well. But this seems like a reasonable > > > > > > > > start. > > > > > > > > > > > > > > > > Basic gputop support: > > > > > > > > https://patchwork.freedesktop.org/series/116236/ > > > > > > > > And already nvtop support: > > > > > > > > https://github.com/Syllo/nvtop/pull/204 > > > > > > > > > > > > > > On a related topic, I'm wondering if it would make sense to report > > > > > > > some more global things (temp, freq, etc) via fdinfo? Some of > > > > > > > this, > > > > > > > tools like nvtop could get by trawling sysfs or other driver > > > > > > > specific > > > > > > > ways. But maybe it makes sense to have these sort of things > > > > > > > reported > > > > > > > in a standardized way (even though they aren't really > > > > > > > per-drm_file) > > > > > > > > > > > > I think that's a bit much layering violation, we'd essentially have > > > > > > to > > > > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business I > > > > > > want to > > > > > > be in :-) > > > > > > > > > > I guess this is true for temp (where there are thermal zones with > > > > > potentially multiple temp sensors.. but I'm still digging my way thru > > > > > the thermal_cooling_device stuff) > > > > > > > > It is slightly ugly. All thermal zones and cooling devices are virtual > > > > devices (so, even no connection to the particular tsens device). One > > > > can either enumerate them by checking > > > > /sys/class/thermal/thermal_zoneN/type or enumerate them through > > > > /sys/class/hwmon. For cooling devices again the only enumeration is > > > > through /sys/class/thermal/cooling_deviceN/type. > > > > > > > > Probably it should be possible to push cooling devices and thermal > > > > zones under corresponding providers. However I do not know if there is > > > > a good way to correlate cooling device (ideally a part of GPU) to the > > > > thermal_zone (which in our case is provided by tsens / temp_alarm > > > > rather than GPU itself). > > > > > > > > > > > > > > But what about freq? I think, esp for cases where some "fw thing" is > > > > > controlling the freq we end up needing to use gpu counters to measure > > > > > the freq. > > > > > > > > For the freq it is slightly easier: /sys/class/devfreq/*, devices are > > > > registered under proper parent (IOW, GPU). So one can read > > > > /sys/class/devfreq/3d0.gpu/cur_freq or > > > > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > > > > > > > However because of the components usage, there is no link from > > > > /sys/class/drm/card0 > > > > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > > > > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. > > > > > > > > Getting all these items together in a platform-independent way would > > > > be definitely an important but complex topic. > > > > > > But I don't believe any of the pci gpu's use devfreq ;-) > > > > > > And also, you can't expect the CPU to actually know the freq when fw > > > is the one controlling freq. We can, currently, have a reasonable > > > approximation from devfreq but that stops if IFPC is implemented. And > > > other GPUs have even less direct control. So freq is a thing that I > > > don't think we should try to get from "common frameworks" > > > > I think it might be useful to add another passive devfreq governor type for > > external frequencies. This way we can use the same interface to export > > non-CPU-controlled frequencies. > > Yeah this sounds like a decent idea to me too. It might also solve the fun > of various pci devices having very non-standard freq controls in sysfs > (looking at least at i915 here ...) I also like the idea of having some common infrastructure for the GPU freq. hwmon have a good infrastructure, but they are more focused on individual monitoring devices and not very welcomed to embedded monitoring and control. I still want to check the opportunity to see if at least some freq control could be
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Wed, Apr 12, 2023 at 01:36:52AM +0300, Dmitry Baryshkov wrote: > On 11/04/2023 21:28, Rob Clark wrote: > > On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov > > wrote: > > > > > > On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > > > > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: > > > > > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark > > > > > > wrote: > > > > > > > > > > > > > > From: Rob Clark > > > > > > > > > > > > > > Similar motivation to other similar recent attempt[1]. But with > > > > > > > an > > > > > > > attempt to have some shared code for this. As well as > > > > > > > documentation. > > > > > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM might > > > > > > > want > > > > > > > some placement stats as well. But this seems like a reasonable > > > > > > > start. > > > > > > > > > > > > > > Basic gputop support: > > > > > > > https://patchwork.freedesktop.org/series/116236/ > > > > > > > And already nvtop support: https://github.com/Syllo/nvtop/pull/204 > > > > > > > > > > > > On a related topic, I'm wondering if it would make sense to report > > > > > > some more global things (temp, freq, etc) via fdinfo? Some of this, > > > > > > tools like nvtop could get by trawling sysfs or other driver > > > > > > specific > > > > > > ways. But maybe it makes sense to have these sort of things > > > > > > reported > > > > > > in a standardized way (even though they aren't really per-drm_file) > > > > > > > > > > I think that's a bit much layering violation, we'd essentially have to > > > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want > > > > > to > > > > > be in :-) > > > > > > > > I guess this is true for temp (where there are thermal zones with > > > > potentially multiple temp sensors.. but I'm still digging my way thru > > > > the thermal_cooling_device stuff) > > > > > > It is slightly ugly. All thermal zones and cooling devices are virtual > > > devices (so, even no connection to the particular tsens device). One > > > can either enumerate them by checking > > > /sys/class/thermal/thermal_zoneN/type or enumerate them through > > > /sys/class/hwmon. For cooling devices again the only enumeration is > > > through /sys/class/thermal/cooling_deviceN/type. > > > > > > Probably it should be possible to push cooling devices and thermal > > > zones under corresponding providers. However I do not know if there is > > > a good way to correlate cooling device (ideally a part of GPU) to the > > > thermal_zone (which in our case is provided by tsens / temp_alarm > > > rather than GPU itself). > > > > > > > > > > > But what about freq? I think, esp for cases where some "fw thing" is > > > > controlling the freq we end up needing to use gpu counters to measure > > > > the freq. > > > > > > For the freq it is slightly easier: /sys/class/devfreq/*, devices are > > > registered under proper parent (IOW, GPU). So one can read > > > /sys/class/devfreq/3d0.gpu/cur_freq or > > > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > > > > > However because of the components usage, there is no link from > > > /sys/class/drm/card0 > > > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > > > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. > > > > > > Getting all these items together in a platform-independent way would > > > be definitely an important but complex topic. > > > > But I don't believe any of the pci gpu's use devfreq ;-) > > > > And also, you can't expect the CPU to actually know the freq when fw > > is the one controlling freq. We can, currently, have a reasonable > > approximation from devfreq but that stops if IFPC is implemented. And > > other GPUs have even less direct control. So freq is a thing that I > > don't think we should try to get from "common frameworks" > > I think it might be useful to add another passive devfreq governor type for > external frequencies. This way we can use the same interface to export > non-CPU-controlled frequencies. Yeah this sounds like a decent idea to me too. It might also solve the fun of various pci devices having very non-standard freq controls in sysfs (looking at least at i915 here ...) I guess it would minimally be a good idea if we could document this, or maybe have a reference implementation in nvtop or whatever the cool thing is rn. -Daniel > > > > > BR, > > -R > > > > > > > > > > > What might be needed is better glue to go from the fd or fdinfo to the > > > > > right hw device and then crawl around the hwmon in sysfs > > > > > automatically. I > > > > > would not be surprised at all if we really suck on this, probably more > > > > > likely on SoC than pci gpus where at least everything should be under > > > > > the > > > > > main pci sysfs device. > > > > > > > > yeah, I *think*
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On 11/04/2023 21:28, Rob Clark wrote: On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov wrote: On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: From: Rob Clark Similar motivation to other similar recent attempt[1]. But with an attempt to have some shared code for this. As well as documentation. It is probably a bit UMA-centric, I guess devices with VRAM might want some placement stats as well. But this seems like a reasonable start. Basic gputop support: https://patchwork.freedesktop.org/series/116236/ And already nvtop support: https://github.com/Syllo/nvtop/pull/204 On a related topic, I'm wondering if it would make sense to report some more global things (temp, freq, etc) via fdinfo? Some of this, tools like nvtop could get by trawling sysfs or other driver specific ways. But maybe it makes sense to have these sort of things reported in a standardized way (even though they aren't really per-drm_file) I think that's a bit much layering violation, we'd essentially have to reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to be in :-) I guess this is true for temp (where there are thermal zones with potentially multiple temp sensors.. but I'm still digging my way thru the thermal_cooling_device stuff) It is slightly ugly. All thermal zones and cooling devices are virtual devices (so, even no connection to the particular tsens device). One can either enumerate them by checking /sys/class/thermal/thermal_zoneN/type or enumerate them through /sys/class/hwmon. For cooling devices again the only enumeration is through /sys/class/thermal/cooling_deviceN/type. Probably it should be possible to push cooling devices and thermal zones under corresponding providers. However I do not know if there is a good way to correlate cooling device (ideally a part of GPU) to the thermal_zone (which in our case is provided by tsens / temp_alarm rather than GPU itself). But what about freq? I think, esp for cases where some "fw thing" is controlling the freq we end up needing to use gpu counters to measure the freq. For the freq it is slightly easier: /sys/class/devfreq/*, devices are registered under proper parent (IOW, GPU). So one can read /sys/class/devfreq/3d0.gpu/cur_freq or /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. However because of the components usage, there is no link from /sys/class/drm/card0 (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. Getting all these items together in a platform-independent way would be definitely an important but complex topic. But I don't believe any of the pci gpu's use devfreq ;-) And also, you can't expect the CPU to actually know the freq when fw is the one controlling freq. We can, currently, have a reasonable approximation from devfreq but that stops if IFPC is implemented. And other GPUs have even less direct control. So freq is a thing that I don't think we should try to get from "common frameworks" I think it might be useful to add another passive devfreq governor type for external frequencies. This way we can use the same interface to export non-CPU-controlled frequencies. BR, -R What might be needed is better glue to go from the fd or fdinfo to the right hw device and then crawl around the hwmon in sysfs automatically. I would not be surprised at all if we really suck on this, probably more likely on SoC than pci gpus where at least everything should be under the main pci sysfs device. yeah, I *think* userspace would have to look at /proc/device-tree to find the cooling device(s) associated with the gpu.. at least I don't see a straightforward way to figure it out just for sysfs BR, -R -Daniel BR, -R [1] https://patchwork.freedesktop.org/series/112397/ Rob Clark (2): drm: Add fdinfo memory stats drm/msm: Add memory stats to fdinfo Documentation/gpu/drm-usage-stats.rst | 21 +++ drivers/gpu/drm/drm_file.c| 79 +++ drivers/gpu/drm/msm/msm_drv.c | 25 - drivers/gpu/drm/msm/msm_gpu.c | 2 - include/drm/drm_file.h| 10 5 files changed, 134 insertions(+), 3 deletions(-) -- 2.39.2 -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch -- With best wishes Dmitry -- With best wishes Dmitry
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On 11/04/2023 21:26, Daniel Vetter wrote: On Tue, Apr 11, 2023 at 08:35:48PM +0300, Dmitry Baryshkov wrote: On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: From: Rob Clark Similar motivation to other similar recent attempt[1]. But with an attempt to have some shared code for this. As well as documentation. It is probably a bit UMA-centric, I guess devices with VRAM might want some placement stats as well. But this seems like a reasonable start. Basic gputop support: https://patchwork.freedesktop.org/series/116236/ And already nvtop support: https://github.com/Syllo/nvtop/pull/204 On a related topic, I'm wondering if it would make sense to report some more global things (temp, freq, etc) via fdinfo? Some of this, tools like nvtop could get by trawling sysfs or other driver specific ways. But maybe it makes sense to have these sort of things reported in a standardized way (even though they aren't really per-drm_file) I think that's a bit much layering violation, we'd essentially have to reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to be in :-) I guess this is true for temp (where there are thermal zones with potentially multiple temp sensors.. but I'm still digging my way thru the thermal_cooling_device stuff) It is slightly ugly. All thermal zones and cooling devices are virtual devices (so, even no connection to the particular tsens device). One can either enumerate them by checking /sys/class/thermal/thermal_zoneN/type or enumerate them through /sys/class/hwmon. For cooling devices again the only enumeration is through /sys/class/thermal/cooling_deviceN/type. Probably it should be possible to push cooling devices and thermal zones under corresponding providers. However I do not know if there is a good way to correlate cooling device (ideally a part of GPU) to the thermal_zone (which in our case is provided by tsens / temp_alarm rather than GPU itself). There's not even sysfs links to connect the pieces in both ways? I missed them in the most obvious place: /sys/class/thermal/thermal_zone1/cdev0 -> ../cooling_device0 So, there is a link from thermal zone to cooling device. But what about freq? I think, esp for cases where some "fw thing" is controlling the freq we end up needing to use gpu counters to measure the freq. For the freq it is slightly easier: /sys/class/devfreq/*, devices are registered under proper parent (IOW, GPU). So one can read /sys/class/devfreq/3d0.gpu/cur_freq or /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. However because of the components usage, there is no link from /sys/class/drm/card0 (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. Hm ... do we need to make component more visible in sysfs, with _lots_ of links? Atm it's just not even there. Maybe. Or maybe we should use DPU (the component master and a parent of drm/card0) as devfreq parent too. Getting all these items together in a platform-independent way would be definitely an important but complex topic. Yeah this sounds like some work. But also sounds like it's all generic issues (thermal zones above and component here) that really should be fixed at that level? Cheers, Daniel What might be needed is better glue to go from the fd or fdinfo to the right hw device and then crawl around the hwmon in sysfs automatically. I would not be surprised at all if we really suck on this, probably more likely on SoC than pci gpus where at least everything should be under the main pci sysfs device. yeah, I *think* userspace would have to look at /proc/device-tree to find the cooling device(s) associated with the gpu.. at least I don't see a straightforward way to figure it out just for sysfs BR, -R -Daniel BR, -R [1] https://patchwork.freedesktop.org/series/112397/ Rob Clark (2): drm: Add fdinfo memory stats drm/msm: Add memory stats to fdinfo Documentation/gpu/drm-usage-stats.rst | 21 +++ drivers/gpu/drm/drm_file.c| 79 +++ drivers/gpu/drm/msm/msm_drv.c | 25 - drivers/gpu/drm/msm/msm_gpu.c | 2 - include/drm/drm_file.h| 10 5 files changed, 134 insertions(+), 3 deletions(-) -- 2.39.2 -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch -- With best wishes Dmitry -- With best wishes Dmitry
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Tue, Apr 11, 2023 at 10:36 AM Dmitry Baryshkov wrote: > > On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: > > > > > > > > > > From: Rob Clark > > > > > > > > > > Similar motivation to other similar recent attempt[1]. But with an > > > > > attempt to have some shared code for this. As well as documentation. > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM might want > > > > > some placement stats as well. But this seems like a reasonable start. > > > > > > > > > > Basic gputop support: https://patchwork.freedesktop.org/series/116236/ > > > > > And already nvtop support: https://github.com/Syllo/nvtop/pull/204 > > > > > > > > On a related topic, I'm wondering if it would make sense to report > > > > some more global things (temp, freq, etc) via fdinfo? Some of this, > > > > tools like nvtop could get by trawling sysfs or other driver specific > > > > ways. But maybe it makes sense to have these sort of things reported > > > > in a standardized way (even though they aren't really per-drm_file) > > > > > > I think that's a bit much layering violation, we'd essentially have to > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to > > > be in :-) > > > > I guess this is true for temp (where there are thermal zones with > > potentially multiple temp sensors.. but I'm still digging my way thru > > the thermal_cooling_device stuff) > > It is slightly ugly. All thermal zones and cooling devices are virtual > devices (so, even no connection to the particular tsens device). One > can either enumerate them by checking > /sys/class/thermal/thermal_zoneN/type or enumerate them through > /sys/class/hwmon. For cooling devices again the only enumeration is > through /sys/class/thermal/cooling_deviceN/type. > > Probably it should be possible to push cooling devices and thermal > zones under corresponding providers. However I do not know if there is > a good way to correlate cooling device (ideally a part of GPU) to the > thermal_zone (which in our case is provided by tsens / temp_alarm > rather than GPU itself). > > > > > But what about freq? I think, esp for cases where some "fw thing" is > > controlling the freq we end up needing to use gpu counters to measure > > the freq. > > For the freq it is slightly easier: /sys/class/devfreq/*, devices are > registered under proper parent (IOW, GPU). So one can read > /sys/class/devfreq/3d0.gpu/cur_freq or > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > However because of the components usage, there is no link from > /sys/class/drm/card0 > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. > > Getting all these items together in a platform-independent way would > be definitely an important but complex topic. But I don't believe any of the pci gpu's use devfreq ;-) And also, you can't expect the CPU to actually know the freq when fw is the one controlling freq. We can, currently, have a reasonable approximation from devfreq but that stops if IFPC is implemented. And other GPUs have even less direct control. So freq is a thing that I don't think we should try to get from "common frameworks" BR, -R > > > > > What might be needed is better glue to go from the fd or fdinfo to the > > > right hw device and then crawl around the hwmon in sysfs automatically. I > > > would not be surprised at all if we really suck on this, probably more > > > likely on SoC than pci gpus where at least everything should be under the > > > main pci sysfs device. > > > > yeah, I *think* userspace would have to look at /proc/device-tree to > > find the cooling device(s) associated with the gpu.. at least I don't > > see a straightforward way to figure it out just for sysfs > > > > BR, > > -R > > > > > -Daniel > > > > > > > > > > > BR, > > > > -R > > > > > > > > > > > > > [1] https://patchwork.freedesktop.org/series/112397/ > > > > > > > > > > Rob Clark (2): > > > > > drm: Add fdinfo memory stats > > > > > drm/msm: Add memory stats to fdinfo > > > > > > > > > > Documentation/gpu/drm-usage-stats.rst | 21 +++ > > > > > drivers/gpu/drm/drm_file.c| 79 > > > > > +++ > > > > > drivers/gpu/drm/msm/msm_drv.c | 25 - > > > > > drivers/gpu/drm/msm/msm_gpu.c | 2 - > > > > > include/drm/drm_file.h| 10 > > > > > 5 files changed, 134 insertions(+), 3 deletions(-) > > > > > > > > > > -- > > > > > 2.39.2 > > > > > > > > > > > -- > > > Daniel Vetter > > > Software Engineer, Intel Corporation > > > http://blog.ffwll.ch > > > > -- > With best wishes > Dmitry
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Tue, Apr 11, 2023 at 08:35:48PM +0300, Dmitry Baryshkov wrote: > On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: > > > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: > > > > > > > > > > From: Rob Clark > > > > > > > > > > Similar motivation to other similar recent attempt[1]. But with an > > > > > attempt to have some shared code for this. As well as documentation. > > > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM might want > > > > > some placement stats as well. But this seems like a reasonable start. > > > > > > > > > > Basic gputop support: https://patchwork.freedesktop.org/series/116236/ > > > > > And already nvtop support: https://github.com/Syllo/nvtop/pull/204 > > > > > > > > On a related topic, I'm wondering if it would make sense to report > > > > some more global things (temp, freq, etc) via fdinfo? Some of this, > > > > tools like nvtop could get by trawling sysfs or other driver specific > > > > ways. But maybe it makes sense to have these sort of things reported > > > > in a standardized way (even though they aren't really per-drm_file) > > > > > > I think that's a bit much layering violation, we'd essentially have to > > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to > > > be in :-) > > > > I guess this is true for temp (where there are thermal zones with > > potentially multiple temp sensors.. but I'm still digging my way thru > > the thermal_cooling_device stuff) > > It is slightly ugly. All thermal zones and cooling devices are virtual > devices (so, even no connection to the particular tsens device). One > can either enumerate them by checking > /sys/class/thermal/thermal_zoneN/type or enumerate them through > /sys/class/hwmon. For cooling devices again the only enumeration is > through /sys/class/thermal/cooling_deviceN/type. > > Probably it should be possible to push cooling devices and thermal > zones under corresponding providers. However I do not know if there is > a good way to correlate cooling device (ideally a part of GPU) to the > thermal_zone (which in our case is provided by tsens / temp_alarm > rather than GPU itself). There's not even sysfs links to connect the pieces in both ways? > > But what about freq? I think, esp for cases where some "fw thing" is > > controlling the freq we end up needing to use gpu counters to measure > > the freq. > > For the freq it is slightly easier: /sys/class/devfreq/*, devices are > registered under proper parent (IOW, GPU). So one can read > /sys/class/devfreq/3d0.gpu/cur_freq or > /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. > > However because of the components usage, there is no link from > /sys/class/drm/card0 > (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) > to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. Hm ... do we need to make component more visible in sysfs, with _lots_ of links? Atm it's just not even there. > Getting all these items together in a platform-independent way would > be definitely an important but complex topic. Yeah this sounds like some work. But also sounds like it's all generic issues (thermal zones above and component here) that really should be fixed at that level? Cheers, Daniel > > > What might be needed is better glue to go from the fd or fdinfo to the > > > right hw device and then crawl around the hwmon in sysfs automatically. I > > > would not be surprised at all if we really suck on this, probably more > > > likely on SoC than pci gpus where at least everything should be under the > > > main pci sysfs device. > > > > yeah, I *think* userspace would have to look at /proc/device-tree to > > find the cooling device(s) associated with the gpu.. at least I don't > > see a straightforward way to figure it out just for sysfs > > > > BR, > > -R > > > > > -Daniel > > > > > > > > > > > BR, > > > > -R > > > > > > > > > > > > > [1] https://patchwork.freedesktop.org/series/112397/ > > > > > > > > > > Rob Clark (2): > > > > > drm: Add fdinfo memory stats > > > > > drm/msm: Add memory stats to fdinfo > > > > > > > > > > Documentation/gpu/drm-usage-stats.rst | 21 +++ > > > > > drivers/gpu/drm/drm_file.c| 79 > > > > > +++ > > > > > drivers/gpu/drm/msm/msm_drv.c | 25 - > > > > > drivers/gpu/drm/msm/msm_gpu.c | 2 - > > > > > include/drm/drm_file.h| 10 > > > > > 5 files changed, 134 insertions(+), 3 deletions(-) > > > > > > > > > > -- > > > > > 2.39.2 > > > > > > > > > > > -- > > > Daniel Vetter > > > Software Engineer, Intel Corporation > > > http://blog.ffwll.ch > > > > -- > With best wishes > Dmitry -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Tue, 11 Apr 2023 at 20:13, Rob Clark wrote: > > On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: > > > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: > > > > > > > > From: Rob Clark > > > > > > > > Similar motivation to other similar recent attempt[1]. But with an > > > > attempt to have some shared code for this. As well as documentation. > > > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM might want > > > > some placement stats as well. But this seems like a reasonable start. > > > > > > > > Basic gputop support: https://patchwork.freedesktop.org/series/116236/ > > > > And already nvtop support: https://github.com/Syllo/nvtop/pull/204 > > > > > > On a related topic, I'm wondering if it would make sense to report > > > some more global things (temp, freq, etc) via fdinfo? Some of this, > > > tools like nvtop could get by trawling sysfs or other driver specific > > > ways. But maybe it makes sense to have these sort of things reported > > > in a standardized way (even though they aren't really per-drm_file) > > > > I think that's a bit much layering violation, we'd essentially have to > > reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to > > be in :-) > > I guess this is true for temp (where there are thermal zones with > potentially multiple temp sensors.. but I'm still digging my way thru > the thermal_cooling_device stuff) It is slightly ugly. All thermal zones and cooling devices are virtual devices (so, even no connection to the particular tsens device). One can either enumerate them by checking /sys/class/thermal/thermal_zoneN/type or enumerate them through /sys/class/hwmon. For cooling devices again the only enumeration is through /sys/class/thermal/cooling_deviceN/type. Probably it should be possible to push cooling devices and thermal zones under corresponding providers. However I do not know if there is a good way to correlate cooling device (ideally a part of GPU) to the thermal_zone (which in our case is provided by tsens / temp_alarm rather than GPU itself). > > But what about freq? I think, esp for cases where some "fw thing" is > controlling the freq we end up needing to use gpu counters to measure > the freq. For the freq it is slightly easier: /sys/class/devfreq/*, devices are registered under proper parent (IOW, GPU). So one can read /sys/class/devfreq/3d0.gpu/cur_freq or /sys/bus/platform/devices/3d0.gpu/devfreq/3d0.gpu/cur_freq. However because of the components usage, there is no link from /sys/class/drm/card0 (/sys/devices/platform/soc@0/ae0.display-subsystem/ae01000.display-controller/drm/card0) to /sys/devices/platform/soc@0/3d0.gpu, the GPU unit. Getting all these items together in a platform-independent way would be definitely an important but complex topic. > > > What might be needed is better glue to go from the fd or fdinfo to the > > right hw device and then crawl around the hwmon in sysfs automatically. I > > would not be surprised at all if we really suck on this, probably more > > likely on SoC than pci gpus where at least everything should be under the > > main pci sysfs device. > > yeah, I *think* userspace would have to look at /proc/device-tree to > find the cooling device(s) associated with the gpu.. at least I don't > see a straightforward way to figure it out just for sysfs > > BR, > -R > > > -Daniel > > > > > > > > BR, > > > -R > > > > > > > > > > [1] https://patchwork.freedesktop.org/series/112397/ > > > > > > > > Rob Clark (2): > > > > drm: Add fdinfo memory stats > > > > drm/msm: Add memory stats to fdinfo > > > > > > > > Documentation/gpu/drm-usage-stats.rst | 21 +++ > > > > drivers/gpu/drm/drm_file.c| 79 +++ > > > > drivers/gpu/drm/msm/msm_drv.c | 25 - > > > > drivers/gpu/drm/msm/msm_gpu.c | 2 - > > > > include/drm/drm_file.h| 10 > > > > 5 files changed, 134 insertions(+), 3 deletions(-) > > > > > > > > -- > > > > 2.39.2 > > > > > > > > -- > > Daniel Vetter > > Software Engineer, Intel Corporation > > http://blog.ffwll.ch -- With best wishes Dmitry
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Tue, Apr 11, 2023 at 9:53 AM Daniel Vetter wrote: > > On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: > > > > > > From: Rob Clark > > > > > > Similar motivation to other similar recent attempt[1]. But with an > > > attempt to have some shared code for this. As well as documentation. > > > > > > It is probably a bit UMA-centric, I guess devices with VRAM might want > > > some placement stats as well. But this seems like a reasonable start. > > > > > > Basic gputop support: https://patchwork.freedesktop.org/series/116236/ > > > And already nvtop support: https://github.com/Syllo/nvtop/pull/204 > > > > On a related topic, I'm wondering if it would make sense to report > > some more global things (temp, freq, etc) via fdinfo? Some of this, > > tools like nvtop could get by trawling sysfs or other driver specific > > ways. But maybe it makes sense to have these sort of things reported > > in a standardized way (even though they aren't really per-drm_file) > > I think that's a bit much layering violation, we'd essentially have to > reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to > be in :-) I guess this is true for temp (where there are thermal zones with potentially multiple temp sensors.. but I'm still digging my way thru the thermal_cooling_device stuff) But what about freq? I think, esp for cases where some "fw thing" is controlling the freq we end up needing to use gpu counters to measure the freq. > What might be needed is better glue to go from the fd or fdinfo to the > right hw device and then crawl around the hwmon in sysfs automatically. I > would not be surprised at all if we really suck on this, probably more > likely on SoC than pci gpus where at least everything should be under the > main pci sysfs device. yeah, I *think* userspace would have to look at /proc/device-tree to find the cooling device(s) associated with the gpu.. at least I don't see a straightforward way to figure it out just for sysfs BR, -R > -Daniel > > > > > BR, > > -R > > > > > > > [1] https://patchwork.freedesktop.org/series/112397/ > > > > > > Rob Clark (2): > > > drm: Add fdinfo memory stats > > > drm/msm: Add memory stats to fdinfo > > > > > > Documentation/gpu/drm-usage-stats.rst | 21 +++ > > > drivers/gpu/drm/drm_file.c| 79 +++ > > > drivers/gpu/drm/msm/msm_drv.c | 25 - > > > drivers/gpu/drm/msm/msm_gpu.c | 2 - > > > include/drm/drm_file.h| 10 > > > 5 files changed, 134 insertions(+), 3 deletions(-) > > > > > > -- > > > 2.39.2 > > > > > -- > Daniel Vetter > Software Engineer, Intel Corporation > http://blog.ffwll.ch
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Tue, Apr 11, 2023 at 09:47:32AM -0700, Rob Clark wrote: > On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: > > > > From: Rob Clark > > > > Similar motivation to other similar recent attempt[1]. But with an > > attempt to have some shared code for this. As well as documentation. > > > > It is probably a bit UMA-centric, I guess devices with VRAM might want > > some placement stats as well. But this seems like a reasonable start. > > > > Basic gputop support: https://patchwork.freedesktop.org/series/116236/ > > And already nvtop support: https://github.com/Syllo/nvtop/pull/204 > > On a related topic, I'm wondering if it would make sense to report > some more global things (temp, freq, etc) via fdinfo? Some of this, > tools like nvtop could get by trawling sysfs or other driver specific > ways. But maybe it makes sense to have these sort of things reported > in a standardized way (even though they aren't really per-drm_file) I think that's a bit much layering violation, we'd essentially have to reinvent the hwmon sysfs uapi in fdinfo. Not really a business I want to be in :-) What might be needed is better glue to go from the fd or fdinfo to the right hw device and then crawl around the hwmon in sysfs automatically. I would not be surprised at all if we really suck on this, probably more likely on SoC than pci gpus where at least everything should be under the main pci sysfs device. -Daniel > > BR, > -R > > > > [1] https://patchwork.freedesktop.org/series/112397/ > > > > Rob Clark (2): > > drm: Add fdinfo memory stats > > drm/msm: Add memory stats to fdinfo > > > > Documentation/gpu/drm-usage-stats.rst | 21 +++ > > drivers/gpu/drm/drm_file.c| 79 +++ > > drivers/gpu/drm/msm/msm_drv.c | 25 - > > drivers/gpu/drm/msm/msm_gpu.c | 2 - > > include/drm/drm_file.h| 10 > > 5 files changed, 134 insertions(+), 3 deletions(-) > > > > -- > > 2.39.2 > > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Re: [Freedreno] [PATCH v2 0/2] drm: fdinfo memory stats
On Mon, Apr 10, 2023 at 2:06 PM Rob Clark wrote: > > From: Rob Clark > > Similar motivation to other similar recent attempt[1]. But with an > attempt to have some shared code for this. As well as documentation. > > It is probably a bit UMA-centric, I guess devices with VRAM might want > some placement stats as well. But this seems like a reasonable start. > > Basic gputop support: https://patchwork.freedesktop.org/series/116236/ > And already nvtop support: https://github.com/Syllo/nvtop/pull/204 On a related topic, I'm wondering if it would make sense to report some more global things (temp, freq, etc) via fdinfo? Some of this, tools like nvtop could get by trawling sysfs or other driver specific ways. But maybe it makes sense to have these sort of things reported in a standardized way (even though they aren't really per-drm_file) BR, -R > [1] https://patchwork.freedesktop.org/series/112397/ > > Rob Clark (2): > drm: Add fdinfo memory stats > drm/msm: Add memory stats to fdinfo > > Documentation/gpu/drm-usage-stats.rst | 21 +++ > drivers/gpu/drm/drm_file.c| 79 +++ > drivers/gpu/drm/msm/msm_drv.c | 25 - > drivers/gpu/drm/msm/msm_gpu.c | 2 - > include/drm/drm_file.h| 10 > 5 files changed, 134 insertions(+), 3 deletions(-) > > -- > 2.39.2 >
