Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-11-21 Thread Tvrtko Ursulin 


On 20/11/2017 21:19, Pierre-Loup A. Griffais wrote:

On 11/20/2017 06:01 AM, Tvrtko Ursulin wrote:


Hi,

On 16/11/2017 20:42, Michael Sartain wrote:

On Wed, Sep 6, 2017, at 02:09 AM, Chris Wilson wrote:

Quoting Daniel Vetter (2017-09-06 08:46:50)

Hi Pierre,

On Tue, Sep 5, 2017 at 11:15 PM, Pierre-Loup A. Griffais
 wrote:

Hi Daniel,

In the past couple of months we've been working on gpuvis, a GPU 
tracing
tool similar to GPUView on Windows. It's lower level than other 
API-based
tracing tools and lets you debug system-wide GPU scheduling 
issues, eg.

interaction between several processes using the GPU, which is pretty
critical for VR usecases.

It's all based on ftrace; we've initially developped it with 
support for
amdgpu, and had to patch the kernel code there to change what 
tracing events
are reported and how. Now that we have a good idea of what's 
needed and it's
more or less proven in production, we were wondering if you had 
any interest
in adding a similar set of events for Intel GPUs so we could read 
them and
present them the same way? We have pretty specific requirements, 
but this
work-in-progress documentation should give a good idea of what 
they are:


https://github.com/mikesart/gpuvis/wiki/Overview


We already have those tracepoint equivs and a script to generate a
similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
looking at the scheduling issue from the gpu pov. But it's really 
only a

dev toy atm, plugging the gap between userspace and the gpu has been on
the perennial wishlist.
-Chris


I added Intel event visualization to gpuvis based on your trace.pl
script. Screenshot at the top of the wiki page here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-Intel

In that screenshot the mouse is hovering over the ctx=30,seqno=1900 bar
which selects those events in the event list and shows a tooltip with
the submit, execute, etc info.


It certainly looks immensely better than my browser based hack. But 
unfortunately I still did not get round actually trying your tool.


How scalable it is - meaning - can it handle very busy and huge traces?


The typical SteamVR trace in our "DVR" plumbing (always tracing in the 
background) has about 500k events over ~20 seconds. I can zoom and scrub 
through it at 60fps without issues here.


Sounds quite good!


Is there any outlook of it getting packaged in some distro?


The tool itself is standalone and very easy to package; Mike recently 
wrote some plumbing example scripts around it that can get useful 
captures, so I would think we're in better shape for packaging now than 
before they existed. We bundle it with SteamVR so haven't really spent 
any time looking into packaging.


Okay, I was just curious and there probably isn't any direct need from 
our side for this. Only that it can help with adding new interesting 
bits to the kernel. Tracepoints are kind of neither here or there in 
this respect.



For the amdgpu driver, we're able to get the submit information from
user space and associate those events to specific processes. Example of
that is here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-AMDGpu

If you ever get a chance to try gpuvis and have any feedback, we'd love
to hear it.  Also if you ever get userspace tracepoint data in, let me
know and I'd be happy to hook that up as well.


What kind of information is missing to wire up this missing bit? I 
mean the thing you are referring to as user space submit data, what is 
that?


The main thing is to be able to associate a chunk of GPU work with a 
userspace process. Currently, there's no tracepoint we're aware of in 
the work submission ioctl, which means that while the GPU tasks are 
properly displayed, they're anonymous. The main intent behind gpuvis is 
to be able to debug multi-process GPU interaction and timing problems, 
so having a tracepoint in the submit ioctl lets us associate the user 
context of the application that submitted the work with the work itself 
if it shares the same identifying seqnos/ids. gpuvis can then show you 
thread information about that process, color-code for easy 
disambiguation, etc.


We have i915_gem_request_queue which is at the submit time (execbuf 
ioctl), can you have a look and see if that would be enough for gpuvis?


Btw one new thing we are close to merging to i915 is the perf PMU 
support. That will enable real-time monitoring of per-engine busyness, 
waits, frequency, power, maybe more in the future like queue-depth. I 
don't know if things like that would be interesting for gpuvis? Some 
of it can be inferred from the tracepoints already in post-processing 
so there is some overlap. I am not sure, but thought to mention it. 
This is the series: https://patchwork.freedesktop.org/series/27488/. 
It is used via existing perf userspace API.


Seems like the kind of thing gpuvis could happily overlay on top of the 
trace with its plotting functionality.


Cool. And it even sounds

Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-11-20 Thread Pierre-Loup A. Griffais 

On 11/20/2017 06:01 AM, Tvrtko Ursulin wrote:


Hi,

On 16/11/2017 20:42, Michael Sartain wrote:

On Wed, Sep 6, 2017, at 02:09 AM, Chris Wilson wrote:

Quoting Daniel Vetter (2017-09-06 08:46:50)

Hi Pierre,

On Tue, Sep 5, 2017 at 11:15 PM, Pierre-Loup A. Griffais
 wrote:

Hi Daniel,

In the past couple of months we've been working on gpuvis, a GPU 
tracing
tool similar to GPUView on Windows. It's lower level than other 
API-based
tracing tools and lets you debug system-wide GPU scheduling issues, 
eg.

interaction between several processes using the GPU, which is pretty
critical for VR usecases.

It's all based on ftrace; we've initially developped it with 
support for
amdgpu, and had to patch the kernel code there to change what 
tracing events
are reported and how. Now that we have a good idea of what's needed 
and it's
more or less proven in production, we were wondering if you had any 
interest
in adding a similar set of events for Intel GPUs so we could read 
them and
present them the same way? We have pretty specific requirements, 
but this
work-in-progress documentation should give a good idea of what they 
are:


https://github.com/mikesart/gpuvis/wiki/Overview


We already have those tracepoint equivs and a script to generate a
similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
looking at the scheduling issue from the gpu pov. But it's really only a
dev toy atm, plugging the gap between userspace and the gpu has been on
the perennial wishlist.
-Chris


I added Intel event visualization to gpuvis based on your trace.pl
script. Screenshot at the top of the wiki page here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-Intel

In that screenshot the mouse is hovering over the ctx=30,seqno=1900 bar
which selects those events in the event list and shows a tooltip with
the submit, execute, etc info.


It certainly looks immensely better than my browser based hack. But 
unfortunately I still did not get round actually trying your tool.


How scalable it is - meaning - can it handle very busy and huge traces?


The typical SteamVR trace in our "DVR" plumbing (always tracing in the 
background) has about 500k events over ~20 seconds. I can zoom and scrub 
through it at 60fps without issues here.




Is there any outlook of it getting packaged in some distro?


The tool itself is standalone and very easy to package; Mike recently 
wrote some plumbing example scripts around it that can get useful 
captures, so I would think we're in better shape for packaging now than 
before they existed. We bundle it with SteamVR so haven't really spent 
any time looking into packaging.





For the amdgpu driver, we're able to get the submit information from
user space and associate those events to specific processes. Example of
that is here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-AMDGpu

If you ever get a chance to try gpuvis and have any feedback, we'd love
to hear it.  Also if you ever get userspace tracepoint data in, let me
know and I'd be happy to hook that up as well.


What kind of information is missing to wire up this missing bit? I mean 
the thing you are referring to as user space submit data, what is that?


The main thing is to be able to associate a chunk of GPU work with a 
userspace process. Currently, there's no tracepoint we're aware of in 
the work submission ioctl, which means that while the GPU tasks are 
properly displayed, they're anonymous. The main intent behind gpuvis is 
to be able to debug multi-process GPU interaction and timing problems, 
so having a tracepoint in the submit ioctl lets us associate the user 
context of the application that submitted the work with the work itself 
if it shares the same identifying seqnos/ids. gpuvis can then show you 
thread information about that process, color-code for easy 
disambiguation, etc.




Btw one new thing we are close to merging to i915 is the perf PMU 
support. That will enable real-time monitoring of per-engine busyness, 
waits, frequency, power, maybe more in the future like queue-depth. I 
don't know if things like that would be interesting for gpuvis? Some of 
it can be inferred from the tracepoints already in post-processing so 
there is some overlap. I am not sure, but thought to mention it. This is 
the series: https://patchwork.freedesktop.org/series/27488/. It is used 
via existing perf userspace API.


Seems like the kind of thing gpuvis could happily overlay on top of the 
trace with its plotting functionality.




Regards,

Tvrtko



___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-11-20 Thread Tvrtko Ursulin 


Hi,

On 16/11/2017 20:42, Michael Sartain wrote:

On Wed, Sep 6, 2017, at 02:09 AM, Chris Wilson wrote:

Quoting Daniel Vetter (2017-09-06 08:46:50)

Hi Pierre,

On Tue, Sep 5, 2017 at 11:15 PM, Pierre-Loup A. Griffais
 wrote:

Hi Daniel,

In the past couple of months we've been working on gpuvis, a GPU tracing
tool similar to GPUView on Windows. It's lower level than other API-based
tracing tools and lets you debug system-wide GPU scheduling issues, eg.
interaction between several processes using the GPU, which is pretty
critical for VR usecases.

It's all based on ftrace; we've initially developped it with support for
amdgpu, and had to patch the kernel code there to change what tracing events
are reported and how. Now that we have a good idea of what's needed and it's
more or less proven in production, we were wondering if you had any interest
in adding a similar set of events for Intel GPUs so we could read them and
present them the same way? We have pretty specific requirements, but this
work-in-progress documentation should give a good idea of what they are:

https://github.com/mikesart/gpuvis/wiki/Overview


We already have those tracepoint equivs and a script to generate a
similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
looking at the scheduling issue from the gpu pov. But it's really only a
dev toy atm, plugging the gap between userspace and the gpu has been on
the perennial wishlist.
-Chris


I added Intel event visualization to gpuvis based on your trace.pl
script. Screenshot at the top of the wiki page here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-Intel

In that screenshot the mouse is hovering over the ctx=30,seqno=1900 bar
which selects those events in the event list and shows a tooltip with
the submit, execute, etc info.


It certainly looks immensely better than my browser based hack. But 
unfortunately I still did not get round actually trying your tool.


How scalable it is - meaning - can it handle very busy and huge traces?

Is there any outlook of it getting packaged in some distro?


For the amdgpu driver, we're able to get the submit information from
user space and associate those events to specific processes. Example of
that is here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-AMDGpu

If you ever get a chance to try gpuvis and have any feedback, we'd love
to hear it.  Also if you ever get userspace tracepoint data in, let me
know and I'd be happy to hook that up as well.


What kind of information is missing to wire up this missing bit? I mean 
the thing you are referring to as user space submit data, what is that?


Btw one new thing we are close to merging to i915 is the perf PMU 
support. That will enable real-time monitoring of per-engine busyness, 
waits, frequency, power, maybe more in the future like queue-depth. I 
don't know if things like that would be interesting for gpuvis? Some of 
it can be inferred from the tracepoints already in post-processing so 
there is some overlap. I am not sure, but thought to mention it. This is 
the series: https://patchwork.freedesktop.org/series/27488/. It is used 
via existing perf userspace API.


Regards,

Tvrtko
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-11-16 Thread Michael Sartain 
On Wed, Sep 6, 2017, at 02:09 AM, Chris Wilson wrote:
> Quoting Daniel Vetter (2017-09-06 08:46:50)
> > Hi Pierre,
> > 
> > On Tue, Sep 5, 2017 at 11:15 PM, Pierre-Loup A. Griffais
> >  wrote:
> > > Hi Daniel,
> > >
> > > In the past couple of months we've been working on gpuvis, a GPU tracing
> > > tool similar to GPUView on Windows. It's lower level than other API-based
> > > tracing tools and lets you debug system-wide GPU scheduling issues, eg.
> > > interaction between several processes using the GPU, which is pretty
> > > critical for VR usecases.
> > >
> > > It's all based on ftrace; we've initially developped it with support for
> > > amdgpu, and had to patch the kernel code there to change what tracing 
> > > events
> > > are reported and how. Now that we have a good idea of what's needed and 
> > > it's
> > > more or less proven in production, we were wondering if you had any 
> > > interest
> > > in adding a similar set of events for Intel GPUs so we could read them and
> > > present them the same way? We have pretty specific requirements, but this
> > > work-in-progress documentation should give a good idea of what they are:
> > >
> > > https://github.com/mikesart/gpuvis/wiki/Overview
> 
> We already have those tracepoint equivs and a script to generate a
> similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
> looking at the scheduling issue from the gpu pov. But it's really only a
> dev toy atm, plugging the gap between userspace and the gpu has been on
> the perennial wishlist.
> -Chris

I added Intel event visualization to gpuvis based on your trace.pl
script. Screenshot at the top of the wiki page here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-Intel

In that screenshot the mouse is hovering over the ctx=30,seqno=1900 bar
which selects those events in the event list and shows a tooltip with
the submit, execute, etc info.

For the amdgpu driver, we're able to get the submit information from
user space and associate those events to specific processes. Example of
that is here:

https://github.com/mikesart/gpuvis/wiki/TechDocs-AMDGpu

If you ever get a chance to try gpuvis and have any feedback, we'd love
to hear it.  Also if you ever get userspace tracepoint data in, let me
know and I'd be happy to hook that up as well.

Thanks much.
 -Mike
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-09-27 Thread Michael Sartain 
On Wed, Sep 6, 2017, at 03:09 AM, Chris Wilson wrote:
> We already have those tracepoint equivs and a script to generate a
> similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
> looking at the scheduling issue from the gpu pov. But it's really only a
> dev toy atm, plugging the gap between userspace and the gpu has been on
> the perennial wishlist.

Ah, cool. I'll take a look at the events in trace.pl and try doing some
captures with it on my intel laptop.

Thanks for the pointer Chris.
 -Mike
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-09-06 Thread Pierre-Loup A. Griffais 



On 09/06/2017 12:45 PM, Michael Sartain wrote:

On Wed, Sep 6, 2017, at 03:09 AM, Chris Wilson wrote:

We already have those tracepoint equivs and a script to generate a
similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
looking at the scheduling issue from the gpu pov. But it's really only a
dev toy atm, plugging the gap between userspace and the gpu has been on
the perennial wishlist.


Ah, cool. I'll take a look at the events in trace.pl and try doing some
captures with it on my intel laptop.


Yep, thanks, that's helpful; not sure why I got dropped there but Mike 
kindly re-added me (am not on intel-gfx).


Since you called out many more events than we were expecting, should we 
assume the situation is not quite as simple as amdgpu where we have a 
tracepoint for work getting queued on HW pipes, and a tracepoint for 
work finishing? With the tracepoint in the initial ioctl for work 
submission and the appropriate seqnos, just these three let us 
reconstruct the whole timeline for all HW pipes, and attribute it to a 
userspace process. I understand that last one does not yet exist for 
Intel, as you pointed out. If you had any pointers on where to add it, 
maybe we could prototype something, but not sure how much work we'll be 
able to do directly on Intel HW.


Thanks,
 - Pierre-Loup



Thanks for the pointer Chris.
  -Mike



___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-09-06 Thread Chris Wilson 
Quoting Daniel Vetter (2017-09-06 08:46:50)
> Hi Pierre,
> 
> On Tue, Sep 5, 2017 at 11:15 PM, Pierre-Loup A. Griffais
>  wrote:
> > Hi Daniel,
> >
> > In the past couple of months we've been working on gpuvis, a GPU tracing
> > tool similar to GPUView on Windows. It's lower level than other API-based
> > tracing tools and lets you debug system-wide GPU scheduling issues, eg.
> > interaction between several processes using the GPU, which is pretty
> > critical for VR usecases.
> >
> > It's all based on ftrace; we've initially developped it with support for
> > amdgpu, and had to patch the kernel code there to change what tracing events
> > are reported and how. Now that we have a good idea of what's needed and it's
> > more or less proven in production, we were wondering if you had any interest
> > in adding a similar set of events for Intel GPUs so we could read them and
> > present them the same way? We have pretty specific requirements, but this
> > work-in-progress documentation should give a good idea of what they are:
> >
> > https://github.com/mikesart/gpuvis/wiki/Overview

We already have those tracepoint equivs and a script to generate a
similar visualisation: intel-gpu-tools/scripts/trace.pl, but only
looking at the scheduling issue from the gpu pov. But it's really only a
dev toy atm, plugging the gap between userspace and the gpu has been on
the perennial wishlist.
-Chris
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] Supporting Intel GPU tracing in gpuvis

2017-09-06 Thread Daniel Vetter 
Hi Pierre,

On Tue, Sep 5, 2017 at 11:15 PM, Pierre-Loup A. Griffais
 wrote:
> Hi Daniel,
>
> In the past couple of months we've been working on gpuvis, a GPU tracing
> tool similar to GPUView on Windows. It's lower level than other API-based
> tracing tools and lets you debug system-wide GPU scheduling issues, eg.
> interaction between several processes using the GPU, which is pretty
> critical for VR usecases.
>
> It's all based on ftrace; we've initially developped it with support for
> amdgpu, and had to patch the kernel code there to change what tracing events
> are reported and how. Now that we have a good idea of what's needed and it's
> more or less proven in production, we were wondering if you had any interest
> in adding a similar set of events for Intel GPUs so we could read them and
> present them the same way? We have pretty specific requirements, but this
> work-in-progress documentation should give a good idea of what they are:
>
> https://github.com/mikesart/gpuvis/wiki/Overview
>
> More raw details here:
>
> https://github.com/mikesart/gpuvis/wiki/TechDocs
>
> Even though VR is not really a factor there yet, we think supporting Intel
> GPUs could be interesting to your userspace driver devs. That tool is
> growing quite useful. We're also working with NVIDIA devs to get us similar
> data exposed from their stack, so it'd be good if gpuvis could 'just work'
> on all systems going forward.
>
> (Ian Romanick suggested I reach out to you first, but if you have a
> different set of people that should be included feel free to add them)

Yeah I'm not exactly the tracing/perf guy around here :-) Adding
relevant people who should be able to point you in a useful direction.

We already have a pile of tracepoints and there's patches floating for
global perf PMUs, but not sure what exactly you need.

Also adding mailing lists, better to discuss stuff in public.

Thanks, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx