Re: Too many PMC implementations
On Thu, Aug 23, 2018 at 10:17:29AM -0700, Jason Thorpe wrote: > > > On Aug 23, 2018, at 8:47 AM, Anders Magnusson wrote: > > > I have used it not long ago for vax. Maybe I did have to do some tweaks, > > do not remember, > > but I really want to be able to use kernel profiling on vax. > > > > So, I really oppose removing it and leaving vax without any kernel > > profiling choice. > > How hard would it be to add support for dtrace on Vax? Without FBT, probably pretty easy. But of course FBT is the only plausible replacement for a statistical profiler that DTrace offers. The basic requirement for FBT is a dynamic patcher (to really do it right); though some __predict-false branches can be inserted at the head of every function and a global used instead. -- Thor Lancelot Simon t...@panix.com "Whether or not there's hope for change is not the question. If you want to be a free person, you don't stand up for human rights because it will work, but because it is right." --Andrei Sakharov
Re: Too many PMC implementations
> On Aug 23, 2018, at 8:47 AM, Anders Magnusson wrote: > I have used it not long ago for vax. Maybe I did have to do some tweaks, do > not remember, > but I really want to be able to use kernel profiling on vax. > > So, I really oppose removing it and leaving vax without any kernel profiling > choice. How hard would it be to add support for dtrace on Vax? -- thorpej
Re: Too many PMC implementations
On 23.08.2018 18:35, Thor Lancelot Simon wrote: > On Thu, Aug 23, 2018 at 06:25:56PM +0200, Kamil Rytarowski wrote: >> >> As useful I mean the number of commits to the src/ tree. If nothing >> landed, probably nothing was useful. When were the most recent patches >> from gprof or similar? > > This is a plainly bogus criterion. After we integrated DTrace, there > were several periods of a year or more during which there were no > "patches from DTrace or similar". I know, and it was pretty frustrating > to me too, since I paid a considerable amount of money to have it ported > and maintained, and I had to justify that to my boss. > > Should it have, then, been ripped out? It's certainly a lot more complex > than gprof. > Contrary to gprof (idea from 1988), actually DTrace can be utilized easier on a modern hardware. Probably DTrace is not the final word in BSD and not something I intend to defend - but it's a good solution for now - (FreeBSD already ports/develops a potential replacement eBPF). signature.asc Description: OpenPGP digital signature
Re: Too many PMC implementations
On Thu, Aug 23, 2018 at 06:25:56PM +0200, Kamil Rytarowski wrote: > > As useful I mean the number of commits to the src/ tree. If nothing > landed, probably nothing was useful. When were the most recent patches > from gprof or similar? This is a plainly bogus criterion. After we integrated DTrace, there were several periods of a year or more during which there were no "patches from DTrace or similar". I know, and it was pretty frustrating to me too, since I paid a considerable amount of money to have it ported and maintained, and I had to justify that to my boss. Should it have, then, been ripped out? It's certainly a lot more complex than gprof. -- Thor Lancelot Simon t...@panix.com "Whether or not there's hope for change is not the question. If you want to be a free person, you don't stand up for human rights because it will work, but because it is right." --Andrei Sakharov
Re: Too many PMC implementations
On 23.08.2018 17:57, Thor Lancelot Simon wrote: > On Thu, Aug 23, 2018 at 05:09:35PM +0200, Kamil Rytarowski wrote: >> >> Observing that all the useful profiling is already done with DTrace, we >> can remove complexity from the kernel with negligible cost. > > I'm not sure what to make of this. I'm trying to come up with a way to > make the above statement true, and I'm having some difficulty. > > You can't possibly mean "Observing that (unproven premise), therefore > (conclusion)", so I'll discard that interpretation. > > Do you perhaps mean "*If* we were to observe that all useful profiling > were done with DTrace, *then* we could remove complexity from the > kernel with negligible cost"? > > Because Ragge and others have been pointing out that in that case, > the premise "all useful profiling is done with DTrace" does not appear > to be true. Profiling kernel code on VAX may not be useful *to you* > but that does not imply it is "not useful" simpliciter. > As useful I mean the number of commits to the src/ tree. If nothing landed, probably nothing was useful. When were the most recent patches from gprof or similar? signature.asc Description: OpenPGP digital signature
Re: Too many PMC implementations
On Thu, Aug 23, 2018 at 05:09:35PM +0200, Kamil Rytarowski wrote: > > Observing that all the useful profiling is already done with DTrace, we > can remove complexity from the kernel with negligible cost. I'm not sure what to make of this. I'm trying to come up with a way to make the above statement true, and I'm having some difficulty. You can't possibly mean "Observing that (unproven premise), therefore (conclusion)", so I'll discard that interpretation. Do you perhaps mean "*If* we were to observe that all useful profiling were done with DTrace, *then* we could remove complexity from the kernel with negligible cost"? Because Ragge and others have been pointing out that in that case, the premise "all useful profiling is done with DTrace" does not appear to be true. Profiling kernel code on VAX may not be useful *to you* but that does not imply it is "not useful" simpliciter. -- Thor Lancelot Simon t...@panix.com "Whether or not there's hope for change is not the question. If you want to be a free person, you don't stand up for human rights because it will work, but because it is right." --Andrei Sakharov
Re: Too many PMC implementations
Den 2018-08-23 kl. 17:09, skrev Kamil Rytarowski: On 23.08.2018 16:59, Anders Magnusson wrote: Den 2018-08-23 kl. 16:48, skrev Kamil Rytarowski: On 23.08.2018 16:28, Anders Magnusson wrote: Den 2018-08-23 kl. 15:53, skrev Maxime Villard: Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : On 17.08.2018 17:13, Maxime Villard wrote: Note that I'm talking about the kernel gprof, and not the userland gprof. In terms of kernel profiling, it's not nonsensical to say that since we support ARM and x86 in tprof, we can cover 99% of the MI parts of whatever architecture. From then on, being able to profile the kernel on other architectures has very little interest. Speaking realistically, probably all the recent software-based kernel profiling was done with DTrace. Yes. So I will proceed. Note that the removal of the kernel gprof implies the removal of kgmon. Just checking: How will it work for ports like vax? When searching for bottlenecks I normally use gprof/kgmon. I don't know anything about DTrace, hence the question. -- Ragge There is no support of DTrace for vax and probably there won't be one. Also probably DTrace is not a final solution per se (DTrace is described as step backwards by people such as Brendan Gregg).. but we are working on better toolchain support to open more possibilities such as XRay. Regarding vax there might be bottlenecks in MD code, but DTrace is a decent one for MI code on supported ports. Hm, so this means that we will be without kernel profiling support at all on non-DTrace architectures? I'm not too happy about that by obvious reasons. It do not work to profile code paths on other architectures, since what takes time is very different. And yes, it is not the MD code that is the case, it's the MI code. I may have missed something, but why remove something that works without replacing it with something new? Only have profiling on a few ports do not sound very clever to me. -- Ragge Evaluating this situation we have to be aware that this description could be reversed and there are ports without meaningful (or any) gprof support. Observing that all the useful profiling is already done with DTrace, we can remove complexity from the kernel with negligible cost. This is not true. Things that you will never notice is a problem on x86 may kill a vax, since there is a large speed factor inbetween. This was true many years ago and is still true. Bottom line: I think it is a bad idea to be without kernel profiling code on vax. -- Ragge
Re: Too many PMC implementations
Den 2018-08-23 kl. 17:03, skrev Maxime Villard: Le 23/08/2018 à 16:28, Anders Magnusson a écrit : Den 2018-08-23 kl. 15:53, skrev Maxime Villard: Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : On 17.08.2018 17:13, Maxime Villard wrote: Note that I'm talking about the kernel gprof, and not the userland gprof. In terms of kernel profiling, it's not nonsensical to say that since we support ARM and x86 in tprof, we can cover 99% of the MI parts of whatever architecture. From then on, being able to profile the kernel on other architectures has very little interest. Speaking realistically, probably all the recent software-based kernel profiling was done with DTrace. Yes. So I will proceed. Note that the removal of the kernel gprof implies the removal of kgmon. Just checking: How will it work for ports like vax? When searching for bottlenecks I normally use gprof/kgmon. I don't know anything about DTrace, hence the question. It looks like there will be no replacement. Are you sure this is really kgmon? Because as far as I can tell, in many architectures GPROF is just dead code that either doesn't compile or doesn't have effect (missing opt_gprof.h, but I did add it in February of this year in the MI parts, so it was likely even more broken before). I have used it not long ago for vax. Maybe I did have to do some tweaks, do not remember, but I really want to be able to use kernel profiling on vax. So, I really oppose removing it and leaving vax without any kernel profiling choice. -- Ragge
Re: Too many PMC implementations
On 23.08.2018 16:59, Anders Magnusson wrote: > Den 2018-08-23 kl. 16:48, skrev Kamil Rytarowski: >> On 23.08.2018 16:28, Anders Magnusson wrote: >>> Den 2018-08-23 kl. 15:53, skrev Maxime Villard: Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : > On 17.08.2018 17:13, Maxime Villard wrote: >> Note that I'm talking about the kernel gprof, and not the userland >> gprof. >> In terms of kernel profiling, it's not nonsensical to say that >> since we >> support ARM and x86 in tprof, we can cover 99% of the MI parts of >> whatever architecture. From then on, being able to profile the >> kernel on >> other architectures has very little interest. > Speaking realistically, probably all the recent software-based kernel > profiling was done with DTrace. Yes. So I will proceed. Note that the removal of the kernel gprof implies the removal of kgmon. >>> Just checking: How will it work for ports like vax? >>> When searching for bottlenecks I normally use gprof/kgmon. I don't know >>> anything about DTrace, hence the question. >>> >>> -- Ragge >> There is no support of DTrace for vax and probably there won't be one. >> Also probably DTrace is not a final solution per se (DTrace is described >> as step backwards by people such as Brendan Gregg).. but we are working >> on better toolchain support to open more possibilities such as XRay. >> >> Regarding vax there might be bottlenecks in MD code, but DTrace is a >> decent one for MI code on supported ports. > Hm, so this means that we will be without kernel profiling support at > all on non-DTrace architectures? > I'm not too happy about that by obvious reasons. > > It do not work to profile code paths on other architectures, since what > takes time is very different. > And yes, it is not the MD code that is the case, it's the MI code. > > I may have missed something, but why remove something that works without > replacing it with something new? > Only have profiling on a few ports do not sound very clever to me. > > -- Ragge > > Evaluating this situation we have to be aware that this description could be reversed and there are ports without meaningful (or any) gprof support. Observing that all the useful profiling is already done with DTrace, we can remove complexity from the kernel with negligible cost. signature.asc Description: OpenPGP digital signature
Re: Too many PMC implementations
Le 23/08/2018 à 16:28, Anders Magnusson a écrit : Den 2018-08-23 kl. 15:53, skrev Maxime Villard: Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : On 17.08.2018 17:13, Maxime Villard wrote: Note that I'm talking about the kernel gprof, and not the userland gprof. In terms of kernel profiling, it's not nonsensical to say that since we support ARM and x86 in tprof, we can cover 99% of the MI parts of whatever architecture. From then on, being able to profile the kernel on other architectures has very little interest. Speaking realistically, probably all the recent software-based kernel profiling was done with DTrace. Yes. So I will proceed. Note that the removal of the kernel gprof implies the removal of kgmon. Just checking: How will it work for ports like vax? When searching for bottlenecks I normally use gprof/kgmon. I don't know anything about DTrace, hence the question. It looks like there will be no replacement. Are you sure this is really kgmon? Because as far as I can tell, in many architectures GPROF is just dead code that either doesn't compile or doesn't have effect (missing opt_gprof.h, but I did add it in February of this year in the MI parts, so it was likely even more broken before).
Re: Too many PMC implementations
Den 2018-08-23 kl. 16:48, skrev Kamil Rytarowski: On 23.08.2018 16:28, Anders Magnusson wrote: Den 2018-08-23 kl. 15:53, skrev Maxime Villard: Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : On 17.08.2018 17:13, Maxime Villard wrote: Note that I'm talking about the kernel gprof, and not the userland gprof. In terms of kernel profiling, it's not nonsensical to say that since we support ARM and x86 in tprof, we can cover 99% of the MI parts of whatever architecture. From then on, being able to profile the kernel on other architectures has very little interest. Speaking realistically, probably all the recent software-based kernel profiling was done with DTrace. Yes. So I will proceed. Note that the removal of the kernel gprof implies the removal of kgmon. Just checking: How will it work for ports like vax? When searching for bottlenecks I normally use gprof/kgmon. I don't know anything about DTrace, hence the question. -- Ragge There is no support of DTrace for vax and probably there won't be one. Also probably DTrace is not a final solution per se (DTrace is described as step backwards by people such as Brendan Gregg).. but we are working on better toolchain support to open more possibilities such as XRay. Regarding vax there might be bottlenecks in MD code, but DTrace is a decent one for MI code on supported ports. Hm, so this means that we will be without kernel profiling support at all on non-DTrace architectures? I'm not too happy about that by obvious reasons. It do not work to profile code paths on other architectures, since what takes time is very different. And yes, it is not the MD code that is the case, it's the MI code. I may have missed something, but why remove something that works without replacing it with something new? Only have profiling on a few ports do not sound very clever to me. -- Ragge
Re: Too many PMC implementations
On 23.08.2018 16:28, Anders Magnusson wrote: > Den 2018-08-23 kl. 15:53, skrev Maxime Villard: >> Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : >>> On 17.08.2018 17:13, Maxime Villard wrote: Note that I'm talking about the kernel gprof, and not the userland gprof. In terms of kernel profiling, it's not nonsensical to say that since we support ARM and x86 in tprof, we can cover 99% of the MI parts of whatever architecture. From then on, being able to profile the kernel on other architectures has very little interest. >>> >>> Speaking realistically, probably all the recent software-based kernel >>> profiling was done with DTrace. >> >> Yes. So I will proceed. >> >> Note that the removal of the kernel gprof implies the removal of kgmon. > Just checking: How will it work for ports like vax? > When searching for bottlenecks I normally use gprof/kgmon. I don't know > anything about DTrace, hence the question. > > -- Ragge There is no support of DTrace for vax and probably there won't be one. Also probably DTrace is not a final solution per se (DTrace is described as step backwards by people such as Brendan Gregg).. but we are working on better toolchain support to open more possibilities such as XRay. Regarding vax there might be bottlenecks in MD code, but DTrace is a decent one for MI code on supported ports. signature.asc Description: OpenPGP digital signature
Re: Too many PMC implementations
Den 2018-08-23 kl. 15:53, skrev Maxime Villard: Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : On 17.08.2018 17:13, Maxime Villard wrote: Note that I'm talking about the kernel gprof, and not the userland gprof. In terms of kernel profiling, it's not nonsensical to say that since we support ARM and x86 in tprof, we can cover 99% of the MI parts of whatever architecture. From then on, being able to profile the kernel on other architectures has very little interest. Speaking realistically, probably all the recent software-based kernel profiling was done with DTrace. Yes. So I will proceed. Note that the removal of the kernel gprof implies the removal of kgmon. Just checking: How will it work for ports like vax? When searching for bottlenecks I normally use gprof/kgmon. I don't know anything about DTrace, hence the question. -- Ragge
Re: Too many PMC implementations
Le 17/08/2018 à 17:42, Kamil Rytarowski a écrit : On 17.08.2018 17:13, Maxime Villard wrote: Note that I'm talking about the kernel gprof, and not the userland gprof. In terms of kernel profiling, it's not nonsensical to say that since we support ARM and x86 in tprof, we can cover 99% of the MI parts of whatever architecture. From then on, being able to profile the kernel on other architectures has very little interest. Speaking realistically, probably all the recent software-based kernel profiling was done with DTrace. Yes. So I will proceed. Note that the removal of the kernel gprof implies the removal of kgmon.
