Re: perf & rasd integration plan
On Mon, Oct 06, 2014 at 08:53:51AM +0200, Jean Pihet wrote: > Hi Jiri, > > > On 5 October 2014 20:24, Jiri Olsa wrote: > > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > >> Top-posting on purpose: > >> > >> Btw, jolsa, if you get your LCE proposal for the perf splitting > >> approved, please post the time here so people can come. > > > > yep, it got accepted, ther schedule is: > > Friday, October 17, 2014 from 9:00am – Noon > > Room 2 > > > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 > Great! Are there slides that we can download? there'll be just one.. I'm on vacation this week, so I'll try send some till end of the week jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Tue, Oct 07, 2014 at 04:02:29PM +0200, Borislav Petkov escreveu: > On Tue, Oct 07, 2014 at 10:55:31AM -0300, Arnaldo Carvalho de Melo wrote: > > So you say that if we change a method, say, perf_evlist__open() to > > accept a new argument, or change the type of one of its arguments, > > with a good reason, it is acceptable and with just a change in the > > README explaining why the build fails, which helps in finding how to fix > > it in a particular project using these "source libraries" then it would > > be ok? > Right, exactly. > Updating the perf bits used in another project will require a > recompilation anyway so adjusting stuff here and there should not be of > issue. > > That would be fine with me. > Yeah. > > We should take care to make sure that the build _fails_ in such cases, > > i.e.avoid changing the semantic of the Nth argument but keeping its > > type kind of changes. > Yep. > > At some point there will be no more reasons to change things, and that > > will be noticed by how long since the last change was made to a > > particular class, at that point we may well think about making library > > type promises. > I was about to say the same thing - once the code doesn't change anymore > and gets even boring, doing the library promise is probably pretty easy. Ok, at some point we need to make sure this gets into tools/lib/api/README. > See, no need for you to get over to LCE: we're practically done :-) Nah, there is much more than this at LCE, I'm sure, I'll try again next time. > We'll run the proposal in jolsa's timeslot again and make sure to poke > holes in it. But it looks good from where I'm standing. Thanks! > Btw, did we just solve the compatibility problem of libraries by > delivering source libraries first!?! For me one of the points of having tools/ is to try and make kernel developers try to do userspace muck, dirtying the hands trying to use the same discipline used in the kernel proper, using code as unchanged as possible from the kernel proper, etc. If we now are moving to share code in a way that keeps what is nice in having code and its users in the same repo, like the kernel guys have with subsystems and its drivers while allowing to do it in different repos, that is a good thing indeed. > Haha. While having fun even! Good deal. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Tue, Oct 07, 2014 at 10:55:31AM -0300, Arnaldo Carvalho de Melo wrote: > So you say that if we change a method, say, perf_evlist__open() to > accept a new argument, or change the type of one of its arguments, > with a good reason, it is acceptable and with just a change in the > README explaining why the build fails, which helps in finding how to fix > it in a particular project using these "source libraries" then it would > be ok? Right, exactly. Updating the perf bits used in another project will require a recompilation anyway so adjusting stuff here and there should not be of issue. > That would be fine with me. Yeah. > We should take care to make sure that the build _fails_ in such cases, > i.e.avoid changing the semantic of the Nth argument but keeping its > type kind of changes. Yep. > At some point there will be no more reasons to change things, and that > will be noticed by how long since the last change was made to a > particular class, at that point we may well think about making library > type promises. I was about to say the same thing - once the code doesn't change anymore and gets even boring, doing the library promise is probably pretty easy. See, no need for you to get over to LCE: we're practically done :-) We'll run the proposal in jolsa's timeslot again and make sure to poke holes in it. But it looks good from where I'm standing. Btw, did we just solve the compatibility problem of libraries by delivering source libraries first!?! Haha. -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Tue, Oct 07, 2014 at 03:49:12PM +0200, Borislav Petkov escreveu: > On Tue, Oct 07, 2014 at 10:40:21AM -0300, Arnaldo Carvalho de Melo wrote: > > Right, we now should always think that the great advantage of having > > this code and its users in the same repo tree is going away, i.e. we > > will not be able to improve the code by fixing mistakes that require > > changing its users. > > Why? I mean, in the rasd case, changing function calling conventions as > part of updating the perf libraries is no act. I mean, we won't link > against a shared lib but build a static executable and thus the perf src > lib (let's call it that - a source library) we're using is a don't care. So you say that if we change a method, say, perf_evlist__open() to accept a new argument, or change the type of one of its arguments, with a good reason, it is acceptable and with just a change in the README explaining why the build fails, which helps in finding how to fix it in a particular project using these "source libraries" then it would be ok? That would be fine with me. We should take care to make sure that the build _fails_ in such cases, i.e.avoid changing the semantic of the Nth argument but keeping its type kind of changes. At some point there will be no more reasons to change things, and that will be noticed by how long since the last change was made to a particular class, at that point we may well think about making library type promises. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Tue, Oct 07, 2014 at 10:40:21AM -0300, Arnaldo Carvalho de Melo wrote: > Right, we now should always think that the great advantage of having > this code and its users in the same repo tree is going away, i.e. we > will not be able to improve the code by fixing mistakes that require > changing its users. Why? I mean, in the rasd case, changing function calling conventions as part of updating the perf libraries is no act. I mean, we won't link against a shared lib but build a static executable and thus the perf src lib (let's call it that - a source library) we're using is a don't care. Thanks. -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Tue, Oct 07, 2014 at 01:23:42PM +0200, Borislav Petkov escreveu: > On Mon, Oct 06, 2014 at 06:22:43PM -0300, Arnaldo Carvalho de Melo wrote: > > My preference would be for single object files, but the pressure to have > > a written in stone library seems to just build up... > Yeah, I think if we have an alternative way to easily integrate that > functionality into external projects, we can avoid the need for a lib. > ... > > After doing a 'make perf-targz-src-pkg' > > I.e. no kernel sources involved on the machines where I build test. > > IOW, it is untangled from the kernel sources. As tools/lib/api/ should > > as well. > Ok, that actually sounds like something we should try: > $ make perf-api-src-pkg > which would spit a tgz with all the generic stuff perf carries. People > would simply unpack it in their projects and build it. > That would require somewhat stable function names so that updating is > relatively painless and if visible functions change, some sort of a Right, we now should always think that the great advantage of having this code and its users in the same repo tree is going away, i.e. we will not be able to improve the code by fixing mistakes that require changing its users. So we should export the bare minimum that tools advocate exporting, like you guys are doing now by providing the code that shows how you use it. > README file should say what and why. Well, at that point we can't do that anymore, we will need instead to add new methods while keeping the existing ones in place and with the same semantics, at most for something really annoying to maintain we will have to add annoying warnings telling that in a year or two something will go away, it was deprecated, annoy the maintainer of your tool to update their project. > Sounds like a good alternative to a lib to me... Yes, should be interesting to experiment with this, at the very least it can be considered a step into the direction of a full blown library, if we ever feel that it is strictly needed. > > Well, the rasd experience is serving to show areas where there is > > unnecessary entanglement (hists inside perf_event, etc, the ifdefs you > > put in place). > Yep. > > I'm working to remove the ones that are in rasd.c, aiming to have a > > tools/lib/api/ tree that can be used to build rasd and tools/perf/. > > What I don't want to do is to simply straight more s/more/move/g > > tools/perf/util/evlist.c to tools/lib/api/perf/, some untanglement work > > is needed. > Yep. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Mon, Oct 06, 2014 at 06:22:43PM -0300, Arnaldo Carvalho de Melo wrote: > My preference would be for single object files, but the pressure to have > a written in stone library seems to just build up... Yeah, I think if we have an alternative way to easily integrate that functionality into external projects, we can avoid the need for a lib. ... > After doing a 'make perf-targz-src-pkg' > > I.e. no kernel sources involved on the machines where I build test. > > IOW, it is untangled from the kernel sources. As tools/lib/api/ should > as well. Ok, that actually sounds like something we should try: $ make perf-api-src-pkg which would spit a tgz with all the generic stuff perf carries. People would simply unpack it in their projects and build it. That would require somewhat stable function names so that updating is relatively painless and if visible functions change, some sort of a README file should say what and why. Sounds like a good alternative to a lib to me... > Well, the rasd experience is serving to show areas where there is > unnecessary entanglement (hists inside perf_event, etc, the ifdefs you > put in place). Yep. > I'm working to remove the ones that are in rasd.c, aiming to have a > tools/lib/api/ tree that can be used to build rasd and tools/perf/. > > What I don't want to do is to simply straight more > tools/perf/util/evlist.c to tools/lib/api/perf/, some untanglement work > is needed. Yep. Thanks acme! -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Mon, Oct 06, 2014 at 09:53:49PM +0200, Borislav Petkov escreveu:
> On Mon, Oct 06, 2014 at 04:12:27PM -0300, Arnaldo Carvalho de Melo wrote:
> > Right, and it is being a great exercise, thanks for the patience so far
> > ;-)
>
> I know, this is your secret agenda to keep me at bay working on this! :-P
>
> > Looking at those ifdefs we see things that are specific to tools/perf/,
> > like perf_evsel having a struct hists embedded... I.e. that is of no
> > interest whatsoever (so far) to rasd, and in turn pulls other object
> > files.
> >
> > So I think that right now we need to look at those ifdefs and go on
> > making what is in tools/perf/util/ stop using it somehow, so that what
> > then gets moved to tools/lib/api/perf/ (I guess this is where it should
> > be, opinions?) have that sorted out.
> > I.e. what goes to tools/lib/api/perf/ is what is common to the needs of
> > tools/perf/ and rasd (wherever it may roam).
> > Perhaps something like I did for sock/inet_sock/inet_connection_sock
> > ages ago... I.e. the tools that want a hists will have a
> > struct hists_evsel {
> > struct perf_evsel evsel;
> > struct hists hists;
> > };
> > etc.
> > Experimenting with that, as it is the only thing ifdefed out in rasd's
> > copy of evsel.h...
> Right, this all reads like it is going in the right direction but the
> more important question IMO is are we doing a libperf or are we still
> doing tools/lib/api/perf/ of single object files to which people can
> link?
My preference would be for single object files, but the pressure to have
a written in stone library seems to just build up...
> Because if it is the second - and it sounds to me like it is - how do
> you propose we link with external projects? IOW, if I want to have rasd
> build *without* the kernel tree, is a simple
> cp -r tools/lib/api/ /path/to/perf/lib/
> work?
It should, no matter if we end up with a library.
What I'm doing these days to test if perf builds on multiple distros is:
#!/bin/sh
remotebuild() {
target=$1
perfpkg=$2
cmd="rm -rf ${perfpkg} ; tar xf ${perfpkg}.tar.gz && cd ${perfpkg} &&
rm -rf /tmp/build/perf ; mkdir -p /tmp/build/perf && make -C tools/perf
O=/tmp/build/perf install && make -C tools/perf build-test"
scp ${perfpkg}.tar.gz gita@${target}:. && \
ssh gita@${target} "${cmd}"
}
targets="fedora14 opensuse12.3 fedora19 ubuntu13.10"
perfpkg=perf-3.17.0-rc4
for target in ${targets} ; do
remotebuild ${target} ${perfpkg}
done
After doing a 'make perf-targz-src-pkg'
I.e. no kernel sources involved on the machines where I build test.
IOW, it is untangled from the kernel sources. As tools/lib/api/ should
as well.
> I mean, I don't know and I'm just asking. Is that the proposed way? Are
> we fine with that? Do we want something different, maybe even a lib? Is
> it time for a lib even?
Well, the rasd experience is serving to show areas where there is
unnecessary entanglement (hists inside perf_event, etc, the ifdefs you
put in place).
I'm working to remove the ones that are in rasd.c, aiming to have a
tools/lib/api/ tree that can be used to build rasd and tools/perf/.
What I don't want to do is to simply straight more
tools/perf/util/evlist.c to tools/lib/api/perf/, some untanglement work
is needed.
> Because the whole perf functionality is being cravingly ogled by other
> users - I know Matt wants it for cache QoS or whatever it is called and
> there are probably a whole lot of projects which would like to process
> events programatically in userspace.
>
> So what I'm saying is, we probably should have a nice clean way
> to be able to share that code with external projects instead of
> external projects duplicating and building a whole library around
> perf_event_open() and the likes...
Agreed, hopefully we'll get that, finally.
> Just a couple of thoughts...
>
> --
> Regards/Gruss,
> Boris.
>
> Sent from a fat crate under my desk. Formatting is fine.
> --
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Mon, Oct 06, 2014 at 04:12:27PM -0300, Arnaldo Carvalho de Melo wrote:
> Right, and it is being a great exercise, thanks for the patience so far
> ;-)
I know, this is your secret agenda to keep me at bay working on this! :-P
> Looking at those ifdefs we see things that are specific to tools/perf/,
> like perf_evsel having a struct hists embedded... I.e. that is of no
> interest whatsoever (so far) to rasd, and in turn pulls other object
> files.
>
> So I think that right now we need to look at those ifdefs and go on
> making what is in tools/perf/util/ stop using it somehow, so that what
> then gets moved to tools/lib/api/perf/ (I guess this is where it should
> be, opinions?) have that sorted out.
>
> I.e. what goes to tools/lib/api/perf/ is what is common to the needs of
> tools/perf/ and rasd (wherever it may roam).
>
> Perhaps something like I did for sock/inet_sock/inet_connection_sock
> ages ago... I.e. the tools that want a hists will have a
>
> struct hists_evsel {
> struct perf_evsel evsel;
> struct hists hists;
> };
>
> etc.
>
> Experimenting with that, as it is the only thing ifdefed out in rasd's
> copy of evsel.h...
Right, this all reads like it is going in the right direction but the
more important question IMO is are we doing a libperf or are we still
doing tools/lib/api/perf/ of single object files to which people can
link?
Because if it is the second - and it sounds to me like it is - how do
you propose we link with external projects? IOW, if I want to have rasd
build *without* the kernel tree, is a simple
cp -r tools/lib/api/ /path/to/perf/lib/
work?
I mean, I don't know and I'm just asking. Is that the proposed way? Are
we fine with that? Do we want something different, maybe even a lib? Is
it time for a lib even?
Because the whole perf functionality is being cravingly ogled by other
users - I know Matt wants it for cache QoS or whatever it is called and
there are probably a whole lot of projects which would like to process
events programatically in userspace.
So what I'm saying is, we probably should have a nice clean way
to be able to share that code with external projects instead of
external projects duplicating and building a whole library around
perf_event_open() and the likes...
Just a couple of thoughts...
--
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Mon, Oct 06, 2014 at 05:16:06PM +0200, Borislav Petkov escreveu:
> On Mon, Oct 06, 2014 at 12:07:41PM -0300, Arnaldo Carvalho de Melo wrote:
> > So, why have you commented out the perf_missing_features fallbacks? Are
> > they getting in the way somehow, what can we do upstream, i.e. in
> > tools/perf/util/ so that it gets closer to what you want to have?
> >
> > Ditto for all the other ifdef'ed code, can you elaborate on why each is
> > needed? That will help in moving what is used by rasd from
> > tools/perf/util/ to tools/perf/lib/ so that we can eliminate the
> > duplicity.
>
> Oh, the reason is very simple: so that it builds. We don't need those
> facilities yet. We will enable them when needed but we wanted to
> concentrate on rasd first and later clean up stuff as we go.
Right, and it is being a great exercise, thanks for the patience so far
;-)
Looking at those ifdefs we see things that are specific to tools/perf/,
like perf_evsel having a struct hists embedded... I.e. that is of no
interest whatsoever (so far) to rasd, and in turn pulls other object
files.
So I think that right now we need to look at those ifdefs and go on
making what is in tools/perf/util/ stop using it somehow, so that what
then gets moved to tools/lib/api/perf/ (I guess this is where it should
be, opinions?) have that sorted out.
I.e. what goes to tools/lib/api/perf/ is what is common to the needs of
tools/perf/ and rasd (wherever it may roam).
Perhaps something like I did for sock/inet_sock/inet_connection_sock
ages ago... I.e. the tools that want a hists will have a
struct hists_evsel {
struct perf_evsel evsel;
struct hists hists;
};
etc.
Experimenting with that, as it is the only thing ifdefed out in rasd's
copy of evsel.h...
- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Mon, Oct 06, 2014 at 12:07:41PM -0300, Arnaldo Carvalho de Melo wrote: > So, why have you commented out the perf_missing_features fallbacks? Are > they getting in the way somehow, what can we do upstream, i.e. in > tools/perf/util/ so that it gets closer to what you want to have? > > Ditto for all the other ifdef'ed code, can you elaborate on why each is > needed? That will help in moving what is used by rasd from > tools/perf/util/ to tools/perf/lib/ so that we can eliminate the > duplicity. Oh, the reason is very simple: so that it builds. We don't need those facilities yet. We will enable them when needed but we wanted to concentrate on rasd first and later clean up stuff as we go. -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Mon, Oct 06, 2014 at 12:08:38PM -0300, Arnaldo Carvalho de Melo wrote: > Em Mon, Oct 06, 2014 at 05:01:47PM +0200, Borislav Petkov escreveu: > > On Mon, Oct 06, 2014 at 11:58:56AM -0300, Arnaldo Carvalho de Melo wrote: > > > Agreed, we can use rasd as a reference, I'm now looking for the git repo > > > url to see what it is using to prototype how I think this should be > > > done. > > > > https://git.kernel.org/cgit/utils/ras/rasd.git > > I got Jean's "development" repo, should I get the g.k.o one? Right, this one above is the official rasd one and Jean's stuff goes ontop. So you can look at both. :-) -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Mon, Oct 06, 2014 at 05:01:47PM +0200, Borislav Petkov escreveu: > On Mon, Oct 06, 2014 at 11:58:56AM -0300, Arnaldo Carvalho de Melo wrote: > > Agreed, we can use rasd as a reference, I'm now looking for the git repo > > url to see what it is using to prototype how I think this should be > > done. > > https://git.kernel.org/cgit/utils/ras/rasd.git I got Jean's "development" repo, should I get the g.k.o one? - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > The RAS Deamon (rasd) as available at [1] and the development version > at [2], allows to enable kernel tracepoints and outputs the > tracepoints fields according to the kernel format. > > [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ > [2] https://git.linaro.org/people/jean.pihet/rasd.git > > rasd currently is a duplicate of the perf tool code, ultimately perf > and rasd will use the same common code. The goal is to factor out the > common code from perf and place it in small librairies in tools/lib. > > Here is the code that rasd currently uses and that should be moved to > tools/lib: > > - debugfs: already in tools/lib/api/fs > . mount and retrieve path > > - evlist: tools/perf/util/evlist.[ch] > . create and init new evlist, > . set cpu and thread maps, > . add events to evlist, > . init and use internal event id, > . alloc and mmap events buffers, manage file descriptors, > . enable events, > . read events buffers, parse data, > . unmap and free buffers > > - evsel: tools/perf/util/evsel.[ch] > . create and init new tracepoints events, > . init and use internal event id, > . open events, manage fds, > . close and free events So, why have you commented out the perf_missing_features fallbacks? Are they getting in the way somehow, what can we do upstream, i.e. in tools/perf/util/ so that it gets closer to what you want to have? Ditto for all the other ifdef'ed code, can you elaborate on why each is needed? That will help in moving what is used by rasd from tools/perf/util/ to tools/perf/lib/ so that we can eliminate the duplicity. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Mon, Oct 06, 2014 at 11:58:56AM -0300, Arnaldo Carvalho de Melo wrote: > Agreed, we can use rasd as a reference, I'm now looking for the git repo > url to see what it is using to prototype how I think this should be > done. https://git.kernel.org/cgit/utils/ras/rasd.git -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Hi Arnaldo, On 6 October 2014 16:58, Arnaldo Carvalho de Melo wrote: > Em Mon, Oct 06, 2014 at 11:07:26AM +0200, Robert Richter escreveu: >> On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: >> > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: >> > > The plan is to move the small and generic functions first: util, >> > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, >> > > trace-event-parse; and finally integrate rasd into the tools/ dir. > >> > > Any thought? Can evlist, evsel etc. be moved at once? > >> > > Patches should come soon, when time allows. > >> > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just >> > go on and add tools/perf/util/evlist.o et all to be linked directly, as >> > a first step. > >> > Then, as a second step, we can create a tools/lib/perf/evlist.c having >> > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is >> > proven to be useful for something other than perf. > >> It would be good to have tools/lib/perf or so with some base >> implementation to setup and connect to perf buffers. This is useful >> for tools not only in tools/. The rasd would be a good reference for > > Agreed, we can use rasd as a reference, I'm now looking for the git repo > url to see what it is using to prototype how I think this should be > done. The latest patches have been submitted to linux-edac [1] and also are available in the dev git tree [2]. [1] http://marc.info/?l=linux-edac&m=141234802311271&w=2 [2] https://git.linaro.org/people/jean.pihet/rasd.git Thx, Jean > > - Arnaldo > >> this regardless if it is in tools/ or not. I am not sure whether and >> when rasd will be moved there. > > - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Mon, Oct 06, 2014 at 11:07:26AM +0200, Robert Richter escreveu: > On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: > > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > > > The plan is to move the small and generic functions first: util, > > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > > > trace-event-parse; and finally integrate rasd into the tools/ dir. > > > Any thought? Can evlist, evsel etc. be moved at once? > > > Patches should come soon, when time allows. > > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just > > go on and add tools/perf/util/evlist.o et all to be linked directly, as > > a first step. > > Then, as a second step, we can create a tools/lib/perf/evlist.c having > > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is > > proven to be useful for something other than perf. > It would be good to have tools/lib/perf or so with some base > implementation to setup and connect to perf buffers. This is useful > for tools not only in tools/. The rasd would be a good reference for Agreed, we can use rasd as a reference, I'm now looking for the git repo url to see what it is using to prototype how I think this should be done. - Arnaldo > this regardless if it is in tools/ or not. I am not sure whether and > when rasd will be moved there. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Hi, On 6 October 2014 11:07, Robert Richter wrote: > On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: >> Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: >> > The plan is to move the small and generic functions first: util, >> > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, >> > trace-event-parse; and finally integrate rasd into the tools/ dir. >> > >> > Any thought? Can evlist, evsel etc. be moved at once? >> > >> > Patches should come soon, when time allows. >> >> Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just >> go on and add tools/perf/util/evlist.o et all to be linked directly, as >> a first step. >> >> Then, as a second step, we can create a tools/lib/perf/evlist.c having >> what is currently used by both tools/perf/ and tools/rasd/, i.e. what is >> proven to be useful for something other than perf. > > It would be good to have tools/lib/perf or so with some base > implementation to setup and connect to perf buffers. This is useful > for tools not only in tools/. The rasd would be a good reference for > this regardless if it is in tools/ or not. I am not sure whether and > when rasd will be moved there. Agree. Having a lib (or a set of libs) that external tools can use is a must. This will encourage tools developers to use this API instead of re-invening the wheel every time, and to provide standard tools that can be merged in the kernel source if needed. > >> As the need arises, we go on moving things into tools/lib/perf/evlist.c >> et all from wherever it appeared first, be it from tools/rasd/, >> tools/perf/util/evlist.c or anywhere else. >> >> Initial rule being that once it is used by multiple tools living in >> tools/, then it deserves a place in tools/lib/perf/. > > So this wouldn't quite work well as it excludes tools not in tools/. Ditto I am willing to propose a first set of patches to factor out the common code from perf, but an agreement is needed on the direction to take. My plan is to provide patches for the proposed integration plan (as in the initial e-mail), is it worth doing so or is it purely wasted time and effort? What do you think? Thx, Jean > > -Robert > >> >> Ditto for other stuff currently living in tools/perf/util/. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On 30.09.14 10:24:16, Arnaldo Carvalho de Melo wrote: > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > > The plan is to move the small and generic functions first: util, > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > > trace-event-parse; and finally integrate rasd into the tools/ dir. > > > > Any thought? Can evlist, evsel etc. be moved at once? > > > > Patches should come soon, when time allows. > > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just > go on and add tools/perf/util/evlist.o et all to be linked directly, as > a first step. > > Then, as a second step, we can create a tools/lib/perf/evlist.c having > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is > proven to be useful for something other than perf. It would be good to have tools/lib/perf or so with some base implementation to setup and connect to perf buffers. This is useful for tools not only in tools/. The rasd would be a good reference for this regardless if it is in tools/ or not. I am not sure whether and when rasd will be moved there. > As the need arises, we go on moving things into tools/lib/perf/evlist.c > et all from wherever it appeared first, be it from tools/rasd/, > tools/perf/util/evlist.c or anywhere else. > > Initial rule being that once it is used by multiple tools living in > tools/, then it deserves a place in tools/lib/perf/. So this wouldn't quite work well as it excludes tools not in tools/. -Robert > > Ditto for other stuff currently living in tools/perf/util/. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Hi Jiri, On 5 October 2014 20:24, Jiri Olsa wrote: > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: >> Top-posting on purpose: >> >> Btw, jolsa, if you get your LCE proposal for the perf splitting >> approved, please post the time here so people can come. > > yep, it got accepted, ther schedule is: > Friday, October 17, 2014 from 9:00am – Noon > Room 2 > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 Great! Are there slides that we can download? I have a couple of weeks to work on this, I can prepare the first set of patches. Note: I will not be at ELCE/LPC but could attend remotely or have a call to discuss the points. Cheers, Jean > > jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Sun, Oct 05, 2014 at 02:24:01PM -0500, Chuck Ebbert wrote: > On Sun, 5 Oct 2014 20:24:42 +0200 > Jiri Olsa wrote: > > > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > > > Top-posting on purpose: > > > > > > Btw, jolsa, if you get your LCE proposal for the perf splitting > > > approved, please post the time here so people can come. > > > > yep, it got accepted, ther schedule is: > > Friday, October 17, 2014 from 9:00am – Noon > > Room 2 > > > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 > > > > The SSL cert for that site is comic. > > 1. Expired over three years ago. > > 2. Was issued to a different domain. > > 3. Issued by an un-verifiable CA. > > This is a joke, right? ;-) it's the link from this page: http://www.linuxplumbersconf.org/2014/ocw/events/LPC2014/schedule jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Sun, 5 Oct 2014 20:24:42 +0200 Jiri Olsa wrote: > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > > Top-posting on purpose: > > > > Btw, jolsa, if you get your LCE proposal for the perf splitting > > approved, please post the time here so people can come. > > yep, it got accepted, ther schedule is: > Friday, October 17, 2014 from 9:00am – Noon > Room 2 > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 > The SSL cert for that site is comic. 1. Expired over three years ago. 2. Was issued to a different domain. 3. Issued by an un-verifiable CA. This is a joke, right? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Sun, Oct 05, 2014 at 08:24:42PM +0200, Jiri Olsa wrote: > On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > > Top-posting on purpose: > > > > Btw, jolsa, if you get your LCE proposal for the perf splitting > > approved, please post the time here so people can come. > > yep, it got accepted, ther schedule is: > Friday, October 17, 2014 from 9:00am – Noon > Room 2 > > https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 Cool, thanks. I'll add it to the calendar - lot to do this conf :-) -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
On Sun, Oct 05, 2014 at 07:48:01PM +0200, Borislav Petkov wrote: > Top-posting on purpose: > > Btw, jolsa, if you get your LCE proposal for the perf splitting > approved, please post the time here so people can come. yep, it got accepted, ther schedule is: Friday, October 17, 2014 from 9:00am – Noon Room 2 https://pdxplumbers.osuosl.org/2014/ocw/events/LPC2014/tracks/351 jirka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Top-posting on purpose: Btw, jolsa, if you get your LCE proposal for the perf splitting approved, please post the time here so people can come. Thanks. On Tue, Sep 30, 2014 at 10:24:16AM -0300, Arnaldo Carvalho de Melo wrote: > Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > > The RAS Deamon (rasd) as available at [1] and the development version > > at [2], allows to enable kernel tracepoints and outputs the > > tracepoints fields according to the kernel format. > > > > [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ > > [2] https://git.linaro.org/people/jean.pihet/rasd.git > > > > rasd currently is a duplicate of the perf tool code, ultimately perf > > and rasd will use the same common code. The goal is to factor out the > > common code from perf and place it in small librairies in tools/lib. > > > > Here is the code that rasd currently uses and that should be moved to > > tools/lib: > > > > - debugfs: already in tools/lib/api/fs > > . mount and retrieve path > > > > - evlist: tools/perf/util/evlist.[ch] > > . create and init new evlist, > > . set cpu and thread maps, > > . add events to evlist, > > . init and use internal event id, > > . alloc and mmap events buffers, manage file descriptors, > > . enable events, > > . read events buffers, parse data, > > . unmap and free buffers > > > > - evsel: tools/perf/util/evsel.[ch] > > . create and init new tracepoints events, > > . init and use internal event id, > > . open events, manage fds, > > . close and free events > > > > - trace-event: tools/perf/util/trace-event.[ch] and > > tools/perf/util/trace-event-parse.c > > . retrieve and parse events format (using event-parse), > > . print out events fields > > > > - event-parse: already in tools/lib/traceevent/event-parse.[ch] > > . retrieve and parse events format, > > . parse events format and print out events fields > > > > - trace-seq: already in tools/lib/traceevent/trace-seq.c > > . format output string for event fields > > > > - events plugins: already in tools/lib/traceevent/event-plugin.c > > > > - util: tools/perf/util/util.[ch] > > . files open/read, > > . manage events attributes, > > . various macros > > > > - test events attributes: tools/perf/tests/attr.c > > . test_attr__open() > > > > - thread: tools/perf/util/thread_map.[ch] and > > - cpu: tools/perf/util/cpumap.[ch] > > . init and manage process maps > > > > - xyarray: tools/perf/util/xyarray.[ch] > > > > - syscall: tools/perf/perf-sys.h > > > > - cgroup: tools/perf/util/cgroup.[ch] > > > > The plan is to move the small and generic functions first: util, > > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > > trace-event-parse; and finally integrate rasd into the tools/ dir. > > > > Any thought? Can evlist, evsel etc. be moved at once? > > > > Patches should come soon, when time allows. > > Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just > go on and add tools/perf/util/evlist.o et all to be linked directly, as > a first step. > > Then, as a second step, we can create a tools/lib/perf/evlist.c having > what is currently used by both tools/perf/ and tools/rasd/, i.e. what is > proven to be useful for something other than perf. > > As the need arises, we go on moving things into tools/lib/perf/evlist.c > et all from wherever it appeared first, be it from tools/rasd/, > tools/perf/util/evlist.c or anywhere else. > > Initial rule being that once it is used by multiple tools living in > tools/, then it deserves a place in tools/lib/perf/. > > Ditto for other stuff currently living in tools/perf/util/. > > - Arnaldo > -- Regards/Gruss, Boris. Sent from a fat crate under my desk. Formatting is fine. -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: perf & rasd integration plan
Em Tue, Sep 30, 2014 at 11:06:21AM +0200, Jean Pihet escreveu: > The RAS Deamon (rasd) as available at [1] and the development version > at [2], allows to enable kernel tracepoints and outputs the > tracepoints fields according to the kernel format. > > [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ > [2] https://git.linaro.org/people/jean.pihet/rasd.git > > rasd currently is a duplicate of the perf tool code, ultimately perf > and rasd will use the same common code. The goal is to factor out the > common code from perf and place it in small librairies in tools/lib. > > Here is the code that rasd currently uses and that should be moved to > tools/lib: > > - debugfs: already in tools/lib/api/fs > . mount and retrieve path > > - evlist: tools/perf/util/evlist.[ch] > . create and init new evlist, > . set cpu and thread maps, > . add events to evlist, > . init and use internal event id, > . alloc and mmap events buffers, manage file descriptors, > . enable events, > . read events buffers, parse data, > . unmap and free buffers > > - evsel: tools/perf/util/evsel.[ch] > . create and init new tracepoints events, > . init and use internal event id, > . open events, manage fds, > . close and free events > > - trace-event: tools/perf/util/trace-event.[ch] and > tools/perf/util/trace-event-parse.c > . retrieve and parse events format (using event-parse), > . print out events fields > > - event-parse: already in tools/lib/traceevent/event-parse.[ch] > . retrieve and parse events format, > . parse events format and print out events fields > > - trace-seq: already in tools/lib/traceevent/trace-seq.c > . format output string for event fields > > - events plugins: already in tools/lib/traceevent/event-plugin.c > > - util: tools/perf/util/util.[ch] > . files open/read, > . manage events attributes, > . various macros > > - test events attributes: tools/perf/tests/attr.c > . test_attr__open() > > - thread: tools/perf/util/thread_map.[ch] and > - cpu: tools/perf/util/cpumap.[ch] > . init and manage process maps > > - xyarray: tools/perf/util/xyarray.[ch] > > - syscall: tools/perf/perf-sys.h > > - cgroup: tools/perf/util/cgroup.[ch] > > The plan is to move the small and generic functions first: util, > xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, > trace-event-parse; and finally integrate rasd into the tools/ dir. > > Any thought? Can evlist, evsel etc. be moved at once? > > Patches should come soon, when time allows. Why don't you add it to tools/rasd/ and in tools/rasd/Makefile you just go on and add tools/perf/util/evlist.o et all to be linked directly, as a first step. Then, as a second step, we can create a tools/lib/perf/evlist.c having what is currently used by both tools/perf/ and tools/rasd/, i.e. what is proven to be useful for something other than perf. As the need arises, we go on moving things into tools/lib/perf/evlist.c et all from wherever it appeared first, be it from tools/rasd/, tools/perf/util/evlist.c or anywhere else. Initial rule being that once it is used by multiple tools living in tools/, then it deserves a place in tools/lib/perf/. Ditto for other stuff currently living in tools/perf/util/. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
perf & rasd integration plan
The RAS Deamon (rasd) as available at [1] and the development version at [2], allows to enable kernel tracepoints and outputs the tracepoints fields according to the kernel format. [1] https://git.kernel.org/cgit/utils/ras/rasd.git/ [2] https://git.linaro.org/people/jean.pihet/rasd.git rasd currently is a duplicate of the perf tool code, ultimately perf and rasd will use the same common code. The goal is to factor out the common code from perf and place it in small librairies in tools/lib. Here is the code that rasd currently uses and that should be moved to tools/lib: - debugfs: already in tools/lib/api/fs . mount and retrieve path - evlist: tools/perf/util/evlist.[ch] . create and init new evlist, . set cpu and thread maps, . add events to evlist, . init and use internal event id, . alloc and mmap events buffers, manage file descriptors, . enable events, . read events buffers, parse data, . unmap and free buffers - evsel: tools/perf/util/evsel.[ch] . create and init new tracepoints events, . init and use internal event id, . open events, manage fds, . close and free events - trace-event: tools/perf/util/trace-event.[ch] and tools/perf/util/trace-event-parse.c . retrieve and parse events format (using event-parse), . print out events fields - event-parse: already in tools/lib/traceevent/event-parse.[ch] . retrieve and parse events format, . parse events format and print out events fields - trace-seq: already in tools/lib/traceevent/trace-seq.c . format output string for event fields - events plugins: already in tools/lib/traceevent/event-plugin.c - util: tools/perf/util/util.[ch] . files open/read, . manage events attributes, . various macros - test events attributes: tools/perf/tests/attr.c . test_attr__open() - thread: tools/perf/util/thread_map.[ch] and - cpu: tools/perf/util/cpumap.[ch] . init and manage process maps - xyarray: tools/perf/util/xyarray.[ch] - syscall: tools/perf/perf-sys.h - cgroup: tools/perf/util/cgroup.[ch] The plan is to move the small and generic functions first: util, xyarray, cpumap, thread_map etc; then evlist, evsel, trace-event, trace-event-parse; and finally integrate rasd into the tools/ dir. Any thought? Can evlist, evsel etc. be moved at once? Patches should come soon, when time allows. Regards, Jean -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
