Re: [PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
* Michael Ellermanwrote: > Jiri Olsa writes: > > > On Wed, Aug 31, 2016 at 09:15:30AM -0700, Andi Kleen wrote: > >> > > > >> > > > > >> > > > I've already made some changes in pmu-events/* to support > >> > > > this hierarchy to see how bad the change would be.. and > >> > > > it's not that bad ;-) > >> > > > >> > > Everything has to be automated, please no manual changes. > >> > > >> > sure > >> > > >> > so, if you're ok with the layout, how do you want to proceed further? > >> > >> If the split version is acceptable it's fine for me to merge it. > >> > >> I'll add split-json to my scripting, so the next update would > >> be split too. > > > > ook, I'll wait for patches then > > Who are you waiting for patches from? > > Would be great if this could go in for 4.9 still. No objections from me - the latest bits were good: Acked-by: Ingo Molnar Thanks, Ingo
Re: [PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
Jiri Olsawrites: > On Wed, Aug 31, 2016 at 09:15:30AM -0700, Andi Kleen wrote: >> > > >> > > > >> > > > I've already made some changes in pmu-events/* to support >> > > > this hierarchy to see how bad the change would be.. and >> > > > it's not that bad ;-) >> > > >> > > Everything has to be automated, please no manual changes. >> > >> > sure >> > >> > so, if you're ok with the layout, how do you want to proceed further? >> >> If the split version is acceptable it's fine for me to merge it. >> >> I'll add split-json to my scripting, so the next update would >> be split too. > > ook, I'll wait for patches then Who are you waiting for patches from? Would be great if this could go in for 4.9 still. cheers
Re: [PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
On Wed, Aug 31, 2016 at 09:15:30AM -0700, Andi Kleen wrote: > > > > > > > > > > > I've already made some changes in pmu-events/* to support > > > > this hierarchy to see how bad the change would be.. and > > > > it's not that bad ;-) > > > > > > Everything has to be automated, please no manual changes. > > > > sure > > > > so, if you're ok with the layout, how do you want to proceed further? > > If the split version is acceptable it's fine for me to merge it. > > I'll add split-json to my scripting, so the next update would > be split too. ook, I'll wait for patches then thanks, jirka
Re: [PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
> > > > > > > > I've already made some changes in pmu-events/* to support > > > this hierarchy to see how bad the change would be.. and > > > it's not that bad ;-) > > > > Everything has to be automated, please no manual changes. > > sure > > so, if you're ok with the layout, how do you want to proceed further? If the split version is acceptable it's fine for me to merge it. I'll add split-json to my scripting, so the next update would be split too. -Andi
Re: [PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
On Wed, Aug 31, 2016 at 07:42:47AM -0700, Andi Kleen wrote: > > hi, > > I had discussion with Ingo about the state of this patchset > > and there's one more requirement from his side - to split > > event files into per topic files > > Thanks Jiri. > > > > I made some initial changes over latest Sukadev's branch > > and came up with something like this: > > Did you just split it by the "Topic" fields? yep > > > > > $ find pmu-events/arch/x86/ > > pmu-events/arch/x86/ > > pmu-events/arch/x86/NehalemEX_core > > pmu-events/arch/x86/NehalemEX_core/Memory.json > > pmu-events/arch/x86/NehalemEX_core/Virtual-Memory.json > > pmu-events/arch/x86/NehalemEX_core/Cache.json > > pmu-events/arch/x86/NehalemEX_core/Pipeline.json > > pmu-events/arch/x86/NehalemEX_core/Floating-point.json > > pmu-events/arch/x86/NehalemEX_core/Other.json > > pmu-events/arch/x86/mapfile.csv > > pmu-events/arch/x86/Broadwell_core > > pmu-events/arch/x86/Broadwell_core/Memory.json > > pmu-events/arch/x86/Broadwell_core/Virtual-Memory.json > > pmu-events/arch/x86/Broadwell_core/Cache.json > > pmu-events/arch/x86/Broadwell_core/Pipeline.json > > pmu-events/arch/x86/Broadwell_core/Floating-point.json > > pmu-events/arch/x86/Broadwell_core/Other.json > > pmu-events/arch/x86/Broadwell_core/Frontend.json > > > > so let's have a discussion if this is acceptable for you guys > > Splitting is fine for me, as long as it's scriptable. > > I already have some scripts to generate the perf json files, > can update them to split. yep, there's split-json.py script earlier in the perf/json branch > > > > > I've already made some changes in pmu-events/* to support > > this hierarchy to see how bad the change would be.. and > > it's not that bad ;-) > > Everything has to be automated, please no manual changes. sure so, if you're ok with the layout, how do you want to proceed further? thanks, jirka
Re: [PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
> hi, > I had discussion with Ingo about the state of this patchset > and there's one more requirement from his side - to split > event files into per topic files Thanks Jiri. > > I made some initial changes over latest Sukadev's branch > and came up with something like this: Did you just split it by the "Topic" fields? > > $ find pmu-events/arch/x86/ > pmu-events/arch/x86/ > pmu-events/arch/x86/NehalemEX_core > pmu-events/arch/x86/NehalemEX_core/Memory.json > pmu-events/arch/x86/NehalemEX_core/Virtual-Memory.json > pmu-events/arch/x86/NehalemEX_core/Cache.json > pmu-events/arch/x86/NehalemEX_core/Pipeline.json > pmu-events/arch/x86/NehalemEX_core/Floating-point.json > pmu-events/arch/x86/NehalemEX_core/Other.json > pmu-events/arch/x86/mapfile.csv > pmu-events/arch/x86/Broadwell_core > pmu-events/arch/x86/Broadwell_core/Memory.json > pmu-events/arch/x86/Broadwell_core/Virtual-Memory.json > pmu-events/arch/x86/Broadwell_core/Cache.json > pmu-events/arch/x86/Broadwell_core/Pipeline.json > pmu-events/arch/x86/Broadwell_core/Floating-point.json > pmu-events/arch/x86/Broadwell_core/Other.json > pmu-events/arch/x86/Broadwell_core/Frontend.json > > so let's have a discussion if this is acceptable for you guys Splitting is fine for me, as long as it's scriptable. I already have some scripts to generate the perf json files, can update them to split. > > I've already made some changes in pmu-events/* to support > this hierarchy to see how bad the change would be.. and > it's not that bad ;-) Everything has to be automated, please no manual changes. -Andi
Re: [PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
On Mon, Jun 20, 2016 at 09:02:30PM -0700, Sukadev Bhattiprolu wrote: > CPUs support a large number of performance monitoring events (PMU events) > and often these events are very specific to an architecture/model of the > CPU. To use most of these PMU events with perf, we currently have to identify > them by their raw codes: > > perf stat -e r100f2 sleep 1 > > This patchset allows architectures to specify these PMU events in JSON > files located in 'tools/perf/pmu-events/arch/' of the mainline tree. > The events from the JSON files for the architecture are then built into > the perf binary. > > At run time, perf identifies the specific set of events for the CPU and > creates "event aliases". These aliases allow users to specify events by > "name" as: > > perf stat -e pm_1plus_ppc_cmpl sleep 1 > > The file, 'tools/perf/pmu-events/README' in [PATCH 16/16] gives more > details. > > Note: > - All known events tables for the architecture are included in the > perf binary. > > - For architectures that don't have any JSON files, an empty mapping > table is created and they should continue to build. > > Thanks to input from Andi Kleen, Jiri Olsa, Namhyung Kim and Ingo Molnar. > > These patches are available from: > > https://github.com/sukadev/linux.git > > Branch Description > -- > json-code-v20 Source Code only > json-data-v20 x86 and Powerpc datafiles only > json-code+data-v20 Both code and data (for build/test) > > NOTE: Only "source code" patches (i.e those in json-code-v20) are > being > emailed. Please pull the "data files" from the json-data-v20 branch. > > Changelog[v20] > - Rebase to recent perf/core > - Add Patch 20/20 to allow perf-stat to work with the period= field hi, I had discussion with Ingo about the state of this patchset and there's one more requirement from his side - to split event files into per topic files I made some initial changes over latest Sukadev's branch and came up with something like this: $ find pmu-events/arch/x86/ pmu-events/arch/x86/ pmu-events/arch/x86/NehalemEX_core pmu-events/arch/x86/NehalemEX_core/Memory.json pmu-events/arch/x86/NehalemEX_core/Virtual-Memory.json pmu-events/arch/x86/NehalemEX_core/Cache.json pmu-events/arch/x86/NehalemEX_core/Pipeline.json pmu-events/arch/x86/NehalemEX_core/Floating-point.json pmu-events/arch/x86/NehalemEX_core/Other.json pmu-events/arch/x86/mapfile.csv pmu-events/arch/x86/Broadwell_core pmu-events/arch/x86/Broadwell_core/Memory.json pmu-events/arch/x86/Broadwell_core/Virtual-Memory.json pmu-events/arch/x86/Broadwell_core/Cache.json pmu-events/arch/x86/Broadwell_core/Pipeline.json pmu-events/arch/x86/Broadwell_core/Floating-point.json pmu-events/arch/x86/Broadwell_core/Other.json pmu-events/arch/x86/Broadwell_core/Frontend.json so let's have a discussion if this is acceptable for you guys I've already made some changes in pmu-events/* to support this hierarchy to see how bad the change would be.. and it's not that bad ;-) you can check followin patches (only 2 Intel files transformed): 1d5ffa8bb969 perf, tools: Change jevents 65919f8901e3 perf, tools: Split Broadwell_core.json 7cd309a85465 perf, tools: Add Broadwell V14 event file e316aff2dd4e perf, tools: Split NehalemEX_core.json e19e8de49408 perf, tools: Add NehalemEX V1 event file It's available in: git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git perf/json thanks, jirka
[PATCH v20 00/20] perf, tools: Add support for PMU events in JSON format
CPUs support a large number of performance monitoring events (PMU events) and often these events are very specific to an architecture/model of the CPU. To use most of these PMU events with perf, we currently have to identify them by their raw codes: perf stat -e r100f2 sleep 1 This patchset allows architectures to specify these PMU events in JSON files located in 'tools/perf/pmu-events/arch/' of the mainline tree. The events from the JSON files for the architecture are then built into the perf binary. At run time, perf identifies the specific set of events for the CPU and creates "event aliases". These aliases allow users to specify events by "name" as: perf stat -e pm_1plus_ppc_cmpl sleep 1 The file, 'tools/perf/pmu-events/README' in [PATCH 16/16] gives more details. Note: - All known events tables for the architecture are included in the perf binary. - For architectures that don't have any JSON files, an empty mapping table is created and they should continue to build. Thanks to input from Andi Kleen, Jiri Olsa, Namhyung Kim and Ingo Molnar. These patches are available from: https://github.com/sukadev/linux.git Branch Description -- json-code-v20 Source Code only json-data-v20 x86 and Powerpc datafiles only json-code+data-v20 Both code and data (for build/test) NOTE: Only "source code" patches (i.e those in json-code-v20) are being emailed. Please pull the "data files" from the json-data-v20 branch. Changelog[v20] - Rebase to recent perf/core - Add Patch 20/20 to allow perf-stat to work with the period= field Changelog[v19] Rebase to recent perf/core; fix couple lines >80 chars. Changelog[v18] Rebase to recent perf/core; fix minor merge conflicts. Changelog[v17] Rebase to recent perf/core; couple of small fixes to processing Intel JSON files; allow case-insensitive PMU event names. Changelog[v16] Rebase to recent perf/core; fix minor merge conflicts; drop 3 patches that were merged into perf/core. Changelog[v15] Code changes: - Fix 'perf list' usage string and update man page. - Remove a redundant __maybe_unused tag. - Rebase to recent perf/core branch. Data files updates: json-files-5 branch - Rebase to perf/intel-json-files-5 from Andi Kleen - Add patch from Madhavan Srinivasan for couple more Powerpc models Changelog[v14] Comments from Jiri Olsa: - Change parameter name/type for pmu_add_cpu_aliases (from void *data to list_head *head) - Use asprintf() in file_name_to_tablename() and simplify/reorg code. - Use __weak definition from - Use fopen() with mode "w" and eliminate unlink() - Remove minor TODO. - Add error check for return value from strdup() in print_pmu_events(). - Move independent changes from patches 3,11,12 .. to separate patches for easier review/backport. - Clarify mapfile's "header line support" in patch description. - Fix build failure with DEBUG=1 Comment from Andi Kleen: - In tools/perf/pmu-events/Build, check for 'mapfile.csv' rather than 'mapfile*' Misc: - Minor changes/clarifications to tools/perf/pmu-events/README. Changelog[v13] Version: Individual patches have their own history :-) that I am preserving. Patchset version (v13) is for overall patchset and is somewhat arbitrary. - Added support for "categories" of events to perf - Add mapfile, jevents build dependency on pmu-events.c - Silence jevents when parsing JSON files unless V=1 is specified - Cleanup error messages - Fix memory leak with ->cpuid - Rebase to Arnaldo's tree - Allow overriding CPUID via environment variable - Support long descriptions for events - Handle header line in mapfile.csv - Cleanup JSON files (trim PublicDescription if identical to/prefix of BriefDescription field) Andi Kleen (12): perf, tools: Add jsmn `jasmine' JSON parser perf, tools, jevents: Program to convert JSON file to C style file perf, tools: Support CPU id matching for x86 v2 perf, tools: Support alias descriptions perf, tools: Query terminal width and use in perf list perf, tools: Add a --no-desc flag to perf list perf, tools: Add override support for event list CPUID perf, tools: Add support for event list topics perf, tools: Handle header line in mapfile perf, tools: Make alias matching case-insensitive perf, tools, pmu-events: Fix fixed counters on Intel perf, tools, pmu-events: Add Skylake frontend MSR support Sukadev Bhattiprolu (8): perf, tools: Use pmu_events table to create aliases