Re: [PATCH v2] perf stat: Align CSV output for summary mode
Hi Arnaldo, On 3/18/2021 9:15 PM, Arnaldo Carvalho de Melo wrote: Em Wed, Mar 17, 2021 at 02:51:42PM -0700, Andi Kleen escreveu: If you care about not breaking existing scripts, then the output they get with what they use as command line options must continue to produce the same output. It's not clear there are any useful ones (except for tools that handle both). It's really hard to parse the previous mess. It's simply not valid CSV. That's why I'm arguing that keeping compatibility is not useful here. We would be stuck with the broken mess as default forever. Fair enough, lets fix the default then. Jin, can you please consider adding a 'perf test' shell entry to parse the CSV mode with/without that summary? This way we'll notice when the new normal gets broken. - Arnaldo Thanks Arnaldo! I will post v3 with the perf test script. Thanks Jin Yao
Re: [PATCH v2] perf stat: Align CSV output for summary mode
Em Wed, Mar 17, 2021 at 02:51:42PM -0700, Andi Kleen escreveu: > > If you care about not breaking existing scripts, then the output they > > get with what they use as command line options must continue to produce > > the same output. > > It's not clear there are any useful ones (except for tools that handle > both). It's really hard to parse the previous mess. It's simply not > valid CSV. > > That's why I'm arguing that keeping compatibility is not useful here. > > We would be stuck with the broken mess as default forever. Fair enough, lets fix the default then. Jin, can you please consider adding a 'perf test' shell entry to parse the CSV mode with/without that summary? This way we'll notice when the new normal gets broken. - Arnaldo
Re: [PATCH v2] perf stat: Align CSV output for summary mode
> If you care about not breaking existing scripts, then the output they > get with what they use as command line options must continue to produce > the same output. It's not clear there are any useful ones (except for tools that handle both). It's really hard to parse the previous mess. It's simply not valid CSV. That's why I'm arguing that keeping compatibility is not useful here. We would be stuck with the broken mess as default forever. -Andi
Re: [PATCH v2] perf stat: Align CSV output for summary mode
Em Wed, Mar 17, 2021 at 03:02:05PM +0800, Jin Yao escreveu: > perf-stat has supported the summary mode. But the summary > lines break the CSV output so it's hard for scripts to parse > the result. > > Before: > > # perf stat -x, -I1000 --interval-count 1 --summary >1.001323097,8013.48,msec,cpu-clock,8013483384,100.00,8.013,CPUs > utilized >1.001323097,270,,context-switches,8013513297,100.00,0.034,K/sec >1.001323097,13,,cpu-migrations,8013530032,100.00,0.002,K/sec >1.001323097,184,,page-faults,8013546992,100.00,0.023,K/sec >1.001323097,20574191,,cycles,8013551506,100.00,0.003,GHz >1.001323097,10562267,,instructions,8013564958,100.00,0.51,insn per > cycle >1.001323097,2019244,,branches,8013575673,100.00,0.252,M/sec >1.001323097,106152,,branch-misses,8013585776,100.00,5.26,of all > branches > 8013.48,msec,cpu-clock,8013483384,100.00,7.984,CPUs utilized > 270,,context-switches,8013513297,100.00,0.034,K/sec > 13,,cpu-migrations,8013530032,100.00,0.002,K/sec > 184,,page-faults,8013546992,100.00,0.023,K/sec > 20574191,,cycles,8013551506,100.00,0.003,GHz > 10562267,,instructions,8013564958,100.00,0.51,insn per cycle > 2019244,,branches,8013575673,100.00,0.252,M/sec > 106152,,branch-misses,8013585776,100.00,5.26,of all branches > > The summary line loses the timestamp column, which breaks the > CVS output. > > We add a column at the original 'timestamp' position and it just says > 'summary' for the summary line. > > After: > > # perf stat -x, -I1000 --interval-count 1 --summary >1.001196053,8012.72,msec,cpu-clock,8012722903,100.00,8.013,CPUs > utilized >1.001196053,218,,context-switches,8012753271,100.00,0.027,K/sec >1.001196053,9,,cpu-migrations,8012769767,100.00,0.001,K/sec >1.001196053,0,,page-faults,8012786257,100.00,0.000,K/sec >1.001196053,15004518,,cycles,8012790637,100.00,0.002,GHz >1.001196053,7954691,,instructions,8012804027,100.00,0.53,insn per cycle >1.001196053,1590259,,branches,8012814766,100.00,0.198,M/sec >1.001196053,82601,,branch-misses,8012824365,100.00,5.19,of all branches >summary,8012.72,msec,cpu-clock,8012722903,100.00,7.986,CPUs > utilized >summary,218,,context-switches,8012753271,100.00,0.027,K/sec >summary,9,,cpu-migrations,8012769767,100.00,0.001,K/sec >summary,0,,page-faults,8012786257,100.00,0.000,K/sec >summary,15004518,,cycles,8012790637,100.00,0.002,GHz >summary,7954691,,instructions,8012804027,100.00,0.53,insn per cycle >summary,1590259,,branches,8012814766,100.00,0.198,M/sec >summary,82601,,branch-misses,8012824365,100.00,5.19,of all branches > > Now it's easy for script to analyse the summary lines. > > Of course, we also consider not to break possible existing scripts which > have fixed the broken CVS format, we provide a optiton '--no-cvs-summary' > to keep original output. If you care about not breaking existing scripts, then the output they get with what they use as command line options must continue to produce the same output. Adding a new option for these pre-existing scripts to use by definition will break them, that will need to be modified to use this new option to ask that the pre-existing output is produced. :-) > # perf stat -x, -I1000 --interval-count 1 --summary --no-cvs-summary >1.001213261,8012.67,msec,cpu-clock,8012672327,100.00,8.013,CPUs > utilized >1.001213261,197,,context-switches,8012703742,100.00,24.586,/sec >1.001213261,9,,cpu-migrations,8012720902,100.00,1.123,/sec >1.001213261,644,,page-faults,8012738266,100.00,80.373,/sec >1.001213261,18350698,,cycles,8012744109,100.00,0.002,GHz >1.001213261,12745021,,instructions,8012759001,100.00,0.69,insn per > cycle >1.001213261,2458033,,branches,8012770864,100.00,306.768,K/sec >1.001213261,102107,,branch-misses,8012781751,100.00,4.15,of all > branches > 8012.67,msec,cpu-clock,8012672327,100.00,7.985,CPUs utilized > 197,,context-switches,8012703742,100.00,24.586,/sec > 9,,cpu-migrations,8012720902,100.00,1.123,/sec > 644,,page-faults,8012738266,100.00,80.373,/sec > 18350698,,cycles,8012744109,100.00,0.002,GHz > 12745021,,instructions,8012759001,100.00,0.69,insn per cycle > 2458033,,branches,8012770864,100.00,306.768,K/sec > 102107,,branch-misses,8012781751,100.00,4.15,of all branches > > This option can be enabled in perf config by setting the variable > 'stat.no-cvs-summary'. > > # perf config stat.no-cvs-summary=true > > # perf config -l > stat.no-cvs-summary=true > > # perf stat -x, -I1000 --interval-count 1 --summary >1.001330198,8013.28,msec,cpu-clock,8013279201,100.00,8.013,CPUs > utilized >1.001330198,205,,context-switches,8013308394,100.00,25.583,/sec >1.001330198,10,,cpu-migrations,8013324681,100.00,1.248,/sec >
[PATCH v2] perf stat: Align CSV output for summary mode
perf-stat has supported the summary mode. But the summary lines break the CSV output so it's hard for scripts to parse the result. Before: # perf stat -x, -I1000 --interval-count 1 --summary 1.001323097,8013.48,msec,cpu-clock,8013483384,100.00,8.013,CPUs utilized 1.001323097,270,,context-switches,8013513297,100.00,0.034,K/sec 1.001323097,13,,cpu-migrations,8013530032,100.00,0.002,K/sec 1.001323097,184,,page-faults,8013546992,100.00,0.023,K/sec 1.001323097,20574191,,cycles,8013551506,100.00,0.003,GHz 1.001323097,10562267,,instructions,8013564958,100.00,0.51,insn per cycle 1.001323097,2019244,,branches,8013575673,100.00,0.252,M/sec 1.001323097,106152,,branch-misses,8013585776,100.00,5.26,of all branches 8013.48,msec,cpu-clock,8013483384,100.00,7.984,CPUs utilized 270,,context-switches,8013513297,100.00,0.034,K/sec 13,,cpu-migrations,8013530032,100.00,0.002,K/sec 184,,page-faults,8013546992,100.00,0.023,K/sec 20574191,,cycles,8013551506,100.00,0.003,GHz 10562267,,instructions,8013564958,100.00,0.51,insn per cycle 2019244,,branches,8013575673,100.00,0.252,M/sec 106152,,branch-misses,8013585776,100.00,5.26,of all branches The summary line loses the timestamp column, which breaks the CVS output. We add a column at the original 'timestamp' position and it just says 'summary' for the summary line. After: # perf stat -x, -I1000 --interval-count 1 --summary 1.001196053,8012.72,msec,cpu-clock,8012722903,100.00,8.013,CPUs utilized 1.001196053,218,,context-switches,8012753271,100.00,0.027,K/sec 1.001196053,9,,cpu-migrations,8012769767,100.00,0.001,K/sec 1.001196053,0,,page-faults,8012786257,100.00,0.000,K/sec 1.001196053,15004518,,cycles,8012790637,100.00,0.002,GHz 1.001196053,7954691,,instructions,8012804027,100.00,0.53,insn per cycle 1.001196053,1590259,,branches,8012814766,100.00,0.198,M/sec 1.001196053,82601,,branch-misses,8012824365,100.00,5.19,of all branches summary,8012.72,msec,cpu-clock,8012722903,100.00,7.986,CPUs utilized summary,218,,context-switches,8012753271,100.00,0.027,K/sec summary,9,,cpu-migrations,8012769767,100.00,0.001,K/sec summary,0,,page-faults,8012786257,100.00,0.000,K/sec summary,15004518,,cycles,8012790637,100.00,0.002,GHz summary,7954691,,instructions,8012804027,100.00,0.53,insn per cycle summary,1590259,,branches,8012814766,100.00,0.198,M/sec summary,82601,,branch-misses,8012824365,100.00,5.19,of all branches Now it's easy for script to analyse the summary lines. Of course, we also consider not to break possible existing scripts which have fixed the broken CVS format, we provide a optiton '--no-cvs-summary' to keep original output. # perf stat -x, -I1000 --interval-count 1 --summary --no-cvs-summary 1.001213261,8012.67,msec,cpu-clock,8012672327,100.00,8.013,CPUs utilized 1.001213261,197,,context-switches,8012703742,100.00,24.586,/sec 1.001213261,9,,cpu-migrations,8012720902,100.00,1.123,/sec 1.001213261,644,,page-faults,8012738266,100.00,80.373,/sec 1.001213261,18350698,,cycles,8012744109,100.00,0.002,GHz 1.001213261,12745021,,instructions,8012759001,100.00,0.69,insn per cycle 1.001213261,2458033,,branches,8012770864,100.00,306.768,K/sec 1.001213261,102107,,branch-misses,8012781751,100.00,4.15,of all branches 8012.67,msec,cpu-clock,8012672327,100.00,7.985,CPUs utilized 197,,context-switches,8012703742,100.00,24.586,/sec 9,,cpu-migrations,8012720902,100.00,1.123,/sec 644,,page-faults,8012738266,100.00,80.373,/sec 18350698,,cycles,8012744109,100.00,0.002,GHz 12745021,,instructions,8012759001,100.00,0.69,insn per cycle 2458033,,branches,8012770864,100.00,306.768,K/sec 102107,,branch-misses,8012781751,100.00,4.15,of all branches This option can be enabled in perf config by setting the variable 'stat.no-cvs-summary'. # perf config stat.no-cvs-summary=true # perf config -l stat.no-cvs-summary=true # perf stat -x, -I1000 --interval-count 1 --summary 1.001330198,8013.28,msec,cpu-clock,8013279201,100.00,8.013,CPUs utilized 1.001330198,205,,context-switches,8013308394,100.00,25.583,/sec 1.001330198,10,,cpu-migrations,8013324681,100.00,1.248,/sec 1.001330198,0,,page-faults,8013340926,100.00,0.000,/sec 1.001330198,8027742,,cycles,8013344503,100.00,0.001,GHz 1.001330198,2871717,,instructions,8013356501,100.00,0.36,insn per cycle 1.001330198,553564,,branches,8013366204,100.00,69.081,K/sec 1.001330198,54021,,branch-misses,8013375952,100.00,9.76,of all branches 8013.28,msec,cpu-clock,8013279201,100.00,7.985,CPUs utilized 205,,context-switches,8013308394,100.00,25.583,/sec 10,,cpu-migrations,8013324681,100.00,1.248,/sec 0,,page-faults,8013340926,100.00,0.000,/sec 8027742,,cycles,8013344503,100.00,0.001,GHz