Hello community, here is the log from the commit of package perf for openSUSE:Factory checked in at 2019-01-11 14:06:01 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/perf (Old) and /work/SRC/openSUSE:Factory/.perf.new.28833 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "perf" Fri Jan 11 14:06:01 2019 rev:47 rq:664472 version:MACRO Changes: -------- --- /work/SRC/openSUSE:Factory/perf/perf.changes 2018-12-13 19:49:30.576618332 +0100 +++ /work/SRC/openSUSE:Factory/.perf.new.28833/perf.changes 2019-01-11 14:06:51.547717337 +0100 @@ -1,0 +2,27 @@ +Fri Jan 11 00:55:17 UTC 2019 - Tony Jones <[email protected]> + +- Enable python3. Remove all uses of python2 (bsc#1082197) + + New patch: update-tests-attr.py-to-support-python3.patch + New patch: remove-shebang-from-scripts-python.patch + New patch: remove-shebang-from-tests-attr.py.patch + New patch: remove-shebang-from-util-setup.py.patch + New patch: add-trace_context-extension-module-to-sys-modules.patch + New patch: use-pybytes-for-attr-in-trace-event-python.patch + New patch: port-event_analyzing_sample-script-to-python3.patch + New patch: port-failed-syscalls-by-pid-script-to-python3.patch + New patch: port-futex-contention-script-to-python3.patch + New patch: port-intel-pt-events-script-to-python3.patch + New patch: port-mem-phys-addr-to-python3.patch + New patch: port-netdev-times-script-to-python3.patch + New patch: port-net_dropmonitor-script-to-python3.patch + New patch: port-powerpc-hcalls-script-to-python3.patch + New patch: port-python-syscall-counts-script-to-python3.patch + New patch: port-sctop-script-to-python3.patch + New patch: port-sql-scripts-to-python3.patch + New patch: port-stackcollapse-script-to-python3.patch + New patch: port-stat-cpi-script-to-python3.patch + New patch: port-syscall-counts-by-pid-script-to-python3.patch + New patch: port-check-perf-trace-script-to-python3.patch + +------------------------------------------------------------------- New: ---- add-trace_context-extension-module-to-sys-modules.patch port-check-perf-trace-script-to-python3.patch port-event_analyzing_sample-script-to-python3.patch port-failed-syscalls-by-pid-script-to-python3.patch port-futex-contention-script-to-python3.patch port-intel-pt-events-script-to-python3.patch port-mem-phys-addr-to-python3.patch port-net_dropmonitor-script-to-python3.patch port-netdev-times-script-to-python3.patch port-powerpc-hcalls-script-to-python3.patch port-python-syscall-counts-script-to-python3.patch port-sctop-script-to-python3.patch port-sql-scripts-to-python3.patch port-stackcollapse-script-to-python3.patch port-stat-cpi-script-to-python3.patch port-syscall-counts-by-pid-script-to-python3.patch remove-shebang-from-scripts-python.patch remove-shebang-from-tests-attr.py.patch remove-shebang-from-util-setup.py.patch update-tests-attr.py-to-support-python3.patch use-pybytes-for-attr-in-trace-event-python.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ perf.spec ++++++ --- /var/tmp/diff_new_pack.uRWCoN/_old 2019-01-11 14:06:52.435716445 +0100 +++ /var/tmp/diff_new_pack.uRWCoN/_new 2019-01-11 14:06:52.435716445 +0100 @@ -1,7 +1,7 @@ # # spec file for package perf # -# Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany. +# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -45,6 +45,7 @@ %endif BuildRequires: newt-devel BuildRequires: openssl-devel +BuildRequires: python3-devel BuildRequires: xmlto BuildRequires: zlib-devel # dl_open requirement so not autodetected @@ -57,6 +58,28 @@ Requires: kernel >= 2.6.31 BuildRoot: %{_tmppath}/%{name}-%{version}-build +Patch1: update-tests-attr.py-to-support-python3.patch +Patch2: remove-shebang-from-scripts-python.patch +Patch3: remove-shebang-from-tests-attr.py.patch +Patch4: remove-shebang-from-util-setup.py.patch +Patch5: add-trace_context-extension-module-to-sys-modules.patch +Patch6: use-pybytes-for-attr-in-trace-event-python.patch +Patch7: port-event_analyzing_sample-script-to-python3.patch +Patch8: port-failed-syscalls-by-pid-script-to-python3.patch +Patch9: port-futex-contention-script-to-python3.patch +Patch10: port-intel-pt-events-script-to-python3.patch +Patch11: port-mem-phys-addr-to-python3.patch +Patch12: port-netdev-times-script-to-python3.patch +Patch13: port-net_dropmonitor-script-to-python3.patch +Patch14: port-powerpc-hcalls-script-to-python3.patch +Patch15: port-python-syscall-counts-script-to-python3.patch +Patch16: port-sctop-script-to-python3.patch +Patch17: port-sql-scripts-to-python3.patch +Patch18: port-stackcollapse-script-to-python3.patch +Patch19: port-stat-cpi-script-to-python3.patch +Patch20: port-syscall-counts-by-pid-script-to-python3.patch +Patch21: port-check-perf-trace-script-to-python3.patch + %description This package provides a userspace tool 'perf', which monitors performance for either unmodified binaries or the entire system. It requires a Linux kernel @@ -68,17 +91,38 @@ # copy necessary files from kernel-source since we need to modify them (cd /usr/src/linux ; tar -cf - COPYING CREDITS README tools include scripts Kbuild Makefile arch/*/{include,lib,Makefile} lib) | tar -xf - chmod +x tools/perf/util/generate-cmdlist.sh +%patch1 -p1 +%patch2 -p1 +%patch3 -p1 +%patch4 -p1 +%patch5 -p1 +%patch6 -p1 +%patch7 -p1 +%patch8 -p1 +%patch9 -p1 +%patch10 -p1 +%patch11 -p1 +%patch12 -p1 +%patch13 -p1 +%patch14 -p1 +%patch15 -p1 +%patch16 -p1 +%patch17 -p1 +%patch18 -p1 +%patch19 -p1 +%patch20 -p1 +%patch21 -p1 %build cd tools/perf export WERROR=0 # PASS rpm optflags as EXTRA_FLAGS, passing as CFLAGS overrides and breaks build -make %{?_smp_mflags} -f Makefile.perf EXTRA_CFLAGS="%{optflags}" ASCIIDOC8=1 prefix=/usr libdir=%{_libdir} perfexecdir=lib/%{name}-core all doc %{_perf_unwind} tipdir=share/doc/packages/perf +make %{?_smp_mflags} -f Makefile.perf PYTHON=python3 EXTRA_CFLAGS="%{optflags}" ASCIIDOC8=1 prefix=/usr libdir=%{_libdir} perfexecdir=lib/%{name}-core all doc %{_perf_unwind} tipdir=share/doc/packages/perf %install cd tools/perf export WERROR=0 -make -f Makefile.perf V=1 EXTRA_CFLAGS="%{optflags}" prefix=/usr libdir=%{_libdir} perfexecdir=lib/%{name}-core DESTDIR=%{buildroot} install install-doc %{_perf_unwind} tipdir=share/doc/packages/perf +make -f Makefile.perf V=1 PYTHON=python3 EXTRA_CFLAGS="%{optflags}" prefix=/usr libdir=%{_libdir} perfexecdir=lib/%{name}-core DESTDIR=%{buildroot} install install-doc %{_perf_unwind} tipdir=share/doc/packages/perf mkdir -p %{buildroot}/%{_docdir}/perf/examples/bpf mv %{buildroot}/usr/lib/perf/include/bpf/* %{buildroot}/%{_docdir}/perf/examples/bpf mv %{buildroot}/usr/lib/perf/examples/bpf/* %{buildroot}/%{_docdir}/perf/examples/bpf ++++++ add-trace_context-extension-module-to-sys-modules.patch ++++++ From: Tony Jones <[email protected]> Subject: Add trace_context extension module to sys,modules References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> In Python3, the result of PyModule_Create (called from scripts/python/Perf-Trace-Util/Context.c) is not automatically added to sys.modules. See: https://bugs.python.org/issue4592 Below is the observed behavior without the fix. # ldd /usr/bin/perf | grep -i python libpython3.6m.so.1.0 => /usr/lib64/libpython3.6m.so.1.0 (0x00007f8e1dfb2000) # perf record -a -e raw_syscalls:sys_enter sleep 5 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 187.177 MB perf.data (1581501 samples) ] # perf script -g python | cat generated Python script: perf-script.py # perf script -s ./perf-script.py Traceback (most recent call last): File "./perf-script.py", line 18, in <module> from perf_trace_context import * ModuleNotFoundError: No module named 'perf_trace_context' Error running python script ./perf-script.py --- tools/perf/util/scripting-engines/trace-event-python.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) --- a/tools/perf/util/scripting-engines/trace-event-python.c +++ b/tools/perf/util/scripting-engines/trace-event-python.c @@ -1494,6 +1494,7 @@ static void _free_command_line(wchar_t * static int python_start_script(const char *script, int argc, const char **argv) { struct tables *tables = &tables_global; + PyMODINIT_FUNC (*initfunc)(void); #if PY_MAJOR_VERSION < 3 const char **command_line; #else @@ -1504,24 +1505,25 @@ static int python_start_script(const cha FILE *fp; #if PY_MAJOR_VERSION < 3 + initfunc = initperf_trace_context; command_line = malloc((argc + 1) * sizeof(const char *)); command_line[0] = script; for (i = 1; i < argc + 1; i++) command_line[i] = argv[i - 1]; #else + initfunc = PyInit_perf_trace_context; command_line = malloc((argc + 1) * sizeof(wchar_t *)); command_line[0] = Py_DecodeLocale(script, NULL); for (i = 1; i < argc + 1; i++) command_line[i] = Py_DecodeLocale(argv[i - 1], NULL); #endif + PyImport_AppendInittab("perf_trace_context", initfunc); Py_Initialize(); #if PY_MAJOR_VERSION < 3 - initperf_trace_context(); PySys_SetArgv(argc + 1, (char **)command_line); #else - PyInit_perf_trace_context(); PySys_SetArgv(argc + 1, command_line); #endif ++++++ port-check-perf-trace-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port check-perf-trace script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Tom Zanussi <[email protected]> Adjust for python3. Also correct inconsistent indentation which was being flagged by Python3. N.B Use of from __future__ implies minimum version is now v2.6 --- tools/perf/scripts/python/check-perf-trace.py | 74 +++++++++++++------------- 1 file changed, 39 insertions(+), 35 deletions(-) --- a/tools/perf/scripts/python/check-perf-trace.py +++ b/tools/perf/scripts/python/check-perf-trace.py @@ -7,11 +7,13 @@ # events, etc. Basically, if this script runs successfully and # displays expected results, Python scripting support should be ok. +from __future__ import print_function + import os import sys -sys.path.append(os.environ['PERF_EXEC_PATH'] + \ - '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') +sys.path.append(os.environ['PERF_EXEC_PATH'] + + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') from Core import * from perf_trace_context import * @@ -19,37 +21,38 @@ from perf_trace_context import * unhandled = autodict() def trace_begin(): - print "trace_begin" - pass + print ("trace_begin") + pass def trace_end(): - print_unhandled() + print_unhandled() def irq__softirq_entry(event_name, context, common_cpu, - common_secs, common_nsecs, common_pid, common_comm, - common_callchain, vec): - print_header(event_name, common_cpu, common_secs, common_nsecs, - common_pid, common_comm) + common_secs, common_nsecs, common_pid, common_comm, + common_callchain, vec): - print_uncommon(context) + print_header(event_name, common_cpu, common_secs, common_nsecs, + common_pid, common_comm) - print "vec=%s\n" % \ - (symbol_str("irq__softirq_entry", "vec", vec)), + print_uncommon(context) -def kmem__kmalloc(event_name, context, common_cpu, - common_secs, common_nsecs, common_pid, common_comm, - common_callchain, call_site, ptr, bytes_req, bytes_alloc, - gfp_flags): - print_header(event_name, common_cpu, common_secs, common_nsecs, - common_pid, common_comm) - - print_uncommon(context) - - print "call_site=%u, ptr=%u, bytes_req=%u, " \ - "bytes_alloc=%u, gfp_flags=%s\n" % \ - (call_site, ptr, bytes_req, bytes_alloc, + print ("vec=%s" % + (symbol_str("irq__softirq_entry", "vec", vec))) - flag_str("kmem__kmalloc", "gfp_flags", gfp_flags)), +def kmem__kmalloc(event_name, context, common_cpu, + common_secs, common_nsecs, common_pid, common_comm, + common_callchain, call_site, ptr, bytes_req, bytes_alloc, + gfp_flags): + + print_header(event_name, common_cpu, common_secs, common_nsecs, + common_pid, common_comm) + + print_uncommon(context) + + print ("call_site=%u, ptr=%u, bytes_req=%u, " + "bytes_alloc=%u, gfp_flags=%s\n" % + (call_site, ptr, bytes_req, bytes_alloc, + flag_str("kmem__kmalloc", "gfp_flags", gfp_flags))), def trace_unhandled(event_name, context, event_fields_dict): try: @@ -58,25 +61,26 @@ def trace_unhandled(event_name, context, unhandled[event_name] = 1 def print_header(event_name, cpu, secs, nsecs, pid, comm): - print "%-20s %5u %05u.%09u %8u %-20s " % \ - (event_name, cpu, secs, nsecs, pid, comm), + print ("%-20s %5u %05u.%09u %8u %-20s " % + (event_name, cpu, secs, nsecs, pid, comm), + end='') # print trace fields not included in handler args def print_uncommon(context): - print "common_preempt_count=%d, common_flags=%s, common_lock_depth=%d, " \ - % (common_pc(context), trace_flag_str(common_flags(context)), \ - common_lock_depth(context)) + print ("common_preempt_count=%d, common_flags=%s, common_lock_depth=%d, " % + (common_pc(context), trace_flag_str(common_flags(context)), + common_lock_depth(context))), def print_unhandled(): keys = unhandled.keys() if not keys: return - print "\nunhandled events:\n\n", + print ("\nunhandled events:\n") - print "%-40s %10s\n" % ("event", "count"), - print "%-40s %10s\n" % ("----------------------------------------", \ - "-----------"), + print ("%-40s %10s" % ("event", "count")) + print ("%-40s %10s" % ("----------------------------------------", + "-----------")) for event_name in keys: - print "%-40s %10d\n" % (event_name, unhandled[event_name]) + print ("%-40s %10d\n" % (event_name, unhandled[event_name])) ++++++ port-event_analyzing_sample-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port event_analyzing_sample script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> --- tools/perf/scripts/python/event_analyzing_sample.py | 42 ++++++++++---------- 1 file changed, 21 insertions(+), 21 deletions(-) --- a/tools/perf/scripts/python/event_analyzing_sample.py +++ b/tools/perf/scripts/python/event_analyzing_sample.py @@ -37,7 +37,7 @@ con = sqlite3.connect("/dev/shm/perf.db" con.isolation_level = None def trace_begin(): - print "In trace_begin:\n" + print ("In trace_begin:\n") # # Will create several tables at the start, pebs_ll is for PEBS data with @@ -102,7 +102,7 @@ def insert_db(event): event.ip, event.status, event.dse, event.dla, event.lat)) def trace_end(): - print "In trace_end:\n" + print ("In trace_end:\n") # We show the basic info for the 2 type of event classes show_general_events() show_pebs_ll() @@ -123,29 +123,29 @@ def show_general_events(): # Check the total record number in the table count = con.execute("select count(*) from gen_events") for t in count: - print "There is %d records in gen_events table" % t[0] + print ("There is %d records in gen_events table" % t[0]) if t[0] == 0: return - print "Statistics about the general events grouped by thread/symbol/dso: \n" + print ("Statistics about the general events grouped by thread/symbol/dso: \n") # Group by thread commq = con.execute("select comm, count(comm) from gen_events group by comm order by -count(comm)") - print "\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42) + print ("\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42)) for row in commq: - print "%16s %8d %s" % (row[0], row[1], num2sym(row[1])) + print ("%16s %8d %s" % (row[0], row[1], num2sym(row[1]))) # Group by symbol - print "\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58) + print ("\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58)) symbolq = con.execute("select symbol, count(symbol) from gen_events group by symbol order by -count(symbol)") for row in symbolq: - print "%32s %8d %s" % (row[0], row[1], num2sym(row[1])) + print ("%32s %8d %s" % (row[0], row[1], num2sym(row[1]))) # Group by dso - print "\n%40s %8s %16s\n%s" % ("dso", "number", "histogram", "="*74) + print ("\n%40s %8s %16s\n%s" % ("dso", "number", "histogram", "="*74)) dsoq = con.execute("select dso, count(dso) from gen_events group by dso order by -count(dso)") for row in dsoq: - print "%40s %8d %s" % (row[0], row[1], num2sym(row[1])) + print ("%40s %8d %s" % (row[0], row[1], num2sym(row[1]))) # # This function just shows the basic info, and we could do more with the @@ -156,35 +156,35 @@ def show_pebs_ll(): count = con.execute("select count(*) from pebs_ll") for t in count: - print "There is %d records in pebs_ll table" % t[0] + print ("There is %d records in pebs_ll table" % t[0]) if t[0] == 0: return - print "Statistics about the PEBS Load Latency events grouped by thread/symbol/dse/latency: \n" + print ("Statistics about the PEBS Load Latency events grouped by thread/symbol/dse/latency: \n") # Group by thread commq = con.execute("select comm, count(comm) from pebs_ll group by comm order by -count(comm)") - print "\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42) + print ("\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42)) for row in commq: - print "%16s %8d %s" % (row[0], row[1], num2sym(row[1])) + print ("%16s %8d %s" % (row[0], row[1], num2sym(row[1]))) # Group by symbol - print "\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58) + print ("\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58)) symbolq = con.execute("select symbol, count(symbol) from pebs_ll group by symbol order by -count(symbol)") for row in symbolq: - print "%32s %8d %s" % (row[0], row[1], num2sym(row[1])) + print ("%32s %8d %s" % (row[0], row[1], num2sym(row[1]))) # Group by dse dseq = con.execute("select dse, count(dse) from pebs_ll group by dse order by -count(dse)") - print "\n%32s %8s %16s\n%s" % ("dse", "number", "histogram", "="*58) + print ("\n%32s %8s %16s\n%s" % ("dse", "number", "histogram", "="*58)) for row in dseq: - print "%32s %8d %s" % (row[0], row[1], num2sym(row[1])) + print ("%32s %8d %s" % (row[0], row[1], num2sym(row[1]))) # Group by latency latq = con.execute("select lat, count(lat) from pebs_ll group by lat order by lat") - print "\n%32s %8s %16s\n%s" % ("latency", "number", "histogram", "="*58) + print ("\n%32s %8s %16s\n%s" % ("latency", "number", "histogram", "="*58)) for row in latq: - print "%32s %8d %s" % (row[0], row[1], num2sym(row[1])) + print ("%32s %8d %s" % (row[0], row[1], num2sym(row[1]))) def trace_unhandled(event_name, context, event_fields_dict): - print ' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())]) + print (' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())])) ++++++ port-failed-syscalls-by-pid-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port failed-syscalls-by-pid script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Tom Zanussi <[email protected]> --- tools/perf/scripts/python/failed-syscalls-by-pid.py | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) --- a/tools/perf/scripts/python/failed-syscalls-by-pid.py +++ b/tools/perf/scripts/python/failed-syscalls-by-pid.py @@ -32,7 +32,7 @@ if len(sys.argv) > 1: syscalls = autodict() def trace_begin(): - print "Press control+C to stop and show the summary" + print ("Press control+C to stop and show the summary") def trace_end(): print_error_totals() @@ -57,22 +57,21 @@ def syscalls__sys_exit(event_name, conte def print_error_totals(): if for_comm is not None: - print "\nsyscall errors for %s:\n\n" % (for_comm), + print ("\nsyscall errors for %s:\n" % (for_comm)) else: - print "\nsyscall errors:\n\n", + print ("\nsyscall errors:\n") - print "%-30s %10s\n" % ("comm [pid]", "count"), - print "%-30s %10s\n" % ("------------------------------", \ - "----------"), + print ("%-30s %10s" % ("comm [pid]", "count")) + print ("%-30s %10s" % ("------------------------------", "----------")) comm_keys = syscalls.keys() for comm in comm_keys: pid_keys = syscalls[comm].keys() for pid in pid_keys: - print "\n%s [%d]\n" % (comm, pid), + print ("\n%s [%d]" % (comm, pid)) id_keys = syscalls[comm][pid].keys() for id in id_keys: - print " syscall: %-16s\n" % syscall_name(id), + print (" syscall: %-16s" % syscall_name(id)) ret_keys = syscalls[comm][pid][id].keys() - for ret, val in sorted(syscalls[comm][pid][id].iteritems(), key = lambda(k, v): (v, k), reverse = True): - print " err = %-20s %10d\n" % (strerror(ret), val), + for ret, val in sorted(syscalls[comm][pid][id].items(), key = lambda kv: (kv[1], kv[0]), reverse = True): + print (" err = %-20s %10d" % (strerror(ret), val)) ++++++ port-futex-contention-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port futex-contention script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Arnaldo Carvalho de Melo <[email protected]> --- tools/perf/scripts/python/futex-contention.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/tools/perf/scripts/python/futex-contention.py +++ b/tools/perf/scripts/python/futex-contention.py @@ -33,18 +33,18 @@ def syscalls__sys_enter_futex(event, ctx def syscalls__sys_exit_futex(event, ctxt, cpu, s, ns, tid, comm, callchain, nr, ret): - if thread_blocktime.has_key(tid): + if tid in thread_blocktime: elapsed = nsecs(s, ns) - thread_blocktime[tid] add_stats(lock_waits, (tid, thread_thislock[tid]), elapsed) del thread_blocktime[tid] del thread_thislock[tid] def trace_begin(): - print "Press control+C to stop and show the summary" + print ("Press control+C to stop and show the summary") def trace_end(): for (tid, lock) in lock_waits: min, max, avg, count = lock_waits[tid, lock] - print "%s[%d] lock %x contended %d times, %d avg ns" % \ - (process_names[tid], tid, lock, count, avg) + print ("%s[%d] lock %x contended %d times, %d avg ns" % + (process_names[tid], tid, lock, count, avg)) ++++++ port-intel-pt-events-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port intel-pt-events script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Adrian Hunter <[email protected]> Fix space/tab inconsistency as python3 enforces consistency. --- tools/perf/scripts/python/intel-pt-events.py | 138 +++++++++++++-------------- 1 file changed, 70 insertions(+), 68 deletions(-) --- a/tools/perf/scripts/python/intel-pt-events.py +++ b/tools/perf/scripts/python/intel-pt-events.py @@ -10,79 +10,81 @@ # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. +from __future__ import print_function + import os import sys import struct sys.path.append(os.environ['PERF_EXEC_PATH'] + \ - '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') # These perf imports are not used at present #from perf_trace_context import * #from Core import * def trace_begin(): - print "Intel PT Power Events and PTWRITE" + print ("Intel PT Power Events and PTWRITE") def trace_end(): - print "End" + print ("End") def trace_unhandled(event_name, context, event_fields_dict): - print ' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())]) + print (' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())])) def print_ptwrite(raw_buf): - data = struct.unpack_from("<IQ", raw_buf) - flags = data[0] - payload = data[1] - exact_ip = flags & 1 - print "IP: %u payload: %#x" % (exact_ip, payload), + data = struct.unpack_from("<IQ", raw_buf) + flags = data[0] + payload = data[1] + exact_ip = flags & 1 + print ("IP: %u payload: %#x" % (exact_ip, payload), end='') def print_cbr(raw_buf): - data = struct.unpack_from("<BBBBII", raw_buf) - cbr = data[0] - f = (data[4] + 500) / 1000 - p = ((cbr * 1000 / data[2]) + 5) / 10 - print "%3u freq: %4u MHz (%3u%%)" % (cbr, f, p), + data = struct.unpack_from("<BBBBII", raw_buf) + cbr = data[0] + f = (data[4] + 500) / 1000 + p = ((cbr * 1000 / data[2]) + 5) / 10 + print ("%3u freq: %4u MHz (%3u%%)" % (cbr, f, p), end='') def print_mwait(raw_buf): - data = struct.unpack_from("<IQ", raw_buf) - payload = data[1] - hints = payload & 0xff - extensions = (payload >> 32) & 0x3 - print "hints: %#x extensions: %#x" % (hints, extensions), + data = struct.unpack_from("<IQ", raw_buf) + payload = data[1] + hints = payload & 0xff + extensions = (payload >> 32) & 0x3 + print ("hints: %#x extensions: %#x" % (hints, extensions), end='') def print_pwre(raw_buf): - data = struct.unpack_from("<IQ", raw_buf) - payload = data[1] - hw = (payload >> 7) & 1 - cstate = (payload >> 12) & 0xf - subcstate = (payload >> 8) & 0xf - print "hw: %u cstate: %u sub-cstate: %u" % (hw, cstate, subcstate), + data = struct.unpack_from("<IQ", raw_buf) + payload = data[1] + hw = (payload >> 7) & 1 + cstate = (payload >> 12) & 0xf + subcstate = (payload >> 8) & 0xf + print ("hw: %u cstate: %u sub-cstate: %u" % (hw, cstate, subcstate), end='') def print_exstop(raw_buf): - data = struct.unpack_from("<I", raw_buf) - flags = data[0] - exact_ip = flags & 1 - print "IP: %u" % (exact_ip), + data = struct.unpack_from("<I", raw_buf) + flags = data[0] + exact_ip = flags & 1 + print ("IP: %u" % (exact_ip), end='') def print_pwrx(raw_buf): - data = struct.unpack_from("<IQ", raw_buf) - payload = data[1] - deepest_cstate = payload & 0xf - last_cstate = (payload >> 4) & 0xf - wake_reason = (payload >> 8) & 0xf - print "deepest cstate: %u last cstate: %u wake reason: %#x" % (deepest_cstate, last_cstate, wake_reason), + data = struct.unpack_from("<IQ", raw_buf) + payload = data[1] + deepest_cstate = payload & 0xf + last_cstate = (payload >> 4) & 0xf + wake_reason = (payload >> 8) & 0xf + print ("deepest cstate: %u last cstate: %u wake reason: %#x" % (deepest_cstate, last_cstate, wake_reason), end='') def print_common_start(comm, sample, name): - ts = sample["time"] - cpu = sample["cpu"] - pid = sample["pid"] - tid = sample["tid"] - print "%16s %5u/%-5u [%03u] %9u.%09u %7s:" % (comm, pid, tid, cpu, ts / 1000000000, ts %1000000000, name), + ts = sample["time"] + cpu = sample["cpu"] + pid = sample["pid"] + tid = sample["tid"] + print ("%16s %5u/%-5u [%03u] %9u.%09u %7s:" % (comm, pid, tid, cpu, ts / 1000000000, ts %1000000000, name), end='') def print_common_ip(sample, symbol, dso): - ip = sample["ip"] - print "%16x %s (%s)" % (ip, symbol, dso) + ip = sample["ip"] + print ("%16x %s (%s)" % (ip, symbol, dso)) def process_event(param_dict): event_attr = param_dict["attr"] @@ -92,37 +94,37 @@ def process_event(param_dict): name = param_dict["ev_name"] # Symbol and dso info are not always resolved - if (param_dict.has_key("dso")): + if "dso" in param_dict: dso = param_dict["dso"] else: dso = "[unknown]" - if (param_dict.has_key("symbol")): + if "symbol" in param_dict: symbol = param_dict["symbol"] else: symbol = "[unknown]" - if name == "ptwrite": - print_common_start(comm, sample, name) - print_ptwrite(raw_buf) - print_common_ip(sample, symbol, dso) - elif name == "cbr": - print_common_start(comm, sample, name) - print_cbr(raw_buf) - print_common_ip(sample, symbol, dso) - elif name == "mwait": - print_common_start(comm, sample, name) - print_mwait(raw_buf) - print_common_ip(sample, symbol, dso) - elif name == "pwre": - print_common_start(comm, sample, name) - print_pwre(raw_buf) - print_common_ip(sample, symbol, dso) - elif name == "exstop": - print_common_start(comm, sample, name) - print_exstop(raw_buf) - print_common_ip(sample, symbol, dso) - elif name == "pwrx": - print_common_start(comm, sample, name) - print_pwrx(raw_buf) - print_common_ip(sample, symbol, dso) + if name == "ptwrite": + print_common_start(comm, sample, name) + print_ptwrite(raw_buf) + print_common_ip(sample, symbol, dso) + elif name == "cbr": + print_common_start(comm, sample, name) + print_cbr(raw_buf) + print_common_ip(sample, symbol, dso) + elif name == "mwait": + print_common_start(comm, sample, name) + print_mwait(raw_buf) + print_common_ip(sample, symbol, dso) + elif name == "pwre": + print_common_start(comm, sample, name) + print_pwre(raw_buf) + print_common_ip(sample, symbol, dso) + elif name == "exstop": + print_common_start(comm, sample, name) + print_exstop(raw_buf) + print_common_ip(sample, symbol, dso) + elif name == "pwrx": + print_common_start(comm, sample, name) + print_pwrx(raw_buf) + print_common_ip(sample, symbol, dso) ++++++ port-mem-phys-addr-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port mem-phys-addr to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> --- tools/perf/scripts/python/mem-phys-addr.py | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) --- a/tools/perf/scripts/python/mem-phys-addr.py +++ b/tools/perf/scripts/python/mem-phys-addr.py @@ -4,6 +4,8 @@ # Copyright (c) 2018, Intel Corporation. from __future__ import division +from __future__ import print_function + import os import sys import struct @@ -31,21 +33,23 @@ def parse_iomem(): for i, j in enumerate(f): m = re.split('-|:',j,2) if m[2].strip() == 'System RAM': - system_ram.append(long(m[0], 16)) - system_ram.append(long(m[1], 16)) + system_ram.append(int(m[0], 16)) + system_ram.append(int(m[1], 16)) if m[2].strip() == 'Persistent Memory': - pmem.append(long(m[0], 16)) - pmem.append(long(m[1], 16)) + pmem.append(int(m[0], 16)) + pmem.append(int(m[1], 16)) def print_memory_type(): - print "Event: %s" % (event_name) - print "%-40s %10s %10s\n" % ("Memory type", "count", "percentage"), - print "%-40s %10s %10s\n" % ("----------------------------------------", \ + print ("Event: %s" % (event_name)) + print ("%-40s %10s %10s\n" % ("Memory type", "count", "percentage"), end='') + print ("%-40s %10s %10s\n" % ("----------------------------------------", "-----------", "-----------"), + end=''); total = sum(load_mem_type_cnt.values()) for mem_type, count in sorted(load_mem_type_cnt.most_common(), \ - key = lambda(k, v): (v, k), reverse = True): - print "%-40s %10d %10.1f%%\n" % (mem_type, count, 100 * count / total), + key = lambda kv: (kv[1], kv[0]), reverse = True): + print ("%-40s %10d %10.1f%%\n" % (mem_type, count, 100 * count / total), + end='') def trace_begin(): parse_iomem() @@ -80,7 +84,7 @@ def find_memory_type(phys_addr): f.seek(0, 0) for j in f: m = re.split('-|:',j,2) - if long(m[0], 16) <= phys_addr <= long(m[1], 16): + if int(m[0], 16) <= phys_addr <= int(m[1], 16): return m[2] return "N/A" ++++++ port-net_dropmonitor-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port net_dropmonitor script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Neil Horman <[email protected]> --- tools/perf/scripts/python/net_dropmonitor.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/tools/perf/scripts/python/net_dropmonitor.py +++ b/tools/perf/scripts/python/net_dropmonitor.py @@ -50,19 +50,19 @@ def get_sym(sloc): return (None, 0) def print_drop_table(): - print "%25s %25s %25s" % ("LOCATION", "OFFSET", "COUNT") + print ("%25s %25s %25s" % ("LOCATION", "OFFSET", "COUNT")) for i in drop_log.keys(): (sym, off) = get_sym(i) if sym == None: sym = i - print "%25s %25s %25s" % (sym, off, drop_log[i]) + print ("%25s %25s %25s" % (sym, off, drop_log[i])) def trace_begin(): - print "Starting trace (Ctrl-C to dump results)" + print ("Starting trace (Ctrl-C to dump results)") def trace_end(): - print "Gathering kallsyms data" + print ("Gathering kallsyms data") get_kallsyms_table() print_drop_table() ++++++ port-netdev-times-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port netdev-times script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Koki Sanagi <[email protected]> --- tools/perf/scripts/python/netdev-times.py | 80 +++++++++++++++--------------- 1 file changed, 40 insertions(+), 40 deletions(-) --- a/tools/perf/scripts/python/netdev-times.py +++ b/tools/perf/scripts/python/netdev-times.py @@ -17,6 +17,7 @@ sys.path.append(os.environ['PERF_EXEC_PA from perf_trace_context import * from Core import * from Util import * +from functools import cmp_to_key all_event_list = []; # insert all tracepoint event related with this script irq_dic = {}; # key is cpu and value is a list which stacks irqs @@ -61,12 +62,12 @@ def diff_msec(src, dst): def print_transmit(hunk): if dev != 0 and hunk['dev'].find(dev) < 0: return - print "%7s %5d %6d.%06dsec %12.3fmsec %12.3fmsec" % \ + print ("%7s %5d %6d.%06dsec %12.3fmsec %12.3fmsec" % (hunk['dev'], hunk['len'], nsecs_secs(hunk['queue_t']), nsecs_nsecs(hunk['queue_t'])/1000, diff_msec(hunk['queue_t'], hunk['xmit_t']), - diff_msec(hunk['xmit_t'], hunk['free_t'])) + diff_msec(hunk['xmit_t'], hunk['free_t']))) # Format for displaying rx packet processing PF_IRQ_ENTRY= " irq_entry(+%.3fmsec irq=%d:%s)" @@ -98,55 +99,55 @@ def print_receive(hunk): if show_hunk == 0: return - print "%d.%06dsec cpu=%d" % \ - (nsecs_secs(base_t), nsecs_nsecs(base_t)/1000, cpu) + print ("%d.%06dsec cpu=%d" % + (nsecs_secs(base_t), nsecs_nsecs(base_t)/1000, cpu)) for i in range(len(irq_list)): - print PF_IRQ_ENTRY % \ + print (PF_IRQ_ENTRY % (diff_msec(base_t, irq_list[i]['irq_ent_t']), - irq_list[i]['irq'], irq_list[i]['name']) - print PF_JOINT + irq_list[i]['irq'], irq_list[i]['name'])) + print (PF_JOINT) irq_event_list = irq_list[i]['event_list'] for j in range(len(irq_event_list)): irq_event = irq_event_list[j] if irq_event['event'] == 'netif_rx': - print PF_NET_RX % \ + print (PF_NET_RX % (diff_msec(base_t, irq_event['time']), - irq_event['skbaddr']) - print PF_JOINT - print PF_SOFT_ENTRY % \ - diff_msec(base_t, hunk['sirq_ent_t']) - print PF_JOINT + irq_event['skbaddr'])) + print (PF_JOINT) + print (PF_SOFT_ENTRY % + diff_msec(base_t, hunk['sirq_ent_t'])) + print (PF_JOINT) event_list = hunk['event_list'] for i in range(len(event_list)): event = event_list[i] if event['event_name'] == 'napi_poll': - print PF_NAPI_POLL % \ - (diff_msec(base_t, event['event_t']), event['dev']) + print (PF_NAPI_POLL % + (diff_msec(base_t, event['event_t']), event['dev'])) if i == len(event_list) - 1: - print "" + print ("") else: - print PF_JOINT + print (PF_JOINT) else: - print PF_NET_RECV % \ + print (PF_NET_RECV % (diff_msec(base_t, event['event_t']), event['skbaddr'], - event['len']) + event['len'])) if 'comm' in event.keys(): - print PF_WJOINT - print PF_CPY_DGRAM % \ + print (PF_WJOINT) + print (PF_CPY_DGRAM % (diff_msec(base_t, event['comm_t']), - event['pid'], event['comm']) + event['pid'], event['comm'])) elif 'handle' in event.keys(): - print PF_WJOINT + print (PF_WJOINT) if event['handle'] == "kfree_skb": - print PF_KFREE_SKB % \ + print (PF_KFREE_SKB % (diff_msec(base_t, event['comm_t']), - event['location']) + event['location'])) elif event['handle'] == "consume_skb": - print PF_CONS_SKB % \ + print (PF_CONS_SKB % diff_msec(base_t, - event['comm_t']) - print PF_JOINT + event['comm_t'])) + print (PF_JOINT) def trace_begin(): global show_tx @@ -172,8 +173,7 @@ def trace_begin(): def trace_end(): # order all events in time - all_event_list.sort(lambda a,b :cmp(a[EINFO_IDX_TIME], - b[EINFO_IDX_TIME])) + all_event_list.sort(key=cmp_to_key(lambda a,b :a[EINFO_IDX_TIME] < b[EINFO_IDX_TIME])) # process all events for i in range(len(all_event_list)): event_info = all_event_list[i] @@ -210,19 +210,19 @@ def trace_end(): print_receive(receive_hunk_list[i]) # display transmit hunks if show_tx: - print " dev len Qdisc " \ - " netdevice free" + print (" dev len Qdisc " + " netdevice free") for i in range(len(tx_free_list)): print_transmit(tx_free_list[i]) if debug: - print "debug buffer status" - print "----------------------------" - print "xmit Qdisc:remain:%d overflow:%d" % \ - (len(tx_queue_list), of_count_tx_queue_list) - print "xmit netdevice:remain:%d overflow:%d" % \ - (len(tx_xmit_list), of_count_tx_xmit_list) - print "receive:remain:%d overflow:%d" % \ - (len(rx_skb_list), of_count_rx_skb_list) + print ("debug buffer status") + print ("----------------------------") + print ("xmit Qdisc:remain:%d overflow:%d" % + (len(tx_queue_list), of_count_tx_queue_list)) + print ("xmit netdevice:remain:%d overflow:%d" % + (len(tx_xmit_list), of_count_tx_xmit_list)) + print ("receive:remain:%d overflow:%d" % + (len(rx_skb_list), of_count_rx_skb_list)) # called from perf, when it finds a correspoinding event def irq__softirq_entry(name, context, cpu, sec, nsec, pid, comm, callchain, vec): ++++++ port-powerpc-hcalls-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port powerpc-hcalls script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Ravi Bangoria <[email protected]> --- tools/perf/scripts/python/powerpc-hcalls.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/tools/perf/scripts/python/powerpc-hcalls.py +++ b/tools/perf/scripts/python/powerpc-hcalls.py @@ -157,8 +157,8 @@ def hcall_table_lookup(opcode): print_ptrn = '%-28s%10s%10s%10s%10s' def trace_end(): - print print_ptrn % ('hcall', 'count', 'min(ns)', 'max(ns)', 'avg(ns)') - print '-' * 68 + print (print_ptrn % ('hcall', 'count', 'min(ns)', 'max(ns)', 'avg(ns)')) + print ('-' * 68) for opcode in output: h_name = hcall_table_lookup(opcode) time = output[opcode]['time'] @@ -166,7 +166,7 @@ def trace_end(): min_t = output[opcode]['min'] max_t = output[opcode]['max'] - print print_ptrn % (h_name, cnt, min_t, max_t, time/cnt) + print (print_ptrn % (h_name, cnt, min_t, max_t, time/cnt)) def powerpc__hcall_exit(name, context, cpu, sec, nsec, pid, comm, callchain, opcode, retval): @@ -190,7 +190,7 @@ def powerpc__hcall_exit(name, context, c del d_enter[cpu][opcode] # else: -# print "Can't find matching hcall_enter event. Ignoring sample" +# print ("Can't find matching hcall_enter event. Ignoring sample") def powerpc__hcall_entry(event_name, context, cpu, sec, nsec, pid, comm, callchain, opcode): ++++++ port-python-syscall-counts-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port python/syscall-counts script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> --- tools/perf/scripts/python/syscall-counts.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) --- a/tools/perf/scripts/python/syscall-counts.py +++ b/tools/perf/scripts/python/syscall-counts.py @@ -28,7 +28,7 @@ if len(sys.argv) > 1: syscalls = autodict() def trace_begin(): - print "Press control+C to stop and show the summary" + print ("Press control+C to stop and show the summary") def trace_end(): print_syscall_totals() @@ -51,14 +51,14 @@ def syscalls__sys_enter(event_name, cont def print_syscall_totals(): if for_comm is not None: - print "\nsyscall events for %s:\n\n" % (for_comm), + print ("\nsyscall events for %s:\n" % (for_comm)) else: - print "\nsyscall events:\n\n", + print ("\nsyscall events:\n\n") - print "%-40s %10s\n" % ("event", "count"), - print "%-40s %10s\n" % ("----------------------------------------", \ - "-----------"), + print ("%-40s %10s" % ("event", "count")) + print ("%-40s %10s\n" % ("----------------------------------------", + "-----------")) - for id, val in sorted(syscalls.iteritems(), key = lambda(k, v): (v, k), \ + for id, val in sorted(syscalls.items(), key = lambda kv: (kv[1], kv[0]), \ reverse = True): - print "%-40s %10d\n" % (syscall_name(id), val), + print ("%-40s %10d" % (syscall_name(id), val)) ++++++ port-sctop-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port sctop script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Tom Zanussi <[email protected]> --- tools/perf/scripts/python/sctop.py | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) --- a/tools/perf/scripts/python/sctop.py +++ b/tools/perf/scripts/python/sctop.py @@ -8,7 +8,12 @@ # will be refreshed every [interval] seconds. The default interval is # 3 seconds. -import os, sys, thread, time +import os, sys, time + +try: + import thread +except ImportError: + import _thread as thread sys.path.append(os.environ['PERF_EXEC_PATH'] + \ '/scripts/python/Perf-Trace-Util/lib/Perf/Trace') @@ -62,18 +67,19 @@ def print_syscall_totals(interval): while 1: clear_term() if for_comm is not None: - print "\nsyscall events for %s:\n\n" % (for_comm), + print ("\nsyscall events for %s:\n" % (for_comm)) else: - print "\nsyscall events:\n\n", + print ("\nsyscall events:\n") - print "%-40s %10s\n" % ("event", "count"), - print "%-40s %10s\n" % ("----------------------------------------", \ - "----------"), + print ("%-40s %10s" % ("event", "count")) + print ("%-40s %10s" % + ("----------------------------------------", + "----------")) - for id, val in sorted(syscalls.iteritems(), key = lambda(k, v): (v, k), \ + for id, val in sorted(syscalls.items(), key = lambda kv: (kv[1], kv[0]), \ reverse = True): try: - print "%-40s %10d\n" % (syscall_name(id), val), + print ("%-40s %10d" % (syscall_name(id), val)) except TypeError: pass syscalls.clear() ++++++ port-sql-scripts-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port SQL scripts to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> --- tools/perf/scripts/python/export-to-postgresql.py | 28 ++++++++++++---------- tools/perf/scripts/python/export-to-sqlite.py | 23 +++++++++++------- 3 files changed, 41 insertions(+), 32 deletions(-) --- a/tools/perf/scripts/python/export-to-postgresql.py +++ b/tools/perf/scripts/python/export-to-postgresql.py @@ -10,6 +10,7 @@ # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. +from __future__ import print_function import os import sys import struct @@ -236,10 +237,10 @@ perf_db_export_callchains = False def usage(): - print >> sys.stderr, "Usage is: export-to-postgresql.py <database name> [<columns>] [<calls>] [<callchains>]" - print >> sys.stderr, "where: columns 'all' or 'branches'" - print >> sys.stderr, " calls 'calls' => create calls and call_paths table" - print >> sys.stderr, " callchains 'callchains' => create call_paths table" + sys.stderr.write("Usage is: export-to-postgresql.py <database name> [<columns>] [<calls>] [<callchains>]\n") + sys.stderr.write("where: columns 'all' or 'branches'\n") + sys.stderr.write(" calls 'calls' => create calls and call_paths table\n") + sys.stderr.write(" callchains 'callchains' => create call_paths table\n") raise Exception("Too few arguments") if (len(sys.argv) < 2): @@ -273,7 +274,10 @@ def do_query(q, s): return raise Exception("Query failed: " + q.lastError().text()) -print datetime.datetime.today(), "Creating database..." +def dtprint(*args): + print(datetime.datetime.today(), *args, sep=' ') + +dtprint("Creating database...") db = QSqlDatabase.addDatabase('QPSQL') query = QSqlQuery(db) @@ -564,7 +568,7 @@ if perf_db_export_calls: call_file = open_output_file("call_table.bin") def trace_begin(): - print datetime.datetime.today(), "Writing to intermediate files..." + dtprint("Writing to intermediate files...") # id == 0 means unknown. It is easier to create records for them than replace the zeroes with NULLs evsel_table(0, "unknown") machine_table(0, 0, "unknown") @@ -579,7 +583,7 @@ def trace_begin(): unhandled_count = 0 def trace_end(): - print datetime.datetime.today(), "Copying to database..." + dtprint("Copying to database...") copy_output_file(evsel_file, "selected_events") copy_output_file(machine_file, "machines") copy_output_file(thread_file, "threads") @@ -594,7 +598,7 @@ def trace_end(): if perf_db_export_calls: copy_output_file(call_file, "calls") - print datetime.datetime.today(), "Removing intermediate files..." + dtprint("Removing intermediate files...") remove_output_file(evsel_file) remove_output_file(machine_file) remove_output_file(thread_file) @@ -609,7 +613,7 @@ def trace_end(): if perf_db_export_calls: remove_output_file(call_file) os.rmdir(output_dir_name) - print datetime.datetime.today(), "Adding primary keys" + dtprint("Adding primary keys") do_query(query, 'ALTER TABLE selected_events ADD PRIMARY KEY (id)') do_query(query, 'ALTER TABLE machines ADD PRIMARY KEY (id)') do_query(query, 'ALTER TABLE threads ADD PRIMARY KEY (id)') @@ -624,7 +628,7 @@ def trace_end(): if perf_db_export_calls: do_query(query, 'ALTER TABLE calls ADD PRIMARY KEY (id)') - print datetime.datetime.today(), "Adding foreign keys" + dtprint("Adding foreign keys") do_query(query, 'ALTER TABLE threads ' 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),' 'ADD CONSTRAINT processfk FOREIGN KEY (process_id) REFERENCES threads (id)') @@ -659,8 +663,8 @@ def trace_end(): do_query(query, 'CREATE INDEX pcpid_idx ON calls (parent_call_path_id)') if (unhandled_count): - print datetime.datetime.today(), "Warning: ", unhandled_count, " unhandled events" - print datetime.datetime.today(), "Done" + dtprint("Warning: ", unhandled_count, " unhandled events") + dtprint("Done") def trace_unhandled(event_name, context, event_fields_dict): global unhandled_count --- a/tools/perf/scripts/python/export-to-sqlite.py +++ b/tools/perf/scripts/python/export-to-sqlite.py @@ -10,6 +10,8 @@ # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. +from __future__ import print_function + import os import sys import struct @@ -61,10 +63,10 @@ perf_db_export_calls = False perf_db_export_callchains = False def usage(): - print >> sys.stderr, "Usage is: export-to-sqlite.py <database name> [<columns>] [<calls>] [<callchains>]" - print >> sys.stderr, "where: columns 'all' or 'branches'" - print >> sys.stderr, " calls 'calls' => create calls and call_paths table" - print >> sys.stderr, " callchains 'callchains' => create call_paths table" + sys.stderr.write("Usage is: export-to-sqlite.py <database name> [<columns>] [<calls>] [<callchains>]\n"); + sys.stderr.write("where: columns 'all' or 'branches'\n"); + sys.stderr.write(" calls 'calls' => create calls and call_paths table\n"); + sys.stderr.write(" callchains 'callchains' => create call_paths table\n"); raise Exception("Too few arguments") if (len(sys.argv) < 2): @@ -100,7 +102,10 @@ def do_query_(q): return raise Exception("Query failed: " + q.lastError().text()) -print datetime.datetime.today(), "Creating database..." +def dtprint(*args): + print(datetime.datetime.today(), *args, sep=' ') + +dtprint("Creating database ...") db_exists = False try: @@ -376,7 +381,7 @@ if perf_db_export_calls: call_query.prepare("INSERT INTO calls VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)") def trace_begin(): - print datetime.datetime.today(), "Writing records..." + dtprint("Writing records...") do_query(query, 'BEGIN TRANSACTION') # id == 0 means unknown. It is easier to create records for them than replace the zeroes with NULLs evsel_table(0, "unknown") @@ -394,13 +399,13 @@ unhandled_count = 0 def trace_end(): do_query(query, 'END TRANSACTION') - print datetime.datetime.today(), "Adding indexes" + dtprint ("Adding indexes") if perf_db_export_calls: do_query(query, 'CREATE INDEX pcpid_idx ON calls (parent_call_path_id)') if (unhandled_count): - print datetime.datetime.today(), "Warning: ", unhandled_count, " unhandled events" - print datetime.datetime.today(), "Done" + dtprint ("Warning: ", unhandled_count, " unhandled events") + dtprint ("Done") def trace_unhandled(event_name, context, event_fields_dict): global unhandled_count ++++++ port-stackcollapse-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port stackcollapse script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Paolo Bonzini <[email protected]> --- tools/perf/scripts/python/stackcollapse.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- a/tools/perf/scripts/python/stackcollapse.py +++ b/tools/perf/scripts/python/stackcollapse.py @@ -120,7 +120,6 @@ def process_event(param_dict): lines[stack_string] = lines[stack_string] + 1 def trace_end(): - list = lines.keys() - list.sort() + list = sorted(lines) for stack in list: - print "%s %d" % (stack, lines[stack]) + print ("%s %d" % (stack, lines[stack])) ++++++ port-stat-cpi-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port stat-cpi script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Jiri Olsa <[email protected]> --- tools/perf/scripts/python/stat-cpi.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/tools/perf/scripts/python/stat-cpi.py +++ b/tools/perf/scripts/python/stat-cpi.py @@ -19,8 +19,8 @@ def store_key(time, cpu, thread): threads.append(thread) def store(time, event, cpu, thread, val, ena, run): - #print "event %s cpu %d, thread %d, time %d, val %d, ena %d, run %d" % \ - # (event, cpu, thread, time, val, ena, run) + #print ("event %s cpu %d, thread %d, time %d, val %d, ena %d, run %d" % + # (event, cpu, thread, time, val, ena, run)) store_key(time, cpu, thread) key = get_key(time, event, cpu, thread) @@ -58,7 +58,7 @@ def stat__interval(time): if ins != 0: cpi = cyc/float(ins) - print "%15f: cpu %d, thread %d -> cpi %f (%d/%d)" % (time/(float(1000000000)), cpu, thread, cpi, cyc, ins) + print ("%15f: cpu %d, thread %d -> cpi %f (%d/%d)" % (time/(float(1000000000)), cpu, thread, cpi, cyc, ins)) def trace_end(): pass @@ -74,4 +74,4 @@ def trace_end(): # if ins != 0: # cpi = cyc/float(ins) # -# print "time %.9f, cpu %d, thread %d -> cpi %f" % (time/(float(1000000000)), cpu, thread, cpi) +# print ("time %.9f, cpu %d, thread %d -> cpi %f" % (time/(float(1000000000)), cpu, thread, cpi)) ++++++ port-syscall-counts-by-pid-script-to-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: port syscall-counts-by-pid script to Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> --- tools/perf/scripts/python/syscall-counts-by-pid.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) --- a/tools/perf/scripts/python/syscall-counts-by-pid.py +++ b/tools/perf/scripts/python/syscall-counts-by-pid.py @@ -31,7 +31,7 @@ if len(sys.argv) > 1: syscalls = autodict() def trace_begin(): - print "Press control+C to stop and show the summary" + print ("Press control+C to stop and show the summary") def trace_end(): print_syscall_totals() @@ -55,20 +55,20 @@ def syscalls__sys_enter(event_name, cont def print_syscall_totals(): if for_comm is not None: - print "\nsyscall events for %s:\n\n" % (for_comm), + print ("\nsyscall events for %s:\n" % (for_comm)) else: - print "\nsyscall events by comm/pid:\n\n", + print ("\nsyscall events by comm/pid:\n") - print "%-40s %10s\n" % ("comm [pid]/syscalls", "count"), - print "%-40s %10s\n" % ("----------------------------------------", \ - "----------"), + print ("%-40s %10s" % ("comm [pid]/syscalls", "count")) + print ("%-40s %10s" % ("----------------------------------------", + "----------")) comm_keys = syscalls.keys() for comm in comm_keys: pid_keys = syscalls[comm].keys() for pid in pid_keys: - print "\n%s [%d]\n" % (comm, pid), + print ("\n%s [%d]" % (comm, pid)) id_keys = syscalls[comm][pid].keys() - for id, val in sorted(syscalls[comm][pid].iteritems(), \ - key = lambda(k, v): (v, k), reverse = True): - print " %-38s %10d\n" % (syscall_name(id), val), + for id, val in sorted(syscalls[comm][pid].items(), \ + key = lambda kv: (kv[1], kv[0]), reverse = True): + print (" %-38s %10d" % (syscall_name(id), val)) ++++++ remove-shebang-from-scripts-python.patch ++++++ From: Tony Jones <[email protected]> Subject: remove shebang from scripts/python References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Frederic Weisbecker <[email protected]> Cc: Adrian Hunter <[email protected]> Scripts are either invoked by bin/ wrapper, or intended to be run via perf script -s <scriptname>. There is therefore no need for an explicit shebang. Also per pep-0394 /usr/bin/python is recommended to refer only to v2 which may not exist on the system (if PYTHON=python3). --- tools/perf/Makefile.perf | 4 ++-- tools/perf/scripts/python/exported-sql-viewer.py | 1 - tools/perf/scripts/python/sched-migration.py | 2 -- tools/perf/scripts/python/stat-cpi.py | 1 - 4 files changed, 2 insertions(+), 6 deletions(-) --- a/tools/perf/scripts/python/sched-migration.py +++ b/tools/perf/scripts/python/sched-migration.py @@ -1,5 +1,3 @@ -#!/usr/bin/python -# # Cpu task migration overview toy # # Copyright (C) 2010 Frederic Weisbecker <[email protected]> --- a/tools/perf/scripts/python/stat-cpi.py +++ b/tools/perf/scripts/python/stat-cpi.py @@ -1,4 +1,3 @@ -#!/usr/bin/env python # SPDX-License-Identifier: GPL-2.0 data = {} --- a/tools/perf/Makefile.perf +++ b/tools/perf/Makefile.perf @@ -863,8 +863,8 @@ ifndef NO_LIBPYTHON $(call QUIET_INSTALL, python-scripts) \ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'; \ - $(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \ - $(INSTALL) scripts/python/*.py -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \ + $(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \ + $(INSTALL) scripts/python/*.py -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \ $(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin' endif $(call QUIET_INSTALL, perf_completion-script) \ ++++++ remove-shebang-from-tests-attr.py.patch ++++++ From: Tony Jones <[email protected]> Subject: remove shebang from tests/attr.py References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> tests/attr.c invokes attr.py via an explicit invocation of python ($PYTHON). There is therefore no need for an explicit shebang. Also per pep-0394 /usr/bin/python is recommended to refer only to v2 which may not exist on the system (if PYTHON=python3). --- tools/perf/tests/attr.py | 1 - 1 file changed, 1 deletion(-) --- a/tools/perf/tests/attr.py +++ b/tools/perf/tests/attr.py @@ -1,4 +1,3 @@ -#! /usr/bin/python # SPDX-License-Identifier: GPL-2.0 import os ++++++ remove-shebang-from-util-setup.py.patch ++++++ From: Tony Jones <[email protected]> Subject: remove shebang from util/setup.py References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Makefile.perf invokes setup.py via an explicit invocation of python (PYTHON_WORD). There is therefore no need for an explicit shebang. Also per pep-0394 /usr/bin/python is recommended to refer only to v2 which may not exist on the system (if PYTHON=python3). --- tools/perf/util/setup.py | 2 -- 1 file changed, 2 deletions(-) --- a/tools/perf/util/setup.py +++ b/tools/perf/util/setup.py @@ -1,5 +1,3 @@ -#!/usr/bin/python - from os import getenv from subprocess import Popen, PIPE from re import sub ++++++ update-tests-attr.py-to-support-python3.patch ++++++ From: Tony Jones <[email protected]> Subject: Update tests/attr.py to support Python3 References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Jiri Olsa <[email protected]> Update tests/attr.py to support Python3. N.B: Python versions earlier than v2.6 (released 2008) do not support the Python3 "except as" syntax. --- tools/perf/tests/attr.py | 30 +++++++++++++++++------------- 1 file changed, 17 insertions(+), 13 deletions(-) --- a/tools/perf/tests/attr.py +++ b/tools/perf/tests/attr.py @@ -8,7 +8,11 @@ import tempfile import logging import shutil -import ConfigParser + +try: + import configparser +except ImportError: + import ConfigParser as configparser def data_equal(a, b): # Allow multiple values in assignment separated by '|' @@ -100,23 +104,23 @@ def equal(self, other): for t in Event.terms: log.debug(" [%s] %s %s" % (t, self[t], other[t])); - if not self.has_key(t) or not other.has_key(t): + if t not in self or t not in other: return False if not data_equal(self[t], other[t]): return False return True def optional(self): - if self.has_key('optional') and self['optional'] == '1': + if 'optional' in self and self['optional'] == '1': return True return False def diff(self, other): for t in Event.terms: - if not self.has_key(t) or not other.has_key(t): + if t not in self or t not in other: continue if not data_equal(self[t], other[t]): - log.warning("expected %s=%s, got %s" % (t, self[t], other[t])) + log.warning("expected %s=%s, got %s" % (t, self[t], other[t])) # Test file description needs to have following sections: # [config] @@ -134,7 +138,7 @@ # - expected values assignments class Test(object): def __init__(self, path, options): - parser = ConfigParser.SafeConfigParser() + parser = configparser.SafeConfigParser() parser.read(path) log.warning("running '%s'" % path) @@ -193,7 +197,7 @@ return True def load_events(self, path, events): - parser_event = ConfigParser.SafeConfigParser() + parser_event = configparser.SafeConfigParser() parser_event.read(path) # The event record section header contains 'event' word, @@ -207,7 +211,7 @@ # Read parent event if there's any if (':' in section): base = section[section.index(':') + 1:] - parser_base = ConfigParser.SafeConfigParser() + parser_base = configparser.SafeConfigParser() parser_base.read(self.test_dir + '/' + base) base_items = parser_base.items('event') @@ -322,9 +326,9 @@ for f in glob.glob(options.test_dir + '/' + options.test): try: Test(f, options).run() - except Unsup, obj: + except Unsup as obj: log.warning("unsupp %s" % obj.getMsg()) - except Notest, obj: + except Notest as obj: log.warning("skipped %s" % obj.getMsg()) def setup_log(verbose): @@ -363,7 +367,7 @@ parser.add_option("-p", "--perf", action="store", type="string", dest="perf") parser.add_option("-v", "--verbose", - action="count", dest="verbose") + default=0, action="count", dest="verbose") options, args = parser.parse_args() if args: @@ -373,7 +377,7 @@ setup_log(options.verbose) if not options.test_dir: - print 'FAILED no -d option specified' + print('FAILED no -d option specified') sys.exit(-1) if not options.test: @@ -382,8 +386,8 @@ try: run_tests(options) - except Fail, obj: - print "FAILED %s" % obj.getMsg(); + except Fail as obj: + print("FAILED %s" % obj.getMsg()) sys.exit(-1) sys.exit(0) ++++++ use-pybytes-for-attr-in-trace-event-python.patch ++++++ From: Tony Jones <[email protected]> Subject: Use PyBytes for attr in trace-event-python References: bsc#1082197 Upstream: pending submission Signed-off-By: Tony Jones <[email protected]> Cc: Jaroslav Škarvada <[email protected]> Fixes: 66dfdff03d196e51322c6a85c0d8db8bb2bdd655 With Python3. PyUnicode_FromStringAndSize is unsafe to call on attr and will return NULL. Use _PyBytes_FromStringAndSize (as with raw_buf). $ perf script -s perf-script.py -i perf.data in trace_begin Segmentation fault (core dumped) --- tools/perf/util/scripting-engines/trace-event-python.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/tools/perf/util/scripting-engines/trace-event-python.c +++ b/tools/perf/util/scripting-engines/trace-event-python.c @@ -733,8 +733,7 @@ static PyObject *get_perf_sample_dict(st Py_FatalError("couldn't create Python dictionary"); pydict_set_item_string_decref(dict, "ev_name", _PyUnicode_FromString(perf_evsel__name(evsel))); - pydict_set_item_string_decref(dict, "attr", _PyUnicode_FromStringAndSize( - (const char *)&evsel->attr, sizeof(evsel->attr))); + pydict_set_item_string_decref(dict, "attr", _PyBytes_FromStringAndSize((const char *)&evsel->attr, sizeof(evsel->attr))); pydict_set_item_string_decref(dict_sample, "pid", _PyLong_FromLong(sample->pid));
