Re: [PATCH 11/11] perf scripts python: Add event_analyzing_sample.py as a sample for general event handling
Hi Arnaldo and Ingo,
On Wed, 8 Aug 2012 14:13:48 -0300, Arnaldo Carvalho de Melo wrote:
> From: Feng Tang
>
> Currently only trace point events are supported in perf/python script,
> the first 3 patches of this serie add the support for all types of
> events. This script is just a simple sample to show how to gather the
> basic information of the events and analyze them.
>
> This script will create one object for each event sample and insert them
> into a table in a database, then leverage the simple SQL commands to
> sort/group them. User can modify or write their brand new functions
> according to their specific requirment.
>
> Here is the sample of how to use the script:
>
> $ perf record -a tree
> $ perf script -s process_event.py
Please edit the script name to event_analyzing_sample.py at least to
prevent future confusion. For other issues, please see my review
comments on Feng's original posts. (They can be incremental.)
>
> There is 100 records in gen_events table
> Statistics about the general events grouped by thread/symbol/dso:
>
> comm number histgram
> ==
> swapper 56 ##
> tree 20 #
> perf 10
> sshd8
> kworker/7:24 ###
> ksoftirqd/71 #
> plugin-containe1 #
>
> symbol number histgram
> ==
>native_write_msr_safe 40 ##
> __lock_acquire8
> ftrace_graph_caller4 ###
>prepare_ftrace_return4 ###
> intel_idle3 ##
> native_sched_clock3 ##
> Unknown_symbol2 ##
> do_softirq2 ##
> lock_release2 ##
>lock_release_holdtime2 ##
>trace_graph_entry2 ##
> _IO_putc1 #
> __d_lookup_rcu1 #
> __do_fault1 #
> __schedule1 #
> _raw_spin_lock1 #
>delay_tsc1 #
> generic_exec_single1 #
> generic_fillattr1 #
>
> dso number histgram
> ==
>[kernel.kallsyms] 95 ###
> /lib/libc-2.12.1.so5 ###
>
> Signed-off-by: Feng Tang
> Cc: Andi Kleen
> Cc: David Ahern
> Cc: Ingo Molnar
> Cc: Peter Zijlstra
> Cc: Robert Richter
> Cc: Stephane Eranian
> Link:
> http://lkml.kernel.org/r/1344419875-21665-6-git-send-email-feng.t...@intel.com
> Signed-off-by: Arnaldo Carvalho de Melo
> ---
> .../perf/scripts/python/event_analyzing_sample.py | 193
>
> 1 file changed, 193 insertions(+)
> create mode 100644 tools/perf/scripts/python/event_analyzing_sample.py
>
> diff --git a/tools/perf/scripts/python/event_analyzing_sample.py
> b/tools/perf/scripts/python/event_analyzing_sample.py
> new file mode 100644
> index 000..46f05aa
> --- /dev/null
> +++ b/tools/perf/scripts/python/event_analyzing_sample.py
> @@ -0,0 +1,193 @@
> +# process_event.py: general event handler in python
Hopefully here also.
Thanks,
Namhyung
> +#
> +# Current perf report is alreay very powerful with the anotation integrated,
> +# and this script is not trying to be as powerful as perf report, but
> +# providing end user/developer a flexible way to analyze the events other
> +# than trace points.
> +#
> +# The 2 database related functions in this script just show how to gather
> +# the basic information, and users can modify and write their own functions
> +# according to their specific requirment.
> +#
> +# The first sample "show_general_events" just does a baisc grouping for all
> +# generic events with the help of sqlite, and the 2nd one "show_pebs_ll" is
> +# for a x86 HW PMU event: PEBS with load latency data.
> +#
> +
> +import os
> +import sys
> +import math
> +import struct
> +import sqlite3
> +
> +sys.path.append(os.environ['PERF_EXEC_PATH'] + \
> +'/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
> +
> +from perf_trace_context import *
> +from EventClass import *
> +
> +#
> +# If the perf.data has a big number of samples, then the insert operation
> +# will be very time consuming (about 10+ minutes for 1 samples) if the
> +# .db database is on disk. Move the .db file to RAM based FS to speedup
> +# the handling, which will cut the time down to several seconds.
> +#
> +con = sqlite3.connect("/dev/shm/perf.db")
> +con.isolation_level = None
> +
> +def trace_begin():
> +
[PATCH 11/11] perf scripts python: Add event_analyzing_sample.py as a sample for general event handling
From: Feng Tang
Currently only trace point events are supported in perf/python script,
the first 3 patches of this serie add the support for all types of
events. This script is just a simple sample to show how to gather the
basic information of the events and analyze them.
This script will create one object for each event sample and insert them
into a table in a database, then leverage the simple SQL commands to
sort/group them. User can modify or write their brand new functions
according to their specific requirment.
Here is the sample of how to use the script:
$ perf record -a tree
$ perf script -s process_event.py
There is 100 records in gen_events table
Statistics about the general events grouped by thread/symbol/dso:
comm number histgram
==
swapper 56 ##
tree 20 #
perf 10
sshd8
kworker/7:24 ###
ksoftirqd/71 #
plugin-containe1 #
symbol number histgram
==
native_write_msr_safe 40 ##
__lock_acquire8
ftrace_graph_caller4 ###
prepare_ftrace_return4 ###
intel_idle3 ##
native_sched_clock3 ##
Unknown_symbol2 ##
do_softirq2 ##
lock_release2 ##
lock_release_holdtime2 ##
trace_graph_entry2 ##
_IO_putc1 #
__d_lookup_rcu1 #
__do_fault1 #
__schedule1 #
_raw_spin_lock1 #
delay_tsc1 #
generic_exec_single1 #
generic_fillattr1 #
dso number histgram
==
[kernel.kallsyms] 95 ###
/lib/libc-2.12.1.so5 ###
Signed-off-by: Feng Tang
Cc: Andi Kleen
Cc: David Ahern
Cc: Ingo Molnar
Cc: Peter Zijlstra
Cc: Robert Richter
Cc: Stephane Eranian
Link:
http://lkml.kernel.org/r/1344419875-21665-6-git-send-email-feng.t...@intel.com
Signed-off-by: Arnaldo Carvalho de Melo
---
.../perf/scripts/python/event_analyzing_sample.py | 193
1 file changed, 193 insertions(+)
create mode 100644 tools/perf/scripts/python/event_analyzing_sample.py
diff --git a/tools/perf/scripts/python/event_analyzing_sample.py
b/tools/perf/scripts/python/event_analyzing_sample.py
new file mode 100644
index 000..46f05aa
--- /dev/null
+++ b/tools/perf/scripts/python/event_analyzing_sample.py
@@ -0,0 +1,193 @@
+# process_event.py: general event handler in python
+#
+# Current perf report is alreay very powerful with the anotation integrated,
+# and this script is not trying to be as powerful as perf report, but
+# providing end user/developer a flexible way to analyze the events other
+# than trace points.
+#
+# The 2 database related functions in this script just show how to gather
+# the basic information, and users can modify and write their own functions
+# according to their specific requirment.
+#
+# The first sample "show_general_events" just does a baisc grouping for all
+# generic events with the help of sqlite, and the 2nd one "show_pebs_ll" is
+# for a x86 HW PMU event: PEBS with load latency data.
+#
+
+import os
+import sys
+import math
+import struct
+import sqlite3
+
+sys.path.append(os.environ['PERF_EXEC_PATH'] + \
+'/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
+
+from perf_trace_context import *
+from EventClass import *
+
+#
+# If the perf.data has a big number of samples, then the insert operation
+# will be very time consuming (about 10+ minutes for 1 samples) if the
+# .db database is on disk. Move the .db file to RAM based FS to speedup
+# the handling, which will cut the time down to several seconds.
+#
+con = sqlite3.connect("/dev/shm/perf.db")
+con.isolation_level = None
+
+def trace_begin():
+ print "In trace_begin:\n"
+
+#
+# Will create several tables at the start, pebs_ll is for PEBS data
with
+# load latency info, while gen_events is for general event.
+#
+con.execute("""
+create table if not exists gen_events (
+name text,
+symbol text,
+comm text,
+dso text
+);""")
+con.execute("""
+create table if not exists pebs_ll (
+
[PATCH 11/11] perf scripts python: Add event_analyzing_sample.py as a sample for general event handling
From: Feng Tang feng.t...@intel.com Currently only trace point events are supported in perf/python script, the first 3 patches of this serie add the support for all types of events. This script is just a simple sample to show how to gather the basic information of the events and analyze them. This script will create one object for each event sample and insert them into a table in a database, then leverage the simple SQL commands to sort/group them. User can modify or write their brand new functions according to their specific requirment. Here is the sample of how to use the script: $ perf record -a tree $ perf script -s process_event.py There is 100 records in gen_events table Statistics about the general events grouped by thread/symbol/dso: comm number histgram == swapper 56 ## tree 20 # perf 10 sshd8 kworker/7:24 ### ksoftirqd/71 # plugin-containe1 # symbol number histgram == native_write_msr_safe 40 ## __lock_acquire8 ftrace_graph_caller4 ### prepare_ftrace_return4 ### intel_idle3 ## native_sched_clock3 ## Unknown_symbol2 ## do_softirq2 ## lock_release2 ## lock_release_holdtime2 ## trace_graph_entry2 ## _IO_putc1 # __d_lookup_rcu1 # __do_fault1 # __schedule1 # _raw_spin_lock1 # delay_tsc1 # generic_exec_single1 # generic_fillattr1 # dso number histgram == [kernel.kallsyms] 95 ### /lib/libc-2.12.1.so5 ### Signed-off-by: Feng Tang feng.t...@intel.com Cc: Andi Kleen a...@firstfloor.org Cc: David Ahern dsah...@gmail.com Cc: Ingo Molnar mi...@elte.hu Cc: Peter Zijlstra pet...@infradead.org Cc: Robert Richter robert.rich...@amd.com Cc: Stephane Eranian eran...@google.com Link: http://lkml.kernel.org/r/1344419875-21665-6-git-send-email-feng.t...@intel.com Signed-off-by: Arnaldo Carvalho de Melo a...@redhat.com --- .../perf/scripts/python/event_analyzing_sample.py | 193 1 file changed, 193 insertions(+) create mode 100644 tools/perf/scripts/python/event_analyzing_sample.py diff --git a/tools/perf/scripts/python/event_analyzing_sample.py b/tools/perf/scripts/python/event_analyzing_sample.py new file mode 100644 index 000..46f05aa --- /dev/null +++ b/tools/perf/scripts/python/event_analyzing_sample.py @@ -0,0 +1,193 @@ +# process_event.py: general event handler in python +# +# Current perf report is alreay very powerful with the anotation integrated, +# and this script is not trying to be as powerful as perf report, but +# providing end user/developer a flexible way to analyze the events other +# than trace points. +# +# The 2 database related functions in this script just show how to gather +# the basic information, and users can modify and write their own functions +# according to their specific requirment. +# +# The first sample show_general_events just does a baisc grouping for all +# generic events with the help of sqlite, and the 2nd one show_pebs_ll is +# for a x86 HW PMU event: PEBS with load latency data. +# + +import os +import sys +import math +import struct +import sqlite3 + +sys.path.append(os.environ['PERF_EXEC_PATH'] + \ +'/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + +from perf_trace_context import * +from EventClass import * + +# +# If the perf.data has a big number of samples, then the insert operation +# will be very time consuming (about 10+ minutes for 1 samples) if the +# .db database is on disk. Move the .db file to RAM based FS to speedup +# the handling, which will cut the time down to several seconds. +# +con = sqlite3.connect(/dev/shm/perf.db) +con.isolation_level = None + +def trace_begin(): + print In trace_begin:\n + +# +# Will create several tables at the start, pebs_ll is for PEBS data with +# load latency info, while gen_events is for general event. +# +con.execute( +create table if not exists gen_events ( +name text, +symbol text, +comm text, +
Re: [PATCH 11/11] perf scripts python: Add event_analyzing_sample.py as a sample for general event handling
Hi Arnaldo and Ingo, On Wed, 8 Aug 2012 14:13:48 -0300, Arnaldo Carvalho de Melo wrote: From: Feng Tang feng.t...@intel.com Currently only trace point events are supported in perf/python script, the first 3 patches of this serie add the support for all types of events. This script is just a simple sample to show how to gather the basic information of the events and analyze them. This script will create one object for each event sample and insert them into a table in a database, then leverage the simple SQL commands to sort/group them. User can modify or write their brand new functions according to their specific requirment. Here is the sample of how to use the script: $ perf record -a tree $ perf script -s process_event.py Please edit the script name to event_analyzing_sample.py at least to prevent future confusion. For other issues, please see my review comments on Feng's original posts. (They can be incremental.) There is 100 records in gen_events table Statistics about the general events grouped by thread/symbol/dso: comm number histgram == swapper 56 ## tree 20 # perf 10 sshd8 kworker/7:24 ### ksoftirqd/71 # plugin-containe1 # symbol number histgram == native_write_msr_safe 40 ## __lock_acquire8 ftrace_graph_caller4 ### prepare_ftrace_return4 ### intel_idle3 ## native_sched_clock3 ## Unknown_symbol2 ## do_softirq2 ## lock_release2 ## lock_release_holdtime2 ## trace_graph_entry2 ## _IO_putc1 # __d_lookup_rcu1 # __do_fault1 # __schedule1 # _raw_spin_lock1 # delay_tsc1 # generic_exec_single1 # generic_fillattr1 # dso number histgram == [kernel.kallsyms] 95 ### /lib/libc-2.12.1.so5 ### Signed-off-by: Feng Tang feng.t...@intel.com Cc: Andi Kleen a...@firstfloor.org Cc: David Ahern dsah...@gmail.com Cc: Ingo Molnar mi...@elte.hu Cc: Peter Zijlstra pet...@infradead.org Cc: Robert Richter robert.rich...@amd.com Cc: Stephane Eranian eran...@google.com Link: http://lkml.kernel.org/r/1344419875-21665-6-git-send-email-feng.t...@intel.com Signed-off-by: Arnaldo Carvalho de Melo a...@redhat.com --- .../perf/scripts/python/event_analyzing_sample.py | 193 1 file changed, 193 insertions(+) create mode 100644 tools/perf/scripts/python/event_analyzing_sample.py diff --git a/tools/perf/scripts/python/event_analyzing_sample.py b/tools/perf/scripts/python/event_analyzing_sample.py new file mode 100644 index 000..46f05aa --- /dev/null +++ b/tools/perf/scripts/python/event_analyzing_sample.py @@ -0,0 +1,193 @@ +# process_event.py: general event handler in python Hopefully here also. Thanks, Namhyung +# +# Current perf report is alreay very powerful with the anotation integrated, +# and this script is not trying to be as powerful as perf report, but +# providing end user/developer a flexible way to analyze the events other +# than trace points. +# +# The 2 database related functions in this script just show how to gather +# the basic information, and users can modify and write their own functions +# according to their specific requirment. +# +# The first sample show_general_events just does a baisc grouping for all +# generic events with the help of sqlite, and the 2nd one show_pebs_ll is +# for a x86 HW PMU event: PEBS with load latency data. +# + +import os +import sys +import math +import struct +import sqlite3 + +sys.path.append(os.environ['PERF_EXEC_PATH'] + \ +'/scripts/python/Perf-Trace-Util/lib/Perf/Trace') + +from perf_trace_context import * +from EventClass import * + +# +# If the perf.data has a big number of samples, then the insert operation +# will be very time consuming (about 10+ minutes for 1 samples) if the +# .db database is on disk. Move the .db file to RAM based FS to speedup +# the handling, which will cut the time down to several seconds. +# +con = sqlite3.connect(/dev/shm/perf.db) +con.isolation_level
