On Tue, Aug 9, 2016 at 12:24 PM, Brenden Blanco <[email protected]> wrote:
>
>
> On Tue, Aug 9, 2016 at 9:16 AM, riya khanna <[email protected]>
> wrote:
>>
>> On Tue, Aug 9, 2016 at 12:05 PM, Brenden Blanco <[email protected]>
>> wrote:
>> > On Tue, Aug 9, 2016 at 8:54 AM, riya khanna <[email protected]>
>> > wrote:
>> >>
>> >> Hi Brenden,
>> >>
>> >> Saw test_perf_event.py in your branch. Its creates and enables per
>> >> counters once during start. Is it also possible to
>> >> enable/disable/reset counters on the fly? Perhaps we need a kernel
>> >> patch for this?
>> >
>> > It doesn't "create" counters, it just attaches to the already available
>> > counters provided by the hardware or OS.
>>
>> Yes, it enables monitoring when attached.
>>
>> > Any type of "reset" infrastructure
>> > would adversely impact other users of those same counters (perf). I
>> > consider
>> > it the job of userspace or the program to compute deltas or other types
>> > of
>> > history.
>>
>> Well, there are limited counters. How to multiplex from userspace on
>> the fly (e.g. monitoring a set of events first, followed by a
>> different set)?
>
> I would just create a different BPF_PERF_ARRAY for each different one.
Yes, but if you create more than available number of hardware counters
(i.e. try to monitor more events concurrently than allowed by the
hardware), counter value is reported as '0'. Verified with a test
program (perf.c, attached) that uses perf_event_open() syscall.
Depending upon the number of counters available on your platform,
change NUM_REQ_HW_CNTRS to verify the behavior
A way to enable/disable events at runtime will help userspace
multiplex over available hardware counters and monitor more more
events (similar to perf stat tool)
>>
>> Also, is it possible to handle counter overflow?
>
> What does it mean to "handle"? If computing deltas, for instance, the
> subtraction will just underflow and wrap around to the correct value,
> assuming the values are both unsigned.
>>
>>
>> >>
>> >>
>> >> Thanks,
>> >> Riya
>> >>
>> >> On Fri, Jul 29, 2016 at 1:57 PM, riya khanna <[email protected]>
>> >> wrote:
>> >> > Thanks Brenden!
>> >> >
>> >> > I'm working with your branch for now. Additionally, I'm unable to
>> >> > create software events (see exception below). Just wanted to bring
>> >> > this to your attention.
>> >> >
>> >> > Traceback (most recent call last):
>> >> >
>> >> > File "./test_bpf.py", line 176, in <module>
>> >> > sw_clock.open_perf_event(1, 0)
>> >> >
>> >> > File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 410, in
>> >> > open_perf_event
>> >> > fd = self._open_perf_event(typ, config, i)
>> >> > File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 416, in
>> >> > _open_perf_event
>> >> > self[self.Key(cpu)] = self.Leaf(fd)
>> >> > File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 320, in
>> >> > __setitem__
>> >> > super(ArrayBase, self).__setitem__(key, leaf)
>> >> > File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 169, in
>> >> > __setitem__
>> >> > raise Exception("Could not update table")
>> >> > Exception: Could not update table
>> >> >
>> >> > On Fri, Jul 29, 2016 at 1:34 PM, Brenden Blanco
>> >> > <[email protected]>
>> >> > wrote:
>> >> >> On Fri, Jul 29, 2016 at 10:21 AM, riya khanna
>> >> >> <[email protected]>
>> >> >> wrote:
>> >> >>>
>> >> >>> I'm testing perf counters on a 8-core machine.
>> >> >>>
>> >> >>> since BPF_PERF_ARRAY.perf_read(cpu) reads from local CPU, I'm
>> >> >>> aggregating counters across all cpus by doing:
>> >> >>>
>> >> >>> BPF_PERF_ARRAY(counter, 32);
>> >> >>>
>> >> >>> for (key = 0; key < 8; key++)
>> >> >>> counter.perf_read(key);
>> >> >>
>> >> >>
>> >> >> I think it would make more sense to only read the counter on the cpu
>> >> >> where
>> >> >> the event is taking place. So:
>> >> >>
>> >> >> u64 key = cycles.perf_read(bpf_get_smp_processor_id());
>> >> >>
>> >> >> And then aggregate counters in userspace.
>> >> >>
>> >> >> I have spent some time over the past couple days cleaning up the
>> >> >> code
>> >> >> in
>> >> >> that private branch, but have been distracted a bit so haven't
>> >> >> finalized it.
>> >> >> Hopefully a PR will come soon.
>> >> >>
>> >> >>>
>> >> >>> However, this reports error:
>> >> >>>
>> >> >>> bpf: Invalid argument
>> >> >>> back-edge from insn 69 to 17
>> >> >>>
>> >> >>> If I loop from 0-4, it works. The code below works:
>> >> >>> for (key = 0; key < 4; key++)
>> >> >>> counter.perf_read(key);
>> >> >>>
>> >> >>>
>> >> >>> What could be wrong here?
>> >> >>
>> >> >> The kernel verifier won't allow loops (i.e. back edges), and
>> >> >> depending
>> >> >> on
>> >> >> the loop unroll optimization decision made by llvm, this short loop
>> >> >> may
>> >> >> have
>> >> >> been automatically unrolled. Still, the solution should be to remove
>> >> >> the
>> >> >> loop and just read the local cpu's perf counter as mentioned above.
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On Tue, Jul 26, 2016 at 7:29 PM, riya khanna
>> >> >>> <[email protected]>
>> >> >>> wrote:
>> >> >>> > From your patches I see that perf support is enabled per-cpu.
>> >> >>> > Could
>> >> >>> > this be extended to enabling all or a group of perf counters on
>> >> >>> > all
>> >> >>> > CPU cores similar to what perf_event_open provides (with args
>> >> >>> > -1)?
>> >> >>> >
>> >> >>> > On Mon, Jul 25, 2016 at 9:55 PM, riya khanna
>> >> >>> > <[email protected]>
>> >> >>> > wrote:
>> >> >>> >> Thanks Brenden!
>> >> >>> >>
>> >> >>> >> I will try with your changes. Meanwhile please let me know if
>> >> >>> >> you
>> >> >>> >> add
>> >> >>> >> missing functionality.
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> On Mon, Jul 25, 2016 at 8:14 PM, Brenden Blanco
>> >> >>> >> <[email protected]>
>> >> >>> >> wrote:
>> >> >>> >>> This needs support in bcc.
>> >> >>> >>>
>> >> >>> >>> I had a patch laying around that I never finished, you can find
>> >> >>> >>> the
>> >> >>> >>> partial
>> >> >>> >>> support here:
>> >> >>> >>> https://github.com/iovisor/bcc/tree/perf-counter
>> >> >>> >>>
>> >> >>> >>> It shouldn't be too hard to finalize that, let me see what I
>> >> >>> >>> can
>> >> >>> >>> do.
>> >> >>> >>>
>> >> >>> >>> On Mon, Jul 25, 2016 at 4:11 PM, riya khanna via iovisor-dev
>> >> >>> >>> <[email protected]> wrote:
>> >> >>> >>>>
>> >> >>> >>>> So I fixed the error above by using "count =
>> >> >>> >>>> my_map.perf_read(key);"
>> >> >>> >>>> as opposed to "count = bpf_perf_event_read(&my_map, key);".
>> >> >>> >>>> However,
>> >> >>> >>>> how do I selectively enable counters (e.g. instructions, cache
>> >> >>> >>>> misses,
>> >> >>> >>>> etc.)?
>> >> >>> >>>>
>> >> >>> >>>> Thanks,
>> >> >>> >>>> Riya
>> >> >>> >>>>
>> >> >>> >>>> On Mon, Jul 25, 2016 at 9:58 AM, riya khanna
>> >> >>> >>>> <[email protected]>
>> >> >>> >>>> wrote:
>> >> >>> >>>> > Hi,
>> >> >>> >>>> >
>> >> >>> >>>> > I'm trying to read perf counters using bpf. However, adding
>> >> >>> >>>> > BPF_PERF_ARRAY reports error:
>> >> >>> >>>> >
>> >> >>> >>>> > bpf: Invalid argument
>> >> >>> >>>> > unrecognized bpf_ld_imm64 inns
>> >> >>> >>>> >
>> >> >>> >>>> > Is there an example/sample to read perf counters that I can
>> >> >>> >>>> > follow?
>> >> >>> >>>> > The code below is what I'm trying to execute.
>> >> >>> >>>> >
>> >> >>> >>>> > Thanks,
>> >> >>> >>>> > Riya
>> >> >>> >>>> >
>> >> >>> >>>> > # load BPF program
>> >> >>> >>>> >
>> >> >>> >>>> > bpf_text = """
>> >> >>> >>>> >
>> >> >>> >>>> > #include <uapi/linux/ptrace.h>
>> >> >>> >>>> >
>> >> >>> >>>> > BPF_PERF_ARRAY(my_map, 32);
>> >> >>> >>>> >
>> >> >>> >>>> > int start_counting(struct pt_regs *ctx) {
>> >> >>> >>>> >
>> >> >>> >>>> > if (!PT_REGS_PARM1(ctx))
>> >> >>> >>>> >
>> >> >>> >>>> > return 0;
>> >> >>> >>>> >
>> >> >>> >>>> > u64 count;
>> >> >>> >>>> >
>> >> >>> >>>> > u32 key = bpf_get_smp_processor_id();
>> >> >>> >>>> >
>> >> >>> >>>> > count = bpf_perf_event_read(&my_map, key);
>> >> >>> >>>> >
>> >> >>> >>>> > bpf_trace_printk("CPU-%d %llu", key, count);
>> >> >>> >>>> >
>> >> >>> >>>> > return 0;
>> >> >>> >>>> >
>> >> >>> >>>> > }
>> >> >>> >>>> >
>> >> >>> >>>> > """
>> >> >>> >>>> _______________________________________________
>> >> >>> >>>> iovisor-dev mailing list
>> >> >>> >>>> [email protected]
>> >> >>> >>>> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>> >> >>> >>>
>> >> >>> >>>
>> >> >>
>> >> >>
>> >
>> >
>
>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/ioctl.h>
#include <linux/perf_event.h>
#include <asm/unistd.h>
#include <stdint.h>
#include <errno.h>
struct perf_event {
const char *name;
uint64_t type;
uint64_t config;
};
#define NUM_REQ_HW_CNTRS 6
static const struct perf_event events[] =
{
{ "Cycles", PERF_TYPE_HARDWARE, PERF_COUNT_HW_CPU_CYCLES },
{ "Instrs", PERF_TYPE_HARDWARE, PERF_COUNT_HW_INSTRUCTIONS },
{ "Cache-Refs", PERF_TYPE_HARDWARE, PERF_COUNT_HW_CACHE_REFERENCES },
{ "Cache-Misses", PERF_TYPE_HARDWARE, PERF_COUNT_HW_CACHE_MISSES },
{ "Branches", PERF_TYPE_HARDWARE, PERF_COUNT_HW_BRANCH_INSTRUCTIONS },
#if NUM_REQ_HW_CNTRS >= 6
{ "Branch-Misses", PERF_TYPE_HARDWARE, PERF_COUNT_HW_BRANCH_MISSES },
#endif
#if NUM_REQ_HW_CNTRS >= 7
{ "Bus-Cycles", PERF_TYPE_HARDWARE, PERF_COUNT_HW_BUS_CYCLES },
#endif
#if NUM_REQ_HW_CNTRS >= 8
{ "L1D Reads", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_L1D |
(PERF_COUNT_HW_CACHE_OP_READ << 8) |
(PERF_COUNT_HW_CACHE_RESULT_ACCESS << 16)) },
#endif
#if NUM_REQ_HW_CNTRS >= 9
{ "L1D Read Misses", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_L1D |
(PERF_COUNT_HW_CACHE_OP_READ << 8) |
(PERF_COUNT_HW_CACHE_RESULT_MISS << 16)) },
#if 0
{ "L1D Prefetches", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_L1D |
(PERF_COUNT_HW_CACHE_OP_PREFETCH << 8) |
(PERF_COUNT_HW_CACHE_RESULT_ACCESS << 16)) },
{ "L1D Prefetch Misses", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_L1D |
(PERF_COUNT_HW_CACHE_OP_PREFETCH << 8) |
(PERF_COUNT_HW_CACHE_RESULT_MISS << 16)) },
#endif
{ "LL Reads", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_LL |
(PERF_COUNT_HW_CACHE_OP_READ << 8) |
(PERF_COUNT_HW_CACHE_RESULT_ACCESS << 16)) },
{ "LL Read Misses", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_LL |
(PERF_COUNT_HW_CACHE_OP_READ << 8) |
(PERF_COUNT_HW_CACHE_RESULT_MISS << 16)) },
{ "LL Prefetches", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_LL |
(PERF_COUNT_HW_CACHE_OP_PREFETCH << 8) |
(PERF_COUNT_HW_CACHE_RESULT_ACCESS << 16)) },
{ "L1D Prefetche Misses", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_LL |
(PERF_COUNT_HW_CACHE_OP_PREFETCH << 8) |
(PERF_COUNT_HW_CACHE_RESULT_MISS << 16)) },
{ "DTLB Reads", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_DTLB |
(PERF_COUNT_HW_CACHE_OP_READ << 8) |
(PERF_COUNT_HW_CACHE_RESULT_ACCESS << 16)) },
{ "DTLB Read Misses", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_DTLB |
(PERF_COUNT_HW_CACHE_OP_READ << 8) |
(PERF_COUNT_HW_CACHE_RESULT_MISS << 16)) },
#if 0
{ "DTLB Prefetches", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_DTLB |
(PERF_COUNT_HW_CACHE_OP_PREFETCH << 8) |
(PERF_COUNT_HW_CACHE_RESULT_ACCESS << 16)) },
{ "DTLB Prefetch Misses", PERF_TYPE_HW_CACHE, (PERF_COUNT_HW_CACHE_DTLB |
(PERF_COUNT_HW_CACHE_OP_PREFETCH << 8) |
(PERF_COUNT_HW_CACHE_RESULT_MISS << 16)) },
#endif
#endif
{ "Cpu-Clock", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_CPU_CLOCK },
{ "Task-Clock", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_TASK_CLOCK },
{ "Page-Faults", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_PAGE_FAULTS },
{ "Ctxt-Switches", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_CONTEXT_SWITCHES },
{ "Cpu-Migrations", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_CPU_MIGRATIONS },
{ "Page-Faults-Minor", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_PAGE_FAULTS_MIN },
{ "Page-Faults-Major", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_PAGE_FAULTS_MAJ },
{ "Align-Faults", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_ALIGNMENT_FAULTS },
{ "Emu-Faults", PERF_TYPE_SOFTWARE, PERF_COUNT_SW_EMULATION_FAULTS },
};
static long perf_event_open(struct perf_event_attr *hw_event, pid_t pid,
int cpu, int group_fd, unsigned long flags)
{
int ret;
ret = syscall(__NR_perf_event_open, hw_event, pid, cpu,
group_fd, flags);
return ret;
}
static int open_event(uint64_t config, uint64_t type, int group_leader)
{
static struct perf_event_attr pe;
pe.size = sizeof(struct perf_event_attr);
pe.type = type;
pe.config = config;
pe.exclude_kernel = 1;
pe.exclude_hv = 1;
pe.mmap = 1;
pe.comm = 1;
if (group_leader == -1)
pe.disabled = 1;
pe.read_format = PERF_FORMAT_GROUP;
return perf_event_open(&pe, 0, -1, group_leader, 0);
}
void perf_enable_perf(int fd)
{
if (fd > 0) {
ioctl(fd, PERF_EVENT_IOC_RESET, PERF_IOC_FLAG_GROUP);
ioctl(fd, PERF_EVENT_IOC_ENABLE, PERF_IOC_FLAG_GROUP);
} else
fprintf(stderr, "enable failed fd: %d", fd);
}
void perf_disable_perf(int fd)
{
if (fd > 0) {
ioctl(fd, PERF_EVENT_IOC_DISABLE, PERF_IOC_FLAG_GROUP);
int num_events = sizeof(events) / sizeof(events[0]);
uint64_t buf[num_events + 1];
memset(buf, 0, ((num_events + 1) * sizeof(uint64_t)));
if (read(fd, &buf, ((num_events + 1) * sizeof(uint64_t))) < 0) {
fprintf(stderr, "fd: %d read failed: %s", fd, strerror(errno));
} else {
//fprintf(stdout, "fd: %d nr_events %ld\n", fd, buf[0]);
int idx;
for (idx = 0; idx < buf[0]; idx++)
fprintf(stdout, "%s: %ld\n", events[idx].name, buf[idx + 1]);
}
} else
fprintf(stderr, "disable failed fd: %d", fd);
}
int perf_init_perf(void)
{
int idx, fd, group_fd = -1;
int num_events = sizeof(events) / sizeof(events[0]);
for (idx = 0; idx < num_events; idx++) {
fd = open_event(events[idx].config, events[idx].type, group_fd);
if (fd == -1) {
fprintf(stderr, "Error opening event %s : %s\n",
events[idx].name, strerror(errno));
return -1;
} else {
//fprintf(stdout, "event %d open fd %d\n", idx, fd);
}
if (group_fd == -1)
group_fd = fd;
}
return group_fd;
}
void perf_exit_perf(int fd)
{
if (fd > 0)
close(fd);
}
int
main(int argc, char **argv)
{
int fd = perf_init_perf();
if (fd < 0)
return -1;
perf_enable_perf(fd);
printf("Measuring instruction count for this printf\n");
perf_disable_perf(fd);
perf_exit_perf(fd);
}
_______________________________________________
iovisor-dev mailing list
[email protected]
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
