Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-19 Thread Mathieu Desnoyers
* Frank Ch. Eigler ([EMAIL PROTECTED]) wrote: > Hi - > > On Fri, Jan 18, 2008 at 10:55:27PM -0500, Steven Rostedt wrote: > > [...] > > > All this complexity is to be justified by keeping the raw prev/next > > > pointers from being sent to a naive tracer? It seems to me way out of > > > proportion

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-18 Thread Frank Ch. Eigler
Hi - On Fri, Jan 18, 2008 at 10:55:27PM -0500, Steven Rostedt wrote: > [...] > > All this complexity is to be justified by keeping the raw prev/next > > pointers from being sent to a naive tracer? It seems to me way out of > > proportion. > > Damn, and I just blew away all my marker code for som

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-18 Thread Steven Rostedt
On Fri, 18 Jan 2008, Frank Ch. Eigler wrote: > > All this complexity is to be justified by keeping the raw prev/next > pointers from being sent to a naive tracer? It seems to me way out of > proportion. Damn, and I just blew away all my marker code for something like this ;-) Actually, you jus

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-18 Thread Frank Ch. Eigler
Hi - On Fri, Jan 18, 2008 at 06:19:29PM -0500, Mathieu Desnoyers wrote: > [...] > Almost.. I would add : > > static int trace_switch_to_enabled; > > > static inline trace_switch_to(struct task_struct *prev, > > struct task_struct *next) > > { > if (likely(!trace_switch_to

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-18 Thread Frank Ch. Eigler
Hi - On Fri, Jan 18, 2008 at 05:49:19PM -0500, Steven Rostedt wrote: > [...] > > But I have not seen a lot of situations where that kind of glue-code was > > needed, so I think it makes sense to keep markers simple to use and > > efficient for the common case. > > > > Then, in this glue-code, we c

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-18 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > On Fri, 18 Jan 2008, Mathieu Desnoyers wrote: > > > > But I have not seen a lot of situations where that kind of glue-code was > > needed, so I think it makes sense to keep markers simple to use and > > efficient for the common case. > > > > Then, in th

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-18 Thread Steven Rostedt
On Fri, 18 Jan 2008, Mathieu Desnoyers wrote: > > But I have not seen a lot of situations where that kind of glue-code was > needed, so I think it makes sense to keep markers simple to use and > efficient for the common case. > > Then, in this glue-code, we can put trace_mark() and calls to in-kern

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-18 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > On Thu, 17 Jan 2008, Frank Ch. Eigler wrote: > > > Hi - > > > > On Thu, Jan 17, 2008 at 03:08:33PM -0500, Steven Rostedt wrote: > > > [...] > > > + trace_mark(kernel_sched_schedule, > > > + "prev_pid %d next_pid %d prev_state %ld

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-17 Thread Steven Rostedt
On Thu, 17 Jan 2008, Frank Ch. Eigler wrote: > Hi - > > On Thu, Jan 17, 2008 at 03:08:33PM -0500, Steven Rostedt wrote: > > [...] > > + trace_mark(kernel_sched_schedule, > > + "prev_pid %d next_pid %d prev_state %ld", > > + prev->pid, next->pid, prev->state); > >

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-17 Thread Frank Ch. Eigler
Hi - On Thu, Jan 17, 2008 at 03:08:33PM -0500, Steven Rostedt wrote: > [...] > + trace_mark(kernel_sched_schedule, > + "prev_pid %d next_pid %d prev_state %ld", > + prev->pid, next->pid, prev->state); > [...] > But... > > Tracers that want to do a bit more work,

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-17 Thread Steven Rostedt
> > > > > > > > One thing I want to clear up. The major difference between this > > > > latency_tracer and LTTng is what we consider fast paths. The latency > > > > tracer is recording things like enabling and disabling interrupts, > > > > preempt > > > > count changes, or simply profiling all fu

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-17 Thread Linus Torvalds
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > Or could we map a per-thread page that would contradict this > "definition" ? Over my dead body. It's been done before. Many times. It's horrible, and means that you need to flush the TLB on context switches between threads and cannot share th

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-17 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > On Thu, 17 Jan 2008, Paul Mackerras wrote: > > > > It's very hard to do a per-thread counter in the VDSO, since threads > > in the same process see the same memory, by definition. You'd have to > > have an array of counters and have some way for eac

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-17 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > On Thu, 17 Jan 2008, Paul Mackerras wrote: > > > > It's very hard to do a per-thread counter in the VDSO, since threads > > in the same process see the same memory, by definition. You'd have to > > have an array of counters and have some way for eac

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-17 Thread Steven Rostedt
> > Crazy ideas : > > Could we do something along the lines of the thread local storage ? > > Or could we map a per-thread page that would contradict this > "definition" ? When working on lguest64, I implemented a "per CPU" shadow page. That the process of a guest running on one real CPU, could n

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Paul Mackerras ([EMAIL PROTECTED]) wrote: > Mathieu Desnoyers writes: > > > Sorry for self-reply, but I thought, in the past, of a way to make this > > possible. > > > > It would imply the creation of a new vsyscall : vgetschedperiod > > > > It would read a counter that would increment each ti

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Thu, 17 Jan 2008, Paul Mackerras wrote: > > It's very hard to do a per-thread counter in the VDSO, since threads > in the same process see the same memory, by definition. You'd have to > have an array of counters and have some way for each thread to know > which entry to read. Also you'd have

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Paul Mackerras
Mathieu Desnoyers writes: > Sorry for self-reply, but I thought, in the past, of a way to make this > possible. > > It would imply the creation of a new vsyscall : vgetschedperiod > > It would read a counter that would increment each time the thread is > scheduled out (or in). It would be a per

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > It would imply the creation of a new vsyscall : vgetschedperiod > > It would read a counter that would increment each time the thread is > scheduled out (or in). It would be a per thread counter (not a per cpu > counter) so we can deal appropriately

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Mathieu Desnoyers ([EMAIL PROTECTED]) wrote: > * john stultz ([EMAIL PROTECTED]) wrote: > > > > On Wed, 2008-01-16 at 18:33 -0500, Steven Rostedt wrote: > > > Thanks John for doing this! > > > > > > (comments imbedded) > > > > > > On Wed, 16 Jan 2008, john stultz wrote: > > > > + int num

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > > > Yep. clocksource_get_cycles() ended up not being as useful as an helper > > function (I was hoping the arch vsyscall implementations could use it, > > but they've done too much optimization - although that may reflect a > > need up the chain to

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* john stultz ([EMAIL PROTECTED]) wrote: > > On Wed, 2008-01-16 at 18:33 -0500, Steven Rostedt wrote: > > Thanks John for doing this! > > > > (comments imbedded) > > > > On Wed, 16 Jan 2008, john stultz wrote: > > > + int num = !cs->base_num; > > > + cycle_t offset = (now - cs->base[!num].cycle_

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* john stultz ([EMAIL PROTECTED]) wrote: > On Wed, 2008-01-16 at 18:39 -0500, Mathieu Desnoyers wrote: > > I would disable preemption in clocksource_get_basecycles. We would not > > want to be scheduled out while we hold a pointer to the old array > > element. > > > > > + int num = cs->base_num; >

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread john stultz
On Wed, 2008-01-16 at 18:33 -0500, Steven Rostedt wrote: > Thanks John for doing this! > > (comments imbedded) > > On Wed, 16 Jan 2008, john stultz wrote: > > + int num = !cs->base_num; > > + cycle_t offset = (now - cs->base[!num].cycle_base_last); > > + offset &= cs->mask; > > + cs->bas

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* john stultz ([EMAIL PROTECTED]) wrote: > > On Wed, 2008-01-16 at 18:39 -0500, Mathieu Desnoyers wrote: > > * john stultz ([EMAIL PROTECTED]) wrote: > > > > > > On Wed, 2008-01-16 at 14:36 -0800, john stultz wrote: > > > > On Jan 16, 2008 6:56 AM, Mathieu Desnoyers <[EMAIL PROTECTED]> wrote: > >

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread john stultz
On Wed, 2008-01-16 at 18:39 -0500, Mathieu Desnoyers wrote: > I would disable preemption in clocksource_get_basecycles. We would not > want to be scheduled out while we hold a pointer to the old array > element. > > > + int num = cs->base_num; > > Since you deal with base_num in a shared manner

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Linus Torvalds ([EMAIL PROTECTED]) wrote: > > > On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > > > > + int num = !cs->base_num; > > > + cycle_t offset = (now - cs->base[!num].cycle_base_last); > > > > !0 is not necessarily 1. > > Incorrect. > Hrm, *digging in my mailbox*, ah, here it is

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Linus Torvalds
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > > + int num = !cs->base_num; > > + cycle_t offset = (now - cs->base[!num].cycle_base_last); > > !0 is not necessarily 1. Incorrect. !0 _is_ necessarily 1. It's how all C logical operators work. If you find a compiler that turns !x into any

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Steven Rostedt wrote: > On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > > > !0 is not necessarily 1. This is why I use cpu_synth->index ? 0 : 1 in > > How about simply "cpu_synth->index ^ 1"? Seems the best choice if you ask > me, if all you are doing is changing it from 1

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread john stultz
On Wed, 2008-01-16 at 18:39 -0500, Mathieu Desnoyers wrote: > * john stultz ([EMAIL PROTECTED]) wrote: > > > > On Wed, 2008-01-16 at 14:36 -0800, john stultz wrote: > > > On Jan 16, 2008 6:56 AM, Mathieu Desnoyers <[EMAIL PROTECTED]> wrote: > > > > If you really want an seqlock free algorithm (I

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > > - cycle_t offset = (now - cs->cycle_last) & cs->mask; > > + /* First update the monotonic base portion. > > +* The dual array update method allows for lock-free reading. > > +*/ > > + int num = !cs->base_num; > > + cycle_t offset

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* john stultz ([EMAIL PROTECTED]) wrote: > > On Wed, 2008-01-16 at 14:36 -0800, john stultz wrote: > > On Jan 16, 2008 6:56 AM, Mathieu Desnoyers <[EMAIL PROTECTED]> wrote: > > > If you really want an seqlock free algorithm (I _do_ want this for > > > tracing!) :) maybe going in the RCU direction

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
Thanks John for doing this! (comments imbedded) On Wed, 16 Jan 2008, john stultz wrote: > > On Wed, 2008-01-16 at 14:36 -0800, john stultz wrote: > > Completely un-tested, but it builds, so I figured I'd send it out for > review. heh, ok, I'll take it and run it. > > I'm not super sure the up

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread john stultz
On Wed, 2008-01-16 at 14:36 -0800, john stultz wrote: > On Jan 16, 2008 6:56 AM, Mathieu Desnoyers <[EMAIL PROTECTED]> wrote: > > If you really want an seqlock free algorithm (I _do_ want this for > > tracing!) :) maybe going in the RCU direction could help (I refer to my > > RCU-based 32-to-64 bi

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread john stultz
On Jan 16, 2008 6:56 AM, Mathieu Desnoyers <[EMAIL PROTECTED]> wrote: > If you really want an seqlock free algorithm (I _do_ want this for > tracing!) :) maybe going in the RCU direction could help (I refer to my > RCU-based 32-to-64 bits lockless timestamp counter extension, which > could be turne

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > > > In-other-words, latency_tracer is LTTng-lite ;-) > > > > If LTTng is already ported to your specific kernel, the learning-curve > is not big at all. Here is what the latency_tracer over LTTng guide > could look like : > > Well, once you have LT

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Tim Bird
Mathieu Desnoyers wrote: > If LTTng is already ported to your specific kernel, the learning-curve > is not big at all. Here is what the latency_tracer over LTTng guide > could look like : > > Well, once you have LTTng in your kernel and have compiled and installed > the ltt-control and lttv packag

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > ... > > > > > > > > > > - Disable preemption at the read-side : > > > > it makes sure the pointer I get will point to a data structure that > > > > will never change while I am in the preempt disabled code. (see *) > > > > - I use per-cpu data to

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > * Steven Rostedt ([EMAIL PROTECTED]) wrote: > > > > Yeah, but if we replace the loop with a seq lock, then it would work. > > albeit, more cacheline bouncing (caused by writes). (maybe not, see below) > > > > Yes, but then you would trigger a deadloc

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Tim Bird
Steven Rostedt wrote: > grmble. Then how do you trace preempt_disable? As my tracer does that > (see the last patch in the series). One way is to make a tracer_preempt_disable() and tracer_preempt_enable(), both of which would be 'notrace'. You could probably optimize them as well. The standard

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Mathieu Desnoyers ([EMAIL PROTECTED]) wrote: > * Steven Rostedt ([EMAIL PROTECTED]) wrote: > > > > > > > One thing I want to clear up. The major difference between this > > latency_tracer and LTTng is what we consider fast paths. The latency > > tracer is recording things like enabling

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > > On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > > No, there's probably issues there too, but no need to worry about it, > > > since I already showed that allowing for clocksource_accumulate to happen > > > inside the get_monotonic_cycles loop is

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > No, there's probably issues there too, but no need to worry about it, > > since I already showed that allowing for clocksource_accumulate to happen > > inside the get_monotonic_cycles loop is already flawed. > > > > Yep, I just re-read through you

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > > Hrm, I will reply to the rest of this email in a separate mail, but > > there is another concern, simpler than memory ordering, that just hit > > me : > > > > If we have CPU A calling clocksource_accu

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
On Wed, 16 Jan 2008, Mathieu Desnoyers wrote: > Hrm, I will reply to the rest of this email in a separate mail, but > there is another concern, simpler than memory ordering, that just hit > me : > > If we have CPU A calling clocksource_accumulate while CPU B is calling > get_monotonic_cycles, but

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > [ CC'd Daniel Walker, since he had problems with this code ] > > On Tue, 15 Jan 2008, Mathieu Desnoyers wrote: > > > > I agree with you that I don't see how the compiler could reorder this. > > So we forget about compiler barriers. Also, the clock s

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-16 Thread Steven Rostedt
[ CC'd Daniel Walker, since he had problems with this code ] On Tue, 15 Jan 2008, Mathieu Desnoyers wrote: > > I agree with you that I don't see how the compiler could reorder this. > So we forget about compiler barriers. Also, the clock source used is a > synchronized clock source (get_cycles_sy

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-15 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > On Tue, 15 Jan 2008, Mathieu Desnoyers wrote: > > > > Ok, but what actually insures that the clock->cycle_* reads won't be > > reordered across the clocksource_read() ? > > > > Hmm, interesting.I didn't notice that clocksource_read() is a static >

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-15 Thread Steven Rostedt
On Tue, 15 Jan 2008, Mathieu Desnoyers wrote: > > Ok, but what actually insures that the clock->cycle_* reads won't be > reordered across the clocksource_read() ? Hmm, interesting.I didn't notice that clocksource_read() is a static inline. I was thinking that since it was passing a pointer to

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-15 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > > > On Tue, 15 Jan 2008, Mathieu Desnoyers wrote: > > > > > > Signed-off-by: Steven Rostedt <[EMAIL PROTECTED]> > > > --- > > > include/linux/clocksource.h |3 ++ > > > kernel/time/timekeeping.c | 48 > > > +++

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-15 Thread Steven Rostedt
On Tue, 15 Jan 2008, Steven Rostedt wrote: > > Also, it just occurred to me that this is an old patch. I thought I > renamed cycle_raw to cycle_monotonic. But I must have lost that patch :-/ Ah, I changed this in the -rt patch queue, and never moved the patch back here. -- Steve -- To unsubscri

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-15 Thread Steven Rostedt
On Tue, 15 Jan 2008, Mathieu Desnoyers wrote: > > > > Signed-off-by: Steven Rostedt <[EMAIL PROTECTED]> > > --- > > include/linux/clocksource.h |3 ++ > > kernel/time/timekeeping.c | 48 > > > > 2 files changed, 51 insertions(+) > > > > Index

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-15 Thread Mathieu Desnoyers
* Steven Rostedt ([EMAIL PROTECTED]) wrote: > The latency tracer needs a way to get an accurate time > without grabbing any locks. Locks themselves might call > the latency tracer and cause at best a slow down. > > This patch adds get_monotonic_cycles that returns cycles > from a reliable clock so

Re: [RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-09 Thread Daniel Walker
On Wed, 2008-01-09 at 18:29 -0500, Steven Rostedt wrote: > +cycle_t notrace get_monotonic_cycles(void) > +{ > + cycle_t cycle_now, cycle_delta, cycle_raw, cycle_last; > + > + do { > + /* > +* cycle_raw and cycle_last can change on > +* anot

[RFC PATCH 16/22 -v2] add get_monotonic_cycles

2008-01-09 Thread Steven Rostedt
The latency tracer needs a way to get an accurate time without grabbing any locks. Locks themselves might call the latency tracer and cause at best a slow down. This patch adds get_monotonic_cycles that returns cycles from a reliable clock source in a monotonic fashion. Signed-off-by: Steven Rost