Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-07-09 Thread Boris Ostrovsky
On 06/28/2018 06:43 AM, Thomas Gleixner wrote: > > For enhanced fun the early calibration stuff was moved from right after > init_hypervisor_platform() to the place where it is now in commit > ccb64941f375a6 ("x86/timers: Move simple_udelay_calibration() past > kvmclock_init()") to speed up KVM

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-07-09 Thread Boris Ostrovsky
On 06/28/2018 06:43 AM, Thomas Gleixner wrote: > > For enhanced fun the early calibration stuff was moved from right after > init_hypervisor_platform() to the place where it is now in commit > ccb64941f375a6 ("x86/timers: Move simple_udelay_calibration() past > kvmclock_init()") to speed up KVM

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-07-03 Thread Thomas Gleixner
On Fri, 29 Jun 2018, Pavel Tatashin wrote: > On 18-06-29 09:30:10, Thomas Gleixner wrote: > For now, in my series, I would like check in tsc_early_init() if we are > running xen, and set tsc_khz to 0 if so. Later in tsc_init() we will get > a proper calibration. The question is how to make this

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-07-03 Thread Thomas Gleixner
On Fri, 29 Jun 2018, Pavel Tatashin wrote: > On 18-06-29 09:30:10, Thomas Gleixner wrote: > For now, in my series, I would like check in tsc_early_init() if we are > running xen, and set tsc_khz to 0 if so. Later in tsc_init() we will get > a proper calibration. The question is how to make this

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-07-02 Thread Konrad Rzeszutek Wilk
On Fri, Jun 29, 2018 at 09:30:10AM +0200, Thomas Gleixner wrote: > On Thu, 28 Jun 2018, Pavel Tatashin wrote: > > On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > > Hi Thomas, > > > > In addition to above, we have xen hvm: > > > > setup_arch() > > ... > >

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-07-02 Thread Konrad Rzeszutek Wilk
On Fri, Jun 29, 2018 at 09:30:10AM +0200, Thomas Gleixner wrote: > On Thu, 28 Jun 2018, Pavel Tatashin wrote: > > On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > > Hi Thomas, > > > > In addition to above, we have xen hvm: > > > > setup_arch() > > ... > >

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Andy Shevchenko
On Fri, 2018-06-29 at 17:30 +0300, Andy Shevchenko wrote: > On Thu, 2018-06-28 at 12:43 +0200, Thomas Gleixner wrote: > > On Thu, 28 Jun 2018, Thomas Gleixner wrote: > > > > Taking above into consideration, I think we may just remove the legacy > code from mfld.c and mrfld.c and see what happen.

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Andy Shevchenko
On Fri, 2018-06-29 at 17:30 +0300, Andy Shevchenko wrote: > On Thu, 2018-06-28 at 12:43 +0200, Thomas Gleixner wrote: > > On Thu, 28 Jun 2018, Thomas Gleixner wrote: > > > > Taking above into consideration, I think we may just remove the legacy > code from mfld.c and mrfld.c and see what happen.

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Andy Shevchenko
On Thu, 2018-06-28 at 12:43 +0200, Thomas Gleixner wrote: > On Thu, 28 Jun 2018, Thomas Gleixner wrote: > > I still want to document the unholy mess of what is initialized and > > available when. We have 5 hypervisors and 3 different points in > > early boot > > where the calibrate_* callbacks are

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Andy Shevchenko
On Thu, 2018-06-28 at 12:43 +0200, Thomas Gleixner wrote: > On Thu, 28 Jun 2018, Thomas Gleixner wrote: > > I still want to document the unholy mess of what is initialized and > > available when. We have 5 hypervisors and 3 different points in > > early boot > > where the calibrate_* callbacks are

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Pavel Tatashin
On 18-06-29 09:30:10, Thomas Gleixner wrote: > On Thu, 28 Jun 2018, Pavel Tatashin wrote: > > On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > > Hi Thomas, > > > > In addition to above, we have xen hvm: > > > > setup_arch() > > ... > > init_hypervisor_platform(); > >

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Pavel Tatashin
On 18-06-29 09:30:10, Thomas Gleixner wrote: > On Thu, 28 Jun 2018, Pavel Tatashin wrote: > > On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > > Hi Thomas, > > > > In addition to above, we have xen hvm: > > > > setup_arch() > > ... > > init_hypervisor_platform(); > >

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Thomas Gleixner
On Thu, 28 Jun 2018, Pavel Tatashin wrote: > On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > Hi Thomas, > > In addition to above, we have xen hvm: > > setup_arch() > ... > init_hypervisor_platform(); > x86_init.hyper.init_platform(); > xen_hvm_guest_init()

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-29 Thread Thomas Gleixner
On Thu, 28 Jun 2018, Pavel Tatashin wrote: > On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > Hi Thomas, > > In addition to above, we have xen hvm: > > setup_arch() > ... > init_hypervisor_platform(); > x86_init.hyper.init_platform(); > xen_hvm_guest_init()

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Pavel Tatashin
On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > > On Thu, 28 Jun 2018, Thomas Gleixner wrote: > > I still want to document the unholy mess of what is initialized and > > available when. We have 5 hypervisors and 3 different points in early boot > > where the calibrate_* callbacks are

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Pavel Tatashin
On Thu, Jun 28, 2018 at 11:23 AM Thomas Gleixner wrote: > > On Thu, 28 Jun 2018, Thomas Gleixner wrote: > > I still want to document the unholy mess of what is initialized and > > available when. We have 5 hypervisors and 3 different points in early boot > > where the calibrate_* callbacks are

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Thomas Gleixner
On Thu, 28 Jun 2018, Peter Zijlstra wrote: > On Thu, Jun 28, 2018 at 12:43:59PM +0200, Thomas Gleixner wrote: > > init_hypervisor_platform() > >vmware: > >Retrieves frequency and store it for the > >calibration function > > > >khz =

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Thomas Gleixner
On Thu, 28 Jun 2018, Peter Zijlstra wrote: > On Thu, Jun 28, 2018 at 12:43:59PM +0200, Thomas Gleixner wrote: > > init_hypervisor_platform() > >vmware: > >Retrieves frequency and store it for the > >calibration function > > > >khz =

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Peter Zijlstra
On Thu, Jun 28, 2018 at 12:43:59PM +0200, Thomas Gleixner wrote: > init_hypervisor_platform() > vmware: > Retrieves frequency and store it for the > calibration function > > khz = vmware_get_khz_magic() >

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Peter Zijlstra
On Thu, Jun 28, 2018 at 12:43:59PM +0200, Thomas Gleixner wrote: > init_hypervisor_platform() > vmware: > Retrieves frequency and store it for the > calibration function > > khz = vmware_get_khz_magic() >

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Thomas Gleixner
On Thu, 28 Jun 2018, Thomas Gleixner wrote: > I still want to document the unholy mess of what is initialized and > available when. We have 5 hypervisors and 3 different points in early boot > where the calibrate_* callbacks are overwritten. The XEN PV one is actually > post tsc_init_early() for

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Thomas Gleixner
On Thu, 28 Jun 2018, Thomas Gleixner wrote: > I still want to document the unholy mess of what is initialized and > available when. We have 5 hypervisors and 3 different points in early boot > where the calibrate_* callbacks are overwritten. The XEN PV one is actually > post tsc_init_early() for

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Thomas Gleixner
On Tue, 26 Jun 2018, Pavel Tatashin wrote: > On Tue, Jun 26, 2018 at 11:44 AM Thomas Gleixner wrote: > > I have an idea how to distangle it and we'll end up in a staged approach, > > which looks like this: > > > > 1) Earliest one (not sure how early yet) > > > >Attempt to use

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-28 Thread Thomas Gleixner
On Tue, 26 Jun 2018, Pavel Tatashin wrote: > On Tue, Jun 26, 2018 at 11:44 AM Thomas Gleixner wrote: > > I have an idea how to distangle it and we'll end up in a staged approach, > > which looks like this: > > > > 1) Earliest one (not sure how early yet) > > > >Attempt to use

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-26 Thread Pavel Tatashin
On Tue, Jun 26, 2018 at 2:42 PM Pavel Tatashin wrote: > > Hi Thomas, > > On Tue, Jun 26, 2018 at 11:44 AM Thomas Gleixner wrote: > > > > Pavel, > > > > first of all, sorry for my last outburst. I just was in a lousy mood after > > staring into too much half baken stuff and failed to make myself

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-26 Thread Pavel Tatashin
On Tue, Jun 26, 2018 at 2:42 PM Pavel Tatashin wrote: > > Hi Thomas, > > On Tue, Jun 26, 2018 at 11:44 AM Thomas Gleixner wrote: > > > > Pavel, > > > > first of all, sorry for my last outburst. I just was in a lousy mood after > > staring into too much half baken stuff and failed to make myself

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-26 Thread Pavel Tatashin
Hi Thomas, On Tue, Jun 26, 2018 at 11:44 AM Thomas Gleixner wrote: > > Pavel, > > first of all, sorry for my last outburst. I just was in a lousy mood after > staring into too much half baken stuff and failed to make myself stay away > from the computer. Thank you. > > On Sun, 24 Jun 2018,

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-26 Thread Pavel Tatashin
Hi Thomas, On Tue, Jun 26, 2018 at 11:44 AM Thomas Gleixner wrote: > > Pavel, > > first of all, sorry for my last outburst. I just was in a lousy mood after > staring into too much half baken stuff and failed to make myself stay away > from the computer. Thank you. > > On Sun, 24 Jun 2018,

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-26 Thread Thomas Gleixner
Pavel, first of all, sorry for my last outburst. I just was in a lousy mood after staring into too much half baken stuff and failed to make myself stay away from the computer. On Sun, 24 Jun 2018, Thomas Gleixner wrote: > On Sat, 23 Jun 2018, Pavel Tatashin wrote: > And this early init sequence

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-26 Thread Thomas Gleixner
Pavel, first of all, sorry for my last outburst. I just was in a lousy mood after staring into too much half baken stuff and failed to make myself stay away from the computer. On Sun, 24 Jun 2018, Thomas Gleixner wrote: > On Sat, 23 Jun 2018, Pavel Tatashin wrote: > And this early init sequence

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-24 Thread Thomas Gleixner
On Sat, 23 Jun 2018, Pavel Tatashin wrote: > > > As soon as sched_clock() starts output non-zero values, we start > > > output time without correcting the output as it is done in > > > sched_clock_local() where unstable TSC and backward motion are > > > detected. But, since early in boot

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-24 Thread Thomas Gleixner
On Sat, 23 Jun 2018, Pavel Tatashin wrote: > > > As soon as sched_clock() starts output non-zero values, we start > > > output time without correcting the output as it is done in > > > sched_clock_local() where unstable TSC and backward motion are > > > detected. But, since early in boot

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Pavel Tatashin
Hi Thomas, Thank you for your feedback. My comments below: > > As soon as sched_clock() starts output non-zero values, we start > > output time without correcting the output as it is done in > > sched_clock_local() where unstable TSC and backward motion are > > detected. But, since early in boot

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Pavel Tatashin
Hi Thomas, Thank you for your feedback. My comments below: > > As soon as sched_clock() starts output non-zero values, we start > > output time without correcting the output as it is done in > > sched_clock_local() where unstable TSC and backward motion are > > detected. But, since early in boot

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Thomas Gleixner
On Sat, 23 Jun 2018, Pavel Tatashin wrote: > > So you forgot to answer this question. I did not find a system yet, which > > actually exposes this behaviour on mainline. > > > > Is this an artifact of your early sched clock thing? > > > > Yes, it is. Let me explain how it happens. > > So, the

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Thomas Gleixner
On Sat, 23 Jun 2018, Pavel Tatashin wrote: > > So you forgot to answer this question. I did not find a system yet, which > > actually exposes this behaviour on mainline. > > > > Is this an artifact of your early sched clock thing? > > > > Yes, it is. Let me explain how it happens. > > So, the

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Pavel Tatashin
> So you forgot to answer this question. I did not find a system yet, which > actually exposes this behaviour on mainline. > > Is this an artifact of your early sched clock thing? > Yes, it is. Let me explain how it happens. So, the problem is introduced in patch "sched: early boot clock" by

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Pavel Tatashin
> So you forgot to answer this question. I did not find a system yet, which > actually exposes this behaviour on mainline. > > Is this an artifact of your early sched clock thing? > Yes, it is. Let me explain how it happens. So, the problem is introduced in patch "sched: early boot clock" by

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Thomas Gleixner
On Sat, 23 Jun 2018, Thomas Gleixner wrote: > On Thu, 21 Jun 2018, Pavel Tatashin wrote: > > We will change sched_clock() to be called early. > > Why is this relevant? Does the issue only appear with that change? So you forgot to answer this question. I did not find a system yet, which actually

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Thomas Gleixner
On Sat, 23 Jun 2018, Thomas Gleixner wrote: > On Thu, 21 Jun 2018, Pavel Tatashin wrote: > > We will change sched_clock() to be called early. > > Why is this relevant? Does the issue only appear with that change? So you forgot to answer this question. I did not find a system yet, which actually

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Pavel Tatashin
> Let me give you an example: > > When tsc_init() enables the usage of TSC for sched_clock() the > initialization of the tsc sched clock conversion data starts from zero > and not from the current jiffies based sched_clock() value. This makes > the timestamps jump backwards: > > [

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Pavel Tatashin
> Let me give you an example: > > When tsc_init() enables the usage of TSC for sched_clock() the > initialization of the tsc sched clock conversion data starts from zero > and not from the current jiffies based sched_clock() value. This makes > the timestamps jump backwards: > > [

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Thomas Gleixner
On Thu, 21 Jun 2018, Pavel Tatashin wrote: > We will change sched_clock() to be called early. Why is this relevant? Does the issue only appear with that change? > But, during boot sched_clock() changes its output without notifying us > about the change of clock source. Why would the sched clock

Re: [PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-23 Thread Thomas Gleixner
On Thu, 21 Jun 2018, Pavel Tatashin wrote: > We will change sched_clock() to be called early. Why is this relevant? Does the issue only appear with that change? > But, during boot sched_clock() changes its output without notifying us > about the change of clock source. Why would the sched clock

[PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-21 Thread Pavel Tatashin
We will change sched_clock() to be called early. But, during boot sched_clock() changes its output without notifying us about the change of clock source. This happens in tsc_init(), when static_branch_enable(&__use_tsc) is called. native_sched_clock() changes from outputing jiffies to reading

[PATCH v12 09/11] x86/tsc: prepare for early sched_clock

2018-06-21 Thread Pavel Tatashin
We will change sched_clock() to be called early. But, during boot sched_clock() changes its output without notifying us about the change of clock source. This happens in tsc_init(), when static_branch_enable(&__use_tsc) is called. native_sched_clock() changes from outputing jiffies to reading