Re: [PATCH] BUG in pv_clock when overflow condition is detected
On 02/20/2012 04:28 PM, Konrad Rzeszutek Wilk wrote:
On Fri, Feb 17, 2012 at 04:25:04PM +0100, Igor Mammedov wrote:
On 02/16/2012 03:03 PM, Avi Kivity wrote:
On 02/15/2012 07:18 PM, Igor Mammedov wrote:
On 02/15/2012 01:23 PM, Igor Mammedov wrote:
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time
*shadow)
{
-u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+u64 delta;
+u64 tsc = native_read_tsc();
+BUG_ON(tscshadow-tsc_timestamp);
+delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor
bug can
kill the guest.
An attempt to print from this place is not perfect since it often
leads
to recursive calling to this very function and it hang there
anyway.
But if you insist I'll re-post it with WARN_ON_ONCE,
It won't make much difference because guest will hang/stall due
overflow
anyway.
Won't a BUG_ON() also result in a printk?
Yes, it will. But stack will still keep failure point and poking
with crash/gdb at core will always show where it's BUGged.
In case it manages to print dump somehow (saw it couple times from ~
30 test cycles), logs from console or from kernel message buffer
(again poking with gdb) will show where it was called from.
If WARN* is used, it will still totaly screwup clock and
last value and system will become unusable, requiring looking with
gdb/crash at the core any way.
So I've just used more stable failure point that will leave trace
everywhere it manages (maybe in console log, but for sure in stack)
in case of WARN it might leave trace on console or not and probably
won't reflect failure point in stack either leaving only kernel
message buffer for clue.
Makes sense. But do get an ack from the Xen people to ensure this
doesn't break for them.
Konrad, Ian
Could you please review patch form point of view of xen?
Whole thread could be found here https://lkml.org/lkml/2012/2/13/286
What are the conditions under which this happens?
You should probably include that in the git description as well?
This happens on cpu hot-plug in kvm guest:
https://lkml.org/lkml/2012/2/7/222
It probably doesn't affect xen pv guest but issue might affect hvm one.
I'm certainly not xen expert to say it for sure after a cursory look
at the code. If you can confirm that it affects xen hvm I will write
early_percpu_clock_init patch for it as well.
Is this something that happens often?
Very seldom and unlikely.
Hm, so are you asking for review for this patch
I was asking for review of subj patch
BUG in pv_clock when overflow condition is detected
I'll update patch description and re-spin it.
If there is an overflow can you synthesize a value instead of
crashing the guest?
or for http://www.spinics.net/lists/kvm/msg68440.html ?
Probably could, but there was argument that it is fixing the symptoms
and not the root cause. It seems that you've already found patch that
proposes this pvclock: Make pv_clock more robust and fixup it if overflow
happens
(which would also entail a early_percpu_clock_init implementation
in the Xen code naturally).
--
Thanks,
Igor
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] BUG in pv_clock when overflow condition is detected
On Fri, Feb 17, 2012 at 04:25:04PM +0100, Igor Mammedov wrote:
On 02/16/2012 03:03 PM, Avi Kivity wrote:
On 02/15/2012 07:18 PM, Igor Mammedov wrote:
On 02/15/2012 01:23 PM, Igor Mammedov wrote:
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time
*shadow)
{
-u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+u64 delta;
+u64 tsc = native_read_tsc();
+BUG_ON(tsc shadow-tsc_timestamp);
+delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor
bug can
kill the guest.
An attempt to print from this place is not perfect since it often
leads
to recursive calling to this very function and it hang there
anyway.
But if you insist I'll re-post it with WARN_ON_ONCE,
It won't make much difference because guest will hang/stall due
overflow
anyway.
Won't a BUG_ON() also result in a printk?
Yes, it will. But stack will still keep failure point and poking
with crash/gdb at core will always show where it's BUGged.
In case it manages to print dump somehow (saw it couple times from ~
30 test cycles), logs from console or from kernel message buffer
(again poking with gdb) will show where it was called from.
If WARN* is used, it will still totaly screwup clock and
last value and system will become unusable, requiring looking with
gdb/crash at the core any way.
So I've just used more stable failure point that will leave trace
everywhere it manages (maybe in console log, but for sure in stack)
in case of WARN it might leave trace on console or not and probably
won't reflect failure point in stack either leaving only kernel
message buffer for clue.
Makes sense. But do get an ack from the Xen people to ensure this
doesn't break for them.
Konrad, Ian
Could you please review patch form point of view of xen?
Whole thread could be found here https://lkml.org/lkml/2012/2/13/286
What are the conditions under which this happens? You should probably
include that in the git description as well? Is this something that happens
often? If there is an overflow can you synthesize a value instead of
crashing the guest?
Hm, so are you asking for review for this patch or for
http://www.spinics.net/lists/kvm/msg68440.html ?
(which would also entail a early_percpu_clock_init implementation
in the Xen code naturally).
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] BUG in pv_clock when overflow condition is detected
On 02/16/2012 03:03 PM, Avi Kivity wrote:
On 02/15/2012 07:18 PM, Igor Mammedov wrote:
On 02/15/2012 01:23 PM, Igor Mammedov wrote:
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time
*shadow)
{
-u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+u64 delta;
+u64 tsc = native_read_tsc();
+BUG_ON(tsc shadow-tsc_timestamp);
+delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor
bug can
kill the guest.
An attempt to print from this place is not perfect since it often
leads
to recursive calling to this very function and it hang there
anyway.
But if you insist I'll re-post it with WARN_ON_ONCE,
It won't make much difference because guest will hang/stall due
overflow
anyway.
Won't a BUG_ON() also result in a printk?
Yes, it will. But stack will still keep failure point and poking
with crash/gdb at core will always show where it's BUGged.
In case it manages to print dump somehow (saw it couple times from ~
30 test cycles), logs from console or from kernel message buffer
(again poking with gdb) will show where it was called from.
If WARN* is used, it will still totaly screwup clock and
last value and system will become unusable, requiring looking with
gdb/crash at the core any way.
So I've just used more stable failure point that will leave trace
everywhere it manages (maybe in console log, but for sure in stack)
in case of WARN it might leave trace on console or not and probably
won't reflect failure point in stack either leaving only kernel
message buffer for clue.
Makes sense. But do get an ack from the Xen people to ensure this
doesn't break for them.
Konrad, Ian
Could you please review patch form point of view of xen?
Whole thread could be found here https://lkml.org/lkml/2012/2/13/286
--
Thanks,
Igor
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] BUG in pv_clock when overflow condition is detected
On 02/15/2012 07:18 PM, Igor Mammedov wrote:
On 02/15/2012 01:23 PM, Igor Mammedov wrote:
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time
*shadow)
{
-u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+u64 delta;
+u64 tsc = native_read_tsc();
+BUG_ON(tsc shadow-tsc_timestamp);
+delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor
bug can
kill the guest.
An attempt to print from this place is not perfect since it often
leads
to recursive calling to this very function and it hang there
anyway.
But if you insist I'll re-post it with WARN_ON_ONCE,
It won't make much difference because guest will hang/stall due
overflow
anyway.
Won't a BUG_ON() also result in a printk?
Yes, it will. But stack will still keep failure point and poking
with crash/gdb at core will always show where it's BUGged.
In case it manages to print dump somehow (saw it couple times from ~
30 test cycles), logs from console or from kernel message buffer
(again poking with gdb) will show where it was called from.
If WARN* is used, it will still totaly screwup clock and
last value and system will become unusable, requiring looking with
gdb/crash at the core any way.
So I've just used more stable failure point that will leave trace
everywhere it manages (maybe in console log, but for sure in stack)
in case of WARN it might leave trace on console or not and probably
won't reflect failure point in stack either leaving only kernel
message buffer for clue.
Makes sense. But do get an ack from the Xen people to ensure this
doesn't break for them.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] BUG in pv_clock when overflow condition is detected
On 02/13/2012 08:20 PM, Igor Mammedov wrote:
BUG when overflow occurs at pvclock.c:pvclock_get_nsec_offset
u64 delta = native_read_tsc() - shadow-tsc_timestamp;
this might happen at an attempt to read an uninitialized yet clock.
It won't prevent stalls and hangs but at least it won't do it silently.
Signed-off-by: Igor Mammedov imamm...@redhat.com
---
arch/x86/kernel/pvclock.c |5 -
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c
index 42eb330..35a6190 100644
--- a/arch/x86/kernel/pvclock.c
+++ b/arch/x86/kernel/pvclock.c
@@ -43,7 +43,10 @@ void pvclock_set_flags(u8 flags)
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time *shadow)
{
- u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+ u64 delta;
+ u64 tsc = native_read_tsc();
+ BUG_ON(tsc shadow-tsc_timestamp);
+ delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor bug can
kill the guest.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] BUG in pv_clock when overflow condition is detected
On 02/15/2012 11:49 AM, Avi Kivity wrote:
On 02/13/2012 08:20 PM, Igor Mammedov wrote:
BUG when overflow occurs at pvclock.c:pvclock_get_nsec_offset
u64 delta = native_read_tsc() - shadow-tsc_timestamp;
this might happen at an attempt to read an uninitialized yet clock.
It won't prevent stalls and hangs but at least it won't do it silently.
Signed-off-by: Igor Mammedovimamm...@redhat.com
---
arch/x86/kernel/pvclock.c |5 -
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c
index 42eb330..35a6190 100644
--- a/arch/x86/kernel/pvclock.c
+++ b/arch/x86/kernel/pvclock.c
@@ -43,7 +43,10 @@ void pvclock_set_flags(u8 flags)
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time *shadow)
{
- u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+ u64 delta;
+ u64 tsc = native_read_tsc();
+ BUG_ON(tsc shadow-tsc_timestamp);
+ delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor bug can
kill the guest.
An attempt to print from this place is not perfect since it often leads
to recursive calling to this very function and it hang there anyway.
But if you insist I'll re-post it with WARN_ON_ONCE,
It won't make much difference because guest will hang/stall due overflow
anyway.
If there is an intention to keep guest functional after the event then
maybe this patch is a way to go
http://www.spinics.net/lists/kvm/msg68463.html
this way clock will be re-silent to this kind of errors, like bare-metal
one is.
--
Thanks,
Igor
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] BUG in pv_clock when overflow condition is detected
On 02/15/2012 01:23 PM, Igor Mammedov wrote:
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time
*shadow)
{
-u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+u64 delta;
+u64 tsc = native_read_tsc();
+BUG_ON(tsc shadow-tsc_timestamp);
+delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor bug can
kill the guest.
An attempt to print from this place is not perfect since it often leads
to recursive calling to this very function and it hang there anyway.
But if you insist I'll re-post it with WARN_ON_ONCE,
It won't make much difference because guest will hang/stall due overflow
anyway.
Won't a BUG_ON() also result in a printk?
If there is an intention to keep guest functional after the event then
maybe this patch is a way to go
http://www.spinics.net/lists/kvm/msg68463.html
this way clock will be re-silent to this kind of errors, like bare-metal
one is.
It's the same patch... do you mean something that detects the overflow
and uses the last value?
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] BUG in pv_clock when overflow condition is detected
- Original Message -
From: Avi Kivity a...@redhat.com
To: Igor Mammedov imamm...@redhat.com
Cc: linux-ker...@vger.kernel.org, kvm@vger.kernel.org, t...@linutronix.de,
mi...@redhat.com, h...@zytor.com,
r...@redhat.com, amit shah amit.s...@redhat.com, mtosa...@redhat.com
Sent: Wednesday, February 15, 2012 3:02:04 PM
Subject: Re: [PATCH] BUG in pv_clock when overflow condition is detected
On 02/15/2012 01:23 PM, Igor Mammedov wrote:
static u64 pvclock_get_nsec_offset(struct pvclock_shadow_time
*shadow)
{
-u64 delta = native_read_tsc() - shadow-tsc_timestamp;
+u64 delta;
+u64 tsc = native_read_tsc();
+BUG_ON(tsc shadow-tsc_timestamp);
+delta = tsc - shadow-tsc_timestamp;
return pvclock_scale_delta(delta, shadow-tsc_to_nsec_mul,
shadow-tsc_shift);
Maybe a WARN_ON_ONCE()? Otherwise a relatively minor hypervisor
bug can
kill the guest.
An attempt to print from this place is not perfect since it often
leads
to recursive calling to this very function and it hang there
anyway.
But if you insist I'll re-post it with WARN_ON_ONCE,
It won't make much difference because guest will hang/stall due
overflow
anyway.
Won't a BUG_ON() also result in a printk?
Yes, it will. But stack will still keep failure point and poking
with crash/gdb at core will always show where it's BUGged.
In case it manages to print dump somehow (saw it couple times from ~
30 test cycles), logs from console or from kernel message buffer
(again poking with gdb) will show where it was called from.
If WARN* is used, it will still totaly screwup clock and
last value and system will become unusable, requiring looking with
gdb/crash at the core any way.
So I've just used more stable failure point that will leave trace
everywhere it manages (maybe in console log, but for sure in stack)
in case of WARN it might leave trace on console or not and probably
won't reflect failure point in stack either leaving only kernel
message buffer for clue.
If there is an intention to keep guest functional after the event
then
maybe this patch is a way to go
http://www.spinics.net/lists/kvm/msg68463.html
this way clock will be re-silent to this kind of errors, like
bare-metal
one is.
It's the same patch... do you mean something that detects the
overflow
and uses the last value?
I'm sorry, pasted wrong link
here it goes:
pvclock: Make pv_clock more robust and fixup it if overflow happens
http://www.spinics.net/lists/kvm/msg68440.html
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
