Re: Questions about KVM TSC trapping

[David Matlack]

On Wed, Sep 23, 2015 at 5:55 PM, yangoliver  wrote:
>
> David,
>
> Sorry for late reply. See my inline comments.
>
>
> On Tue, 15 Sep 2015, David Matlack wrote:
>
>> On Tue, Sep 15, 2015 at 12:04 AM, Oliver Yang  
>> wrote:
>> > Hi Guys,
>> >
>> > I found below patch for KVM TSC trapping / migration support,
>> >
>> > https://lkml.org/lkml/2011/1/6/90
>> >
>> > It seemed the patch were not merged in Linux mainline.
>> >
>> > So I have 3 questions here,
>> >
>> > 1.  Can KVM support TSC trapping today? If not, what is the plan?
>>
>> Not without a patch. Did you want to trap RDTSC? RDTSC works without
>> trapping thanks to hardware support.
> In fact, I think TSC trap is the only way to make TSC sync in guest OS.
>
> For exmaple, user space applications could use rdtsc directly, which
> may cause the problem if there is no TSC emulation.

Ah, I see. KVM does not provide an option to trap and emulate RDTSC to
provide a stable TSC. I do not know of any plans to change this.

>>
>> >
>> > 2. What is the solution if my SMP Linux guest OS doesn't have reliable
>> > TSC?
>>
>> If you are seeing an unreliable TSC in your guest, maybe your host
>> hardware does not support a synchronized TSC across CPUs. I can't
>> recall how to check for this. There might be a flag in your host's
>> /proc/cpuinfo.
>
> Yes. I knew how to check the hardware capabilitiy.
>
> My major question was whether KVM support TSC sync even running over
> TSC unreliable hardware.
>>
>> >
>> > Because the no TSC trapping support, will kvmclock driver handle all
>> > TSC sync issues?
>> >
>> > 3. What if my Linux guest doesn't have kvmclock driver?
>>
>> The guest will use a different clock source (e.g. acpi-pm). Note the
>> RDTSC[P] instruction will still work just fine.
>
> The major problem is user application, as my application calls rstsc under
> multi-thread environment.
>
>> >
>> > Does that mean I shouldn't run TSC sensitive application in my guest
>> > OS?
>> >
>> > BTW, my application is written with lots of rdtsc instructions, and
>> > which performs well in VMware guest.
>>
>> Does it not work well in KVM?
>
> Not quite sure. We plan to run our application over KVM.
> VMware ESX5.x supported TSC sync even underlying hardware TSC reliable,
> so I'm just wondering KVM cases.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Questions about KVM TSC trapping

[yangoliver]

David,

Sorry for late reply. See my inline comments.


On Tue, 15 Sep 2015, David Matlack wrote:

> On Tue, Sep 15, 2015 at 12:04 AM, Oliver Yang  wrote:
> > Hi Guys,
> >
> > I found below patch for KVM TSC trapping / migration support,
> >
> > https://lkml.org/lkml/2011/1/6/90
> >
> > It seemed the patch were not merged in Linux mainline.
> >
> > So I have 3 questions here,
> >
> > 1.  Can KVM support TSC trapping today? If not, what is the plan?
> 
> Not without a patch. Did you want to trap RDTSC? RDTSC works without
> trapping thanks to hardware support.
In fact, I think TSC trap is the only way to make TSC sync in guest OS.

For exmaple, user space applications could use rdtsc directly, which
may cause the problem if there is no TSC emulation.
> 
> >
> > 2. What is the solution if my SMP Linux guest OS doesn't have reliable
> > TSC?
> 
> If you are seeing an unreliable TSC in your guest, maybe your host
> hardware does not support a synchronized TSC across CPUs. I can't
> recall how to check for this. There might be a flag in your host's
> /proc/cpuinfo.

Yes. I knew how to check the hardware capabilitiy.

My major question was whether KVM support TSC sync even running over
TSC unreliable hardware.
> 
> >
> > Because the no TSC trapping support, will kvmclock driver handle all
> > TSC sync issues?
> >
> > 3. What if my Linux guest doesn't have kvmclock driver?
> 
> The guest will use a different clock source (e.g. acpi-pm). Note the
> RDTSC[P] instruction will still work just fine.

The major problem is user application, as my application calls rstsc under
multi-thread environment.

> >
> > Does that mean I shouldn't run TSC sensitive application in my guest
> > OS?
> >
> > BTW, my application is written with lots of rdtsc instructions, and
> > which performs well in VMware guest.
> 
> Does it not work well in KVM?

Not quite sure. We plan to run our application over KVM.
VMware ESX5.x supported TSC sync even underlying hardware TSC reliable,
so I'm just wondering KVM cases.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Questions about KVM TSC trapping

[Wanpeng Li]

On 9/16/15 6:00 AM, David Matlack wrote:

On Tue, Sep 15, 2015 at 12:04 AM, Oliver Yang  wrote:

Hi Guys,

I found below patch for KVM TSC trapping / migration support,

https://lkml.org/lkml/2011/1/6/90

It seemed the patch were not merged in Linux mainline.

So I have 3 questions here,

1.  Can KVM support TSC trapping today? If not, what is the plan?

Not without a patch. Did you want to trap RDTSC? RDTSC works without
trapping thanks to hardware support.


2. What is the solution if my SMP Linux guest OS doesn't have reliable
TSC?

If you are seeing an unreliable TSC in your guest, maybe your host
hardware does not support a synchronized TSC across CPUs. I can't
recall how to check for this. There might be a flag in your host's
/proc/cpuinfo.


constant_tsc




Because the no TSC trapping support, will kvmclock driver handle all
TSC sync issues?

3. What if my Linux guest doesn't have kvmclock driver?

The guest will use a different clock source (e.g. acpi-pm). Note the
RDTSC[P] instruction will still work just fine.


Does that mean I shouldn't run TSC sensitive application in my guest
OS?

BTW, my application is written with lots of rdtsc instructions, and
which performs well in VMware guest.

Does it not work well in KVM?
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Questions about KVM TSC trapping

[David Matlack]

On Tue, Sep 15, 2015 at 12:04 AM, Oliver Yang  wrote:
> Hi Guys,
>
> I found below patch for KVM TSC trapping / migration support,
>
> https://lkml.org/lkml/2011/1/6/90
>
> It seemed the patch were not merged in Linux mainline.
>
> So I have 3 questions here,
>
> 1.  Can KVM support TSC trapping today? If not, what is the plan?

Not without a patch. Did you want to trap RDTSC? RDTSC works without
trapping thanks to hardware support.

>
> 2. What is the solution if my SMP Linux guest OS doesn't have reliable
> TSC?

If you are seeing an unreliable TSC in your guest, maybe your host
hardware does not support a synchronized TSC across CPUs. I can't
recall how to check for this. There might be a flag in your host's
/proc/cpuinfo.

>
> Because the no TSC trapping support, will kvmclock driver handle all
> TSC sync issues?
>
> 3. What if my Linux guest doesn't have kvmclock driver?

The guest will use a different clock source (e.g. acpi-pm). Note the
RDTSC[P] instruction will still work just fine.

>
> Does that mean I shouldn't run TSC sensitive application in my guest
> OS?
>
> BTW, my application is written with lots of rdtsc instructions, and
> which performs well in VMware guest.

Does it not work well in KVM?
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Questions about KVM TSC trapping

[Oliver Yang]

Hi Guys,

I found below patch for KVM TSC trapping / migration support, 

https://lkml.org/lkml/2011/1/6/90

It seemed the patch were not merged in Linux mainline.

So I have 3 questions here, 

1.  Can KVM support TSC trapping today? If not, what is the plan?

2. What is the solution if my SMP Linux guest OS doesn't have reliable 
TSC?

Because the no TSC trapping support, will kvmclock driver handle all 
TSC sync issues?

3. What if my Linux guest doesn't have kvmclock driver?

Does that mean I shouldn't run TSC sensitive application in my guest 
OS? 

BTW, my application is written with lots of rdtsc instructions, and
which performs well in VMware guest.