Re: How to test the schedule latency?

[Pekka Enberg]

On Thu, Oct 11, 2018 at 12:22 PM Avi Kivity  wrote:

>
> On 11/10/2018 12.14, Nadav Har'El wrote:
>
> On Thu, Oct 11, 2018 at 12:10 PM Pekka Enberg 
> wrote:
>
>>
>>
>> On Thu, Oct 11, 2018 at 11:50 AM Nadav Har'El  wrote:
>>
>>>
>>> tests/misc-ctxsw.cc
>>>
>>> To run it in OSv you can do "scripts/build image=tests" and then
>>> "scripts/run.py -e tests/misc-ctxsw.so"
>>>
>>> What this test does is to create two threads (on the same core, or on
>>> two different cores) which alternate waking the other thread and then
>>> waiting (via a condition variable).
>>> Strangely I just tested this again on my laptop (which is newer than the
>>> paper), and got (for the colocated case) in OSv is around 400ns, and in
>>> Linux around 1800ns, so the difference is even more pronounced today.
>>>
>>
>> Perhaps you are seeing the effects of Meltdown mitigation in Linux
>> context switching paths?
>>
>
> Perhaps, I didn't look into this carefully, I just ran it now to make sure
> I was pointing the OP to the right test :-)
> Does meltdown mitigation have a performance cost even when switching
> between two threads belonging to the same process?
>
>
>
> Yes, the kernel and userspace run in different address spaces (CR3 switch
> on every kernel entry and exit). It's not a thread context switch cost, but
> a syscall cost.
>
Right, but AFAICT, it's also an interrupt cost. So whatever the mechanism
that causes the context switch to another thread, you need to switch to
kernel CR3 and to the userspace thread CR3.

- Pekka

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to test the schedule latency?

[Avi Kivity]

On 11/10/2018 12.14, Nadav Har'El wrote:
On Thu, Oct 11, 2018 at 12:10 PM Pekka Enberg > wrote:




On Thu, Oct 11, 2018 at 11:50 AM Nadav Har'El mailto:n...@scylladb.com>> wrote:


tests/misc-ctxsw.cc

To run it in OSv you can do "scripts/build image=tests" and
then "scripts/run.py -e tests/misc-ctxsw.so"

What this test does is to create two threads (on the same
core, or on two different cores) which alternate waking the
other thread and then waiting (via a condition variable).
Strangely I just tested this again on my laptop (which is
newer than the paper), and got (for the colocated case) in OSv
is around 400ns, and in Linux around 1800ns, so the difference
is even more pronounced today.


Perhaps you are seeing the effects of Meltdown mitigation in Linux
context switching paths?


Perhaps, I didn't look into this carefully, I just ran it now to make 
sure I was pointing the OP to the right test :-)
Does meltdown mitigation have a performance cost even when switching 
between two threads belonging to the same process?





Yes, the kernel and userspace run in different address spaces (CR3 
switch on every kernel entry and exit). It's not a thread context switch 
cost, but a syscall cost.


--
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to test the schedule latency?

[Pekka Enberg]

On Thu, Oct 11, 2018 at 12:14 PM Nadav Har'El  wrote:

> On Thu, Oct 11, 2018 at 12:10 PM Pekka Enberg 
> wrote:
>
>>
>>
>> On Thu, Oct 11, 2018 at 11:50 AM Nadav Har'El  wrote:
>>
>>>
>>> tests/misc-ctxsw.cc
>>>
>>> To run it in OSv you can do "scripts/build image=tests" and then
>>> "scripts/run.py -e tests/misc-ctxsw.so"
>>>
>>> What this test does is to create two threads (on the same core, or on
>>> two different cores) which alternate waking the other thread and then
>>> waiting (via a condition variable).
>>> Strangely I just tested this again on my laptop (which is newer than the
>>> paper), and got (for the colocated case) in OSv is around 400ns, and in
>>> Linux around 1800ns, so the difference is even more pronounced today.
>>>
>>
>> Perhaps you are seeing the effects of Meltdown mitigation in Linux
>> context switching paths?
>>
>
> Perhaps, I didn't look into this carefully, I just ran it now to make sure
> I was pointing the OP to the right test :-)
> Does meltdown mitigation have a performance cost even when switching
> between two threads belonging to the same process?
>

Yes, that's my understanding.

Linux mitigates against Meltdown with kernel page table isolation.
Userspace processes no longer have kernel memory mapped in the upper half
of the address space (to avoid side-channel attacks from reading arbitrary
mapped memory), which means that the kernel needs to switch page tables to
access kernel memory. Search for SWITCH_TO_KERNEL_CR3 for system call entry
in arch/x86/entry/entry_64.S, for example.

- Pekka

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to test the schedule latency?

[Nadav Har'El]

On Thu, Oct 11, 2018 at 12:10 PM Pekka Enberg  wrote:

>
>
> On Thu, Oct 11, 2018 at 11:50 AM Nadav Har'El  wrote:
>
>>
>> tests/misc-ctxsw.cc
>>
>> To run it in OSv you can do "scripts/build image=tests" and then
>> "scripts/run.py -e tests/misc-ctxsw.so"
>>
>> What this test does is to create two threads (on the same core, or on two
>> different cores) which alternate waking the other thread and then waiting
>> (via a condition variable).
>> Strangely I just tested this again on my laptop (which is newer than the
>> paper), and got (for the colocated case) in OSv is around 400ns, and in
>> Linux around 1800ns, so the difference is even more pronounced today.
>>
>
> Perhaps you are seeing the effects of Meltdown mitigation in Linux context
> switching paths?
>

Perhaps, I didn't look into this carefully, I just ran it now to make sure
I was pointing the OP to the right test :-)
Does meltdown mitigation have a performance cost even when switching
between two threads belonging to the same process?

Nadav.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How to test the schedule latency?

[Pekka Enberg]

On Thu, Oct 11, 2018 at 11:50 AM Nadav Har'El  wrote:

>
> On Thu, Oct 11, 2018 at 10:45 AM Qixuan Wu  wrote:
>
>> Hi All,
>>
>> I saw the paper,  it said schedule latency is less than linux thread,
>> about 328ns.
>>
>> Guest OS
>>
>> Colocated
>>
>> Apart
>>
>> Linux
>>
>> 905 ns
>>
>> 13148 ns
>>
>> OSv
>>
>> 328 ns
>>
>> 1402 ns
>>
>> Anyone can tell where is the test case code ?
>>
>
> tests/misc-ctxsw.cc
>
> To run it in OSv you can do "scripts/build image=tests" and then
> "scripts/run.py -e tests/misc-ctxsw.so"
>
> What this test does is to create two threads (on the same core, or on two
> different cores) which alternate waking the other thread and then waiting
> (via a condition variable).
> Strangely I just tested this again on my laptop (which is newer than the
> paper), and got (for the colocated case) in OSv is around 400ns, and in
> Linux around 1800ns, so the difference is even more pronounced today.
>

Perhaps you are seeing the effects of Meltdown mitigation in Linux context
switching paths?

- Pekka

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

How to test the schedule latency?

[Qixuan Wu]

Hi All, 

I saw the paper,  it said schedule latency is less than linux thread, about 
328ns. 

Guest OS

Colocated

Apart

Linux

905 ns

13148 ns

OSv

328 ns

1402 ns

Anyone can tell where is the test case code ? 

Thanks & Regards
Qixuan.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.