Re: [osv-dev] Does OSv have its own scheduler?

[Nadav Har'El]

On Tue, Feb 19, 2019 at 12:21 AM Joe Duarte  wrote:

>
>
> On Monday, February 18, 2019 at 1:43:29 PM UTC-8, Nadav Har'El wrote:
>>
>>
>>
>> KVM and Xen are *hypervisors*, i.e., software which allows running
>> multiple virtual-machines on one physical machine.
>> The "Linux scheduler" and "Xen scheduler" you mention are implementation
>> details of that hypervisor, and has nothing to do with OSv.
>>
>>
> Right, I know all that. What I was saying is that KVM is supposed to have
> an *advantage* over Xen because there's only one scheduler, because while
> KVM is technically a hypervisor, it's in the Linux kernel (not underneath
> it like Xen).
>

Whether this is a benefit of KVM vs Xen or not (there are many other
differences, and I personally have a soft-spot for KVM),
is really orthogonal to OSv. OSv runs on top of either of these
hypervisors, and have nothing to do with their schedulers.


> I got this point from Brendan Gregg, a performance engineer at Joyent and
> now Netflix. One place he touches on it is in this doc
> :
> "With KVM, the hypervisor is a kernel module (kvm) which is scheduled by
> the OS scheduler. It can be tuned using the usual OS kernel scheduler
> classes, policies and priorities. The I/O path takes fewer steps than Xen."
>
> I took this as having implications for OSv since OSv also runs on top of
> Xen with, apparently, its own scheduler. (I don't know if it's restricted
> to Xen anymore.)
>

OSv was never restricted to Xen. Maybe you are thinking of some other
unikernel which is Xen-only. OSv supported KVM right from the start.


> I don't know if or how the schedulers interact. Is OSv I/O (on Xen) faster
> than KVM VM I/O? Table 1 from the paper might bear on it, but it's five
> years old and I'm not clear as to what the Linux setup was (was it Fedora
> 20 running KVM with a Fedora 20 guest on top?).
>
> JD
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [osv-dev] Does OSv have its own scheduler?

[Roman Shaposhnik]

FWIW: KVM is a type-2 hypervisor, Xen (and ACRN and a few others) are
type-1. Both have advantages and disadvantages.

For example, a type-2 hypervisor is very difficult to tune for running RT
OSes while ACRN (and even Xen lately -- especially on ARM) do it with no
sweat.

Thanks,
Roman.


On Mon, Feb 18, 2019 at 11:21 PM Joe Duarte  wrote:

>
>
> On Monday, February 18, 2019 at 1:43:29 PM UTC-8, Nadav Har'El wrote:
>>
>>
>>
>> KVM and Xen are *hypervisors*, i.e., software which allows running
>> multiple virtual-machines on one physical machine.
>> The "Linux scheduler" and "Xen scheduler" you mention are implementation
>> details of that hypervisor, and has nothing to do with OSv.
>>
>>
> Right, I know all that. What I was saying is that KVM is supposed to have
> an *advantage* over Xen because there's only one scheduler, because while
> KVM is technically a hypervisor, it's in the Linux kernel (not underneath
> it like Xen). I got this point from Brendan Gregg, a performance engineer
> at Joyent and now Netflix. One place he touches on it is in this doc
> :
> "With KVM, the hypervisor is a kernel module (kvm) which is scheduled by
> the OS scheduler. It can be tuned using the usual OS kernel scheduler
> classes, policies and priorities. The I/O path takes fewer steps than Xen."
>
> I took this as having implications for OSv since OSv also runs on top of
> Xen with, apparently, its own scheduler. (I don't know if it's restricted
> to Xen anymore.) I don't know if or how the schedulers interact. Is OSv I/O
> (on Xen) faster than KVM VM I/O? Table 1 from the paper might bear on it,
> but it's five years old and I'm not clear as to what the Linux setup was
> (was it Fedora 20 running KVM with a Fedora 20 guest on top?).
>
> JD
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [osv-dev] Does OSv have its own scheduler?

[Joe Duarte]

On Monday, February 18, 2019 at 1:43:29 PM UTC-8, Nadav Har'El wrote:
>
>
>
> KVM and Xen are *hypervisors*, i.e., software which allows running 
> multiple virtual-machines on one physical machine.
> The "Linux scheduler" and "Xen scheduler" you mention are implementation 
> details of that hypervisor, and has nothing to do with OSv.
>
>
Right, I know all that. What I was saying is that KVM is supposed to have 
an *advantage* over Xen because there's only one scheduler, because while 
KVM is technically a hypervisor, it's in the Linux kernel (not underneath 
it like Xen). I got this point from Brendan Gregg, a performance engineer 
at Joyent and now Netflix. One place he touches on it is in this doc 
:
 
"With KVM, the hypervisor is a kernel module (kvm) which is scheduled by 
the OS scheduler. It can be tuned using the usual OS kernel scheduler 
classes, policies and priorities. The I/O path takes fewer steps than Xen."

I took this as having implications for OSv since OSv also runs on top of 
Xen with, apparently, its own scheduler. (I don't know if it's restricted 
to Xen anymore.) I don't know if or how the schedulers interact. Is OSv I/O 
(on Xen) faster than KVM VM I/O? Table 1 from the paper might bear on it, 
but it's five years old and I'm not clear as to what the Linux setup was 
(was it Fedora 20 running KVM with a Fedora 20 guest on top?).

JD

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [osv-dev] Does OSv have its own scheduler?

[Nadav Har'El]

On Mon, Feb 18, 2019 at 8:07 PM Joe Duarte  wrote:

> Thanks Pekka, I'll check it out.
>

Good. I think this will clear some of the misconceptions you may have.


> One of the advantages of KVM is supposed to be that it uses the Linux
> kernel scheduler instead of having two schedulers, such as Xen's scheduler
> + a VM's or unikernel's scheduler. I'll see what the paper has to say about
> that.
>

KVM and Xen are *hypervisors*, i.e., software which allows running multiple
virtual-machines on one physical machine.
The "Linux scheduler" and "Xen scheduler" you mention are implementation
details of that hypervisor, and has nothing to do with OSv.

OSv is a an operating system for *one* virtual machine.
This virtual machine may have several virtual CPUs, and may (and usually
does) want to run many threads on them, and for that OSv needs a scheduler
for those threads, to decide which of the threads runs on which virtual
CPU, and when.

OSv does support threads, as almost any modern application needs them, and
they are absolutely needed if you have multiple virtual CPUs in one VM (you
cannot run just one thread on multiple CPUs).  What OSv does not support
are old-school "processes", which are threads which are isolated from each
other - each having a different address space, and often running a
different application. This sort of isolation comes at a cost, which OSv
wanted to avoid. This is explained in detail in the paper.



> JD
>
> On Sunday, February 17, 2019 at 10:28:26 PM UTC-8, Pekka Enberg wrote:
>>
>> Hello,
>>
>> On Sun, Feb 17, 2019 at 11:45 PM Joe Duarte  wrote:
>>
>>> Hi all -- Does OSv have its own scheduler, or does it rely on the Xen
>>> scheduler?
>>>
>>> I guess this might be similar to asking if OSv has threads. I have no
>>> idea. I'm not sure what being a unikernel is supposed to imply for threads.
>>>
>>
>> OSv indeed has as a scheduler to support threads. You can find a more
>> detailed discussion in the OSv paper, for example:
>>
>>
>> https://www.usenix.org/system/files/conference/atc14/atc14-paper-kivity.pdf
>>
>> - Pekka
>>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [osv-dev] Does OSv have its own scheduler?

[Joe Duarte]

Thanks Pekka, I'll check it out. One of the advantages of KVM is supposed 
to be that it uses the Linux kernel scheduler instead of having two 
schedulers, such as Xen's scheduler + a VM's or unikernel's scheduler. I'll 
see what the paper has to say about that.

JD

On Sunday, February 17, 2019 at 10:28:26 PM UTC-8, Pekka Enberg wrote:
>
> Hello,
>
> On Sun, Feb 17, 2019 at 11:45 PM Joe Duarte  > wrote:
>
>> Hi all -- Does OSv have its own scheduler, or does it rely on the Xen 
>> scheduler?
>>
>> I guess this might be similar to asking if OSv has threads. I have no 
>> idea. I'm not sure what being a unikernel is supposed to imply for threads.
>>
>
> OSv indeed has as a scheduler to support threads. You can find a more 
> detailed discussion in the OSv paper, for example:
>
> https://www.usenix.org/system/files/conference/atc14/atc14-paper-kivity.pdf
>
> - Pekka
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [osv-dev] Does OSv have its own scheduler?

[Pekka Enberg]

Hello,

On Sun, Feb 17, 2019 at 11:45 PM Joe Duarte  wrote:

> Hi all -- Does OSv have its own scheduler, or does it rely on the Xen
> scheduler?
>
> I guess this might be similar to asking if OSv has threads. I have no
> idea. I'm not sure what being a unikernel is supposed to imply for threads.
>

OSv indeed has as a scheduler to support threads. You can find a more
detailed discussion in the OSv paper, for example:

https://www.usenix.org/system/files/conference/atc14/atc14-paper-kivity.pdf

- Pekka

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.