[RFC]How to improve KVM VM resource assignment and per-vm process/thread scheduling.
Hi, ALL Currently kvm hypervisor have lots of features depend on linux standard apis, like vcpupin/mempin/processpin etc. But in the real production environment, we need an automated resource assign and/or scheduling, is there any plan to implement it? resource assignment requirements like: cpu eligible by the VMs In case it is eligible, whether it is in use if it is in use, whether it is dedicated to one VM, or can be shared by many VMs In case of Shared CPU need to configure oversubscription ratio used ratio info So does memory, I/O device assignment requirements. per-vm process/thread scheduling requirements like: On hypervisor side, VMs use vhost-net, virtio-scsi devices have qemu io-thread, vhost-net thread, ovs thread, hNIC interrupt context(hirq/softirq), you shoud place there threads on the same numa node to gain best performance, another important thing, you should balance these threads' cpuload on all the numa cores to avoid unbalance between vcpu usable resources. Thanks. Peter Huang
[RFC]How to improve KVM VM resource assignment and per-vm process/thread scheduling.
Hi, ALL Currently kvm hypervisor have lots of features depend on linux standard apis, like vcpupin/mempin/processpin etc. But in the real production environment, we need an automated resource assign and/or scheduling, is there any plan to implement it? resource assignment requirements like: cpu eligible by the VMs In case it is eligible, whether it is in use if it is in use, whether it is dedicated to one VM, or can be shared by many VMs In case of Shared CPU need to configure oversubscription ratio used ratio info So does memory, I/O device assignment requirements. per-vm process/thread scheduling requirements like: On hypervisor side, VMs use vhost-net, virtio-scsi devices have qemu io-thread, vhost-net thread, ovs thread, hNIC interrupt context(hirq/softirq), you shoud place there threads on the same numa node to gain best performance, another important thing, you should balance these threads' cpuload on all the numa cores to avoid unbalance between vcpu usable resources. Thanks. Peter Huang
Re: On thread scheduling
On Sep 14, 2007, at 18:40:00, Heikki Orsila wrote: Consider a simple embedded system: void interrupt_handler(void) int main(void) I would like to "emulate" this system with a workstation to make development faster. I would create two threads, one executing the main() function, and the other occasionally calling interrupt_handler(). Before interrupt_handler() is called, the main () thread should be stopped asynchronously. Actually you probably just want to do this with an ordinary single- threaded program. Just do your main() thing normally and schedule a signal-based timer at program start (or whenever you actually set up your periodic interrupt source); the timer signal-handler will act as the interrupt handler. Alternatively you could have 2 threads where the *only* thing the second thread does is a timed poll() loop on an FD where it reads commands from the main thread. When the main thread wants to change the interval-between-interrupts or some other interrupt configuration it writes the info down the FD to the alternate thread which actually sets up the change. The main program then continues running and periodically receives its SIGUSR1 or whatever from the other thread and uses that as the interrupt. Cheers, Kyle Moffett - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: On thread scheduling
On Sep 14, 2007, at 18:40:00, Heikki Orsila wrote: Consider a simple embedded system: void interrupt_handler(void) int main(void) I would like to emulate this system with a workstation to make development faster. I would create two threads, one executing the main() function, and the other occasionally calling interrupt_handler(). Before interrupt_handler() is called, the main () thread should be stopped asynchronously. Actually you probably just want to do this with an ordinary single- threaded program. Just do your main() thing normally and schedule a signal-based timer at program start (or whenever you actually set up your periodic interrupt source); the timer signal-handler will act as the interrupt handler. Alternatively you could have 2 threads where the *only* thing the second thread does is a timed poll() loop on an FD where it reads commands from the main thread. When the main thread wants to change the interval-between-interrupts or some other interrupt configuration it writes the info down the FD to the alternate thread which actually sets up the change. The main program then continues running and periodically receives its SIGUSR1 or whatever from the other thread and uses that as the interrupt. Cheers, Kyle Moffett - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: On thread scheduling
On 9/14/07, Heikki Orsila <[EMAIL PROTECTED]> wrote: > Consider a simple embedded system: > > void interrupt_handler(void) > int main(void) > > I would like to "emulate" this system with a workstation to make > development faster. I would create two threads, one executing the > main() function, and the other occasionally calling interrupt_handler(). > Before interrupt_handler() is called, the main() thread should be > stopped asynchronously. Are you open to doing something other than pthreads? Using fork() to get real processes, and then sending a SIGSTOP (and SIGCONT) to the one running main() should work. Or emulating the interrupt arrival with a signal, and having interrupt_handler() be registered as the signal handler. Of course, you then have to deal with the constraints that come with signal handlers, so that can be a bit of a pain depending on what you're trying to do. Ray - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: On thread scheduling
On 9/14/07, Heikki Orsila [EMAIL PROTECTED] wrote: Consider a simple embedded system: void interrupt_handler(void) int main(void) I would like to emulate this system with a workstation to make development faster. I would create two threads, one executing the main() function, and the other occasionally calling interrupt_handler(). Before interrupt_handler() is called, the main() thread should be stopped asynchronously. Are you open to doing something other than pthreads? Using fork() to get real processes, and then sending a SIGSTOP (and SIGCONT) to the one running main() should work. Or emulating the interrupt arrival with a signal, and having interrupt_handler() be registered as the signal handler. Of course, you then have to deal with the constraints that come with signal handlers, so that can be a bit of a pain depending on what you're trying to do. Ray - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: On thread scheduling
El Sat, Sep 15, 2007 at 01:40:00AM +0300 Heikki Orsila ha dit:
> Consider a simple embedded system:
>
> void interrupt_handler(void)
> {
> ...
> }
>
> int main(void)
> {
> ...
> }
>
> I would like to "emulate" this system with a workstation to make
> development faster. I would create two threads, one executing the
> main() function, and the other occasionally calling interrupt_handler().
> Before interrupt_handler() is called, the main() thread should be
> stopped asynchronously.
>
> I looked into pthreads documentation and found only pthread_kill(thread,
> SIGTSTP) to do asynchronous stop, but then I also have to play terminal
> tricks to avoid problems..
>
> Is there are function to just disable scheduling of a single thread
> without having other side-effects (such as terminal stuff)? Functions
> like pthread_disable_scheduling(thread) and
> pthread_enable_scheduling(thread) would be good for this..
i wonder if assigning a real time priority with
pthread_setschedparam() to the caller of your interrupt_handler()
function would solve your problem. this way the main() thread is never
scheduled when the interrupt_handler() thread is
runnable. interrupt_handler() should be as short as possible,
otherwise you risk starving of the rest of your systems processes.
--
Matthias Kaehlcke
Linux Application Developer
Barcelona
Representation of the world, like the world itself, is
the work of men; they describe it from their own point
of view, which they confuse with the absolute truth
(Simone de Beauvoir)
.''`.
using free software / Debian GNU/Linux | http://debian.org : :' :
`. `'`
gpg --keyserver pgp.mit.edu --recv-keys 47D8E5D4 `-
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
On thread scheduling
Consider a simple embedded system:
void interrupt_handler(void)
{
...
}
int main(void)
{
...
}
I would like to "emulate" this system with a workstation to make
development faster. I would create two threads, one executing the
main() function, and the other occasionally calling interrupt_handler().
Before interrupt_handler() is called, the main() thread should be
stopped asynchronously.
I looked into pthreads documentation and found only pthread_kill(thread,
SIGTSTP) to do asynchronous stop, but then I also have to play terminal
tricks to avoid problems..
Is there are function to just disable scheduling of a single thread
without having other side-effects (such as terminal stuff)? Functions
like pthread_disable_scheduling(thread) and
pthread_enable_scheduling(thread) would be good for this..
--
Heikki Orsila Barbie's law:
[EMAIL PROTECTED] "Math is hard, let's go shopping!"
http://www.iki.fi/shd
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
On thread scheduling
Consider a simple embedded system:
void interrupt_handler(void)
{
...
}
int main(void)
{
...
}
I would like to emulate this system with a workstation to make
development faster. I would create two threads, one executing the
main() function, and the other occasionally calling interrupt_handler().
Before interrupt_handler() is called, the main() thread should be
stopped asynchronously.
I looked into pthreads documentation and found only pthread_kill(thread,
SIGTSTP) to do asynchronous stop, but then I also have to play terminal
tricks to avoid problems..
Is there are function to just disable scheduling of a single thread
without having other side-effects (such as terminal stuff)? Functions
like pthread_disable_scheduling(thread) and
pthread_enable_scheduling(thread) would be good for this..
--
Heikki Orsila Barbie's law:
[EMAIL PROTECTED] Math is hard, let's go shopping!
http://www.iki.fi/shd
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: On thread scheduling
El Sat, Sep 15, 2007 at 01:40:00AM +0300 Heikki Orsila ha dit:
Consider a simple embedded system:
void interrupt_handler(void)
{
...
}
int main(void)
{
...
}
I would like to emulate this system with a workstation to make
development faster. I would create two threads, one executing the
main() function, and the other occasionally calling interrupt_handler().
Before interrupt_handler() is called, the main() thread should be
stopped asynchronously.
I looked into pthreads documentation and found only pthread_kill(thread,
SIGTSTP) to do asynchronous stop, but then I also have to play terminal
tricks to avoid problems..
Is there are function to just disable scheduling of a single thread
without having other side-effects (such as terminal stuff)? Functions
like pthread_disable_scheduling(thread) and
pthread_enable_scheduling(thread) would be good for this..
i wonder if assigning a real time priority with
pthread_setschedparam() to the caller of your interrupt_handler()
function would solve your problem. this way the main() thread is never
scheduled when the interrupt_handler() thread is
runnable. interrupt_handler() should be as short as possible,
otherwise you risk starving of the rest of your systems processes.
--
Matthias Kaehlcke
Linux Application Developer
Barcelona
Representation of the world, like the world itself, is
the work of men; they describe it from their own point
of view, which they confuse with the absolute truth
(Simone de Beauvoir)
.''`.
using free software / Debian GNU/Linux | http://debian.org : :' :
`. `'`
gpg --keyserver pgp.mit.edu --recv-keys 47D8E5D4 `-
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
