Re: [Libvir] Re: Proposal : add 3 functions to Libvirt API, for virtual CPUs
On Fri, Jul 28, 2006 at 03:18:17PM -0600, Jim Fehlig wrote: Has any progress been made on this proposal? I don't see anything in current CVS. This functionality would be useful for Xen-CIM project. IIRC Michel Ponceau might have started on some prototype code, but he didn't post it to the list yet is apparently on vacation for another couple of weeks (according to his email auto-responder). If no one else has started work on it in the meantime, I'm planning to get working on it when I return from vacation Aug 4th since I also need it for the virt-manager application. Regards, Dan. [EMAIL PROTECTED] wrote: Corrections on proposal: 1) PinVcpus Replace: * @cpumap: pointer to a bit map of real CPUs (format in virVcpuInfo) * @maplen: length of cpumap, in 8-bit bytes by: * @cpumap: pointer to a bit map of real CPUs (in 8-bit bytes). *Each bit set to 1 means that corresponding CPU is usable. *Bytes are stored in little-endian order: CPU0-7, 8-15... *In each byte, lowest CPU number is least significant bit. * @maplen: number of bytes in cpumap, from 1 up to size of CPU map in *underlying virtualization system (Xen...). *If maplen size, missing bytes are set to zero. *If maplen size, failure code is returned. 2) GetVcpu Add 4rth argument: * @maplen: number of bytes in cpumap field of virVcpuInfo virDomainGetVcpus(virDomainPtr domain, virVcpuInfoPtr info, int maxinfo, int maplen) 3) Structure VcpuInfo Suppress: #define VIR_MAX_CPUS256 Replace: unsigned char cpumap[VIR_MAX_CPUS/8]; /* Bit map of usable real CPUs. by: unsigned char cpumap[];/* Bit map of usable real CPUs. Variable length: it may be less or greater than size of CPU map in underlying virtualization system (Xen...). 4) Accessor macros: to be defined later. Veuillez répondre à [EMAIL PROTECTED] Pour :[EMAIL PROTECTED] cc :libvir-list@redhat.com Objet :Re: Proposal : add 3 functions to Libvirt API, for virtual CPUs On Fri, Jun 30, 2006 at 04:00:45PM +0200, [EMAIL PROTECTED] wrote: For our administration, we need the following actions, while concerned domain is running: 1) Change the number of virtual CPUs. 2) Change the pinning (affinity) of a virtual CPU on real CPUs. 3) Get detailed information for each virtual CPU. Currently there is no Libvirt function provided for that. We suggest to add the following 3 functions (in libvirt.c): /** * virDomainSetVcpus: * @domain: pointer to domain object, or NULL for Domain0 * @nvcpus: the new number of virtual CPUs for this domain * * Dynamically change the number of virtual CPUs used by the domain. * Note that this call may fail if the underlying virtualization hypervisor * does not support it or if growing the number is arbitrary limited. * This function requires priviledged access to the hypervisor. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainSetVcpus(virDomainPtr domain, unsigned int nvcpus) okay /** * virDomainPinVcpu: * @domain: pointer to domain object, or NULL for Domain0 * @vcpu: virtual CPU number * @cpumap: pointer to a bit map of real CPUs (format in virVcpuInfo) * @maplen: length of cpumap, in 8-bit bytes * * Dynamically change the real CPUs which can be allocated to a virtual CPU. * This function requires priviledged access to the hypervisor. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainPinVcpu(virDomainPtr domain, unsigned int vcpu, unsigned char *cpumap, int maplen) Can you explain more clearly what is the format of cpumap ? An example would be welcome, and that would be needed for the doc and maybe testing. What would happen if maplen is or than the number of CPU divided by 8 ? /** * virDomainGetVcpus: * @domain: pointer to domain object, or NULL for Domain0 * @info: pointer to an array of virVcpuInfo structures * @maxinfo: number of structures in info array * * Extract information about virtual CPUs of a domain, store it in info array. * * Returns the number of info filled in case of success, -1 in case of failure. */ int virDomainGetVcpus(virDomainPtr domain, virVcpuInfoPtr info, int maxinfo) Hum ... now the problem with that API entry point is that we 'burn' the maximum 256 processors in the ABI, i.e. if we ever need to go past 256 client and servers need to be recompiled. Maybe this is not a real problem in practice but that's annoying. Is there existing APIs doing this kind of things (in POSIX for example), and what hard limit did they use ? Maybe int virDomainGetVcpusNr(virDomainPtr domain, int nr, virVcpuInfoPtr info, int maxCPU); Where the maxCPU is defined by the client as the number of real CPU it defined in its virVcpuInfoPtr and then an
Re: [Libvir] Re: Proposal : add 3 functions to Libvirt API, for virtual CPUs
Has any progress been made on this proposal? I don't see anything in current CVS. This functionality would be useful for Xen-CIM project. Regards, Jim [EMAIL PROTECTED] wrote: Corrections on proposal: 1) PinVcpus Replace: * @cpumap: pointer to a bit map of real CPUs (format in virVcpuInfo) * @maplen: length of cpumap, in 8-bit bytes by: * @cpumap: pointer to a bit map of real CPUs (in 8-bit bytes). *Each bit set to 1 means that corresponding CPU is usable. *Bytes are stored in little-endian order: CPU0-7, 8-15... *In each byte, lowest CPU number is least significant bit. * @maplen: number of bytes in cpumap, from 1 up to size of CPU map in *underlying virtualization system (Xen...). *If maplen size, missing bytes are set to zero. *If maplen size, failure code is returned. 2) GetVcpu Add 4rth argument: * @maplen: number of bytes in cpumap field of virVcpuInfo virDomainGetVcpus(virDomainPtr domain, virVcpuInfoPtr info, int maxinfo, int maplen) 3) Structure VcpuInfo Suppress: #define VIR_MAX_CPUS256 Replace: unsigned char cpumap[VIR_MAX_CPUS/8]; /* Bit map of usable real CPUs. by: unsigned char cpumap[];/* Bit map of usable real CPUs. Variable length: it may be less or greater than size of CPU map in underlying virtualization system (Xen...). 4) Accessor macros: to be defined later. Veuillez répondre à [EMAIL PROTECTED] Pour :[EMAIL PROTECTED] cc :libvir-list@redhat.com Objet :Re: Proposal : add 3 functions to Libvirt API, for virtual CPUs On Fri, Jun 30, 2006 at 04:00:45PM +0200, [EMAIL PROTECTED] wrote: For our administration, we need the following actions, while concerned domain is running: 1) Change the number of virtual CPUs. 2) Change the pinning (affinity) of a virtual CPU on real CPUs. 3) Get detailed information for each virtual CPU. Currently there is no Libvirt function provided for that. We suggest to add the following 3 functions (in libvirt.c): /** * virDomainSetVcpus: * @domain: pointer to domain object, or NULL for Domain0 * @nvcpus: the new number of virtual CPUs for this domain * * Dynamically change the number of virtual CPUs used by the domain. * Note that this call may fail if the underlying virtualization hypervisor * does not support it or if growing the number is arbitrary limited. * This function requires priviledged access to the hypervisor. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainSetVcpus(virDomainPtr domain, unsigned int nvcpus) okay /** * virDomainPinVcpu: * @domain: pointer to domain object, or NULL for Domain0 * @vcpu: virtual CPU number * @cpumap: pointer to a bit map of real CPUs (format in virVcpuInfo) * @maplen: length of cpumap, in 8-bit bytes * * Dynamically change the real CPUs which can be allocated to a virtual CPU. * This function requires priviledged access to the hypervisor. * * Returns 0 in case of success, -1 in case of failure. */ int virDomainPinVcpu(virDomainPtr domain, unsigned int vcpu, unsigned char *cpumap, int maplen) Can you explain more clearly what is the format of cpumap ? An example would be welcome, and that would be needed for the doc and maybe testing. What would happen if maplen is or than the number of CPU divided by 8 ? /** * virDomainGetVcpus: * @domain: pointer to domain object, or NULL for Domain0 * @info: pointer to an array of virVcpuInfo structures * @maxinfo: number of structures in info array * * Extract information about virtual CPUs of a domain, store it in info array. * * Returns the number of info filled in case of success, -1 in case of failure. */ int virDomainGetVcpus(virDomainPtr domain, virVcpuInfoPtr info, int maxinfo) Hum ... now the problem with that API entry point is that we 'burn' the maximum 256 processors in the ABI, i.e. if we ever need to go past 256 client and servers need to be recompiled. Maybe this is not a real problem in practice but that's annoying. Is there existing APIs doing this kind of things (in POSIX for example), and what hard limit did they use ? Maybe int virDomainGetVcpusNr(virDomainPtr domain, int nr, virVcpuInfoPtr info, int maxCPU); Where the maxCPU is defined by the client as the number of real CPU it defined in its virVcpuInfoPtr and then an iteration over the virtual CPU defined in the domain is possible too. Of course if the domain uses many virtual CPUs this would become expensive but somehow I don't see that being the common use, I would rather guess the domains created use a few CPUs even if instantiated on a very large machine. This with the following structure (in libvirt.h): /** * virVcpuInfo: structure for information about a virtual CPU in a domain. */ #define VIR_MAX_CPUS256 Hum, there is already NUMA machines with more than 256 processors, it's
Re: [Libvir] Re: Proposal : add 3 functions to Libvirt API, for virtual CPUs
On Mon, Jul 17, 2006 at 01:50:25PM +0100, Daniel P. Berrange wrote:
On Tue, Jul 11, 2006 at 10:50:53AM -0400, Daniel Veillard wrote:
On Tue, Jul 11, 2006 at 04:14:12PM +0200, [EMAIL PROTECTED] wrote:
Hum, I could see compilers righteously complaining about
unsigned char cpumap[];
in a structure. Maybe we should default to 256 / 8 but allow at the API
level
to grow over that value. What we could do is define a default of 256
available
in the structure and allow an extra parameter which could point to a larger
array like the following:
- restore VIR_MAX_CPUS as VIR_STD_MAX_CPUS 256
- virDomainGetVcpus(virDomainPtr domain, virVcpuInfoPtr info, int
maxinfo,
char *lcpumap, int lmaplen);
lcpumap and lmaplen being extra arguments for very large arrays
for most use case, we will fit in 256 processors, those will be
respectively
NULL and 0, but assuming we have more than 256 processors:
+ maxinfo VIR_STD_MAX_CPUS
+ lcpumap points to a array of bytes, they are interpreted as an
array of cpumap of ((maxinfo + 7) div 8) bytes each. So
if lmaplen != ((maxinfo + 7) div 8) * maxinfo then there is an
error.
in that case the cpumap structures of info are not filled on return.
We still have a relatively simple API for the common case, and for special
cases we have an extension capability with relatively clear definitions.
it's
a bitstrange but I think that should cover most case as best as possible
I dont particularly like this as an API because I think it will be error
prone for application developers. Most app developers will only ever have
a handful of CPUs in their test machines, so they'll never the alternate
codepath for 256 cpu case. Likewise I don't like the idea of a virVcpuInfo
struct which has a variable size because it will totally confuse people who
haven't read the API docs very carefully, again leading to obscure bugs.
The root problem is that we have two conflicting goals here
1. Want to have virVcpuInfo be a fixed size struct
2. We want a cpumap of arbitrary size
The obvious solution to this problem is to *remove* the cpumap data from
the virVcpuInfo structure completely, and always pass in a separately
malloc'd array of the correct size. So I'd suggest:
typedef struct _virVcpuInfo virVcpuInfo;
struct _virVcpuInfo {
unsigned int number;/* virtual CPU number */
int state; /* value from virVcpuState */
unsigned long long cpuTime; /* CPU time used, in nanoseconds */
int cpu;/* real CPU number, or -1 if offline */
}
virDomainGetVcpus(virDomainPtr domain, virVcpuInfoPtr info, int maxinfo,
char *cpumap, int maplen);
The client applications calling this API already have to malloc() the memory
region for the 'info' parameter of a correct size, so having to also malloc
the cpumap parameter is no extra trouble.
virDomainInfo info;
virDomainVpuInfoPtr cpuInfo;
int cpuMapLen;
char *cpuMap;
virDomainGetInfo(domain, info);
cpuInfo = malloc(sizeof(virDomainVcpuInfo)*info.nrVirtCpu);
cpuMapLen = (info.nrVirtCpu + 7) / 8 ;
cpuMap = malloc(cpuMapLen);
virDomainGetVCpus(domain, cpuInfo, info.nrVirtCpu, cpuMap, cpuMapLen);
... do stuff with the data ...
free(cpuInfo);
free(cpuMap);
So you can see there is minimal extra work to always pass in cpuMap as
a separate parameter. If an application didn't care about the cpuMap
data they could simply pass in NULL.
Oh one other benefit of this, is if the application wishes to use C-99
they will not have to use malloc at all, instead making use of local
declarations dynamically sized stack variables:
virDomainInfo info;
virDomainGetInfo(domain, info);
virDomainVcpuInfo cpuInfo[info.nrVirtCpu];
int cpuMapLen = (info.nrVirtCpu + 7) / 8 ;
char cpuMap[cpuMapLen];
virDomainGetVCpus(domain, cpuInfo, info.nrVirtCpu, cpuMap, cpuMapLen);
Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
--
Libvir-list mailing list
Libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[Libvir] Re: Proposal : add 3 functions to Libvirt API, for virtual CPUs
On Tue, Jul 11, 2006 at 05:29:52PM +0200, [EMAIL PROTECTED] wrote: 2) CPU map cut into standard + extension : It would be more simple to let each Libvirt user modify the value of VIR_MAX_CPUS in private libvirt.h. It could be similar to the version number update, from libvirt.h.in to libvirt.h : #define LIBVIR_VERSION_NUMBER @LIBVIRT_VERSION_NUMBER@ No, I do think it is instead very hard. You start to get into very annoying problems from an API point of view. How do you go from the first element in the info array to the next one ? in the client it's already not nice, you must do pointer arithmetic (and I really don't want to push an API which forces that to most users), it expose potential serious problem like alignment and packing, problems between versions of compilers and tools. In a nutshell I don't want an API where one need to access an array of structure where the size of the structure is not defined by the API itself. What I suggested did that in both case, either low number of CPUs and fixed size records like you initial solutions, or high number of CPUs and you work on a two dimentional array of bytes. Daniel -- Daniel Veillard | Red Hat http://redhat.com/ [EMAIL PROTECTED] | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/ -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [Libvir] Re: Proposal : add 3 functions to Libvirt API, for virtual CPUs
On Mon, Jul 10, 2006 at 12:44:34PM -0400, Daniel Veillard wrote: /** * virDomainGetVcpus: * @domain: pointer to domain object, or NULL for Domain0 * @info: pointer to an array of virVcpuInfo structures * @maxinfo: number of structures in info array * * Extract information about virtual CPUs of a domain, store it in info array. * * Returns the number of info filled in case of success, -1 in case of failure. */ int virDomainGetVcpus(virDomainPtr domain, virVcpuInfoPtr info, int maxinfo) Hum ... now the problem with that API entry point is that we 'burn' the maximum 256 processors in the ABI, i.e. if we ever need to go past 256 client and servers need to be recompiled. Maybe this is not a real problem in practice but that's annoying. Is there existing APIs doing this kind of things (in POSIX for example), and what hard limit did they use ? Maybe int virDomainGetVcpusNr(virDomainPtr domain, int nr, virVcpuInfoPtr info, int maxCPU); Where the maxCPU is defined by the client as the number of real CPU it defined in its virVcpuInfoPtr and then an iteration over the virtual CPU defined in the domain is possible too. Of course if the domain uses many virtual CPUs this would become expensive but somehow I don't see that being the common use, I would rather guess the domains created use a few CPUs even if instantiated on a very large machine. This with the following structure (in libvirt.h): /** * virVcpuInfo: structure for information about a virtual CPU in a domain. */ #define VIR_MAX_CPUS256 Hum, there is already NUMA machines with more than 256 processors, it's annoying to define an API limit when you know it is already breakable. Yeah, if we define such a hard limit in the ABI, then I can pretty much guarentee this will come back to bite us. As Daniel says, there are already machines in existance with huge number of CPUS, so I can easily see us hitting that limit within a pretty short time period - particularly now that Xen has been ported to IA64 - cf HP Superdome machines with 128 physical CPUs - add in dual core hyperthreading to that Regards, Dan -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=| -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
