[et-mgmt-tools] Re: RFC: libosinfo: Library for virt OS/distro metadata 3
On Mon, Jun 15, 2009 at 05:00:00PM +0100, Daniel P. Berrange wrote: > > I don't really like the idea of filtering. Instead it should just be a > > callback API full stop. It's up to the client if they want to filter, or > > find a specific entry, or whatever. > > So you mean that it can just be a list iterator pattern > >typedef int (*osdistro_iter)(osdistro_t *distro, void *opaque); >int os_distro_iterate(osinfo_t, osdistro_iter iter, void *opaque); Yep. regards john ___ et-mgmt-tools mailing list et-mgmt-tools@redhat.com https://www.redhat.com/mailman/listinfo/et-mgmt-tools
[et-mgmt-tools] Re: RFC: libosinfo: Library for virt OS/distro metadata 3
On Mon, Jun 15, 2009 at 11:50:18AM -0400, John Levon wrote: > On Mon, Jun 15, 2009 at 04:27:49PM +0100, Daniel P. Berrange wrote: > > > > > - Find OS distros, matching a specific set of properties > > > > > > What uses do you have in mind for this? Being able to say e.g. 'all > > > distros that support xen PV for IA64'? > > > > > > How do you think an API call would look? > > > > Just a function callback for doing the filtering. If the callback > > returned 1 the distro object is kept, otherwise it is discarded. > > > > typedef int (*osfilter-t)(osdistro_t *distro, void *opaque); > > osdistro_t **os_distro_find(osinfo_t, osfilter_t filter, void *opaque); > > I don't really like the idea of filtering. Instead it should just be a > callback API full stop. It's up to the client if they want to filter, or > find a specific entry, or whatever. So you mean that it can just be a list iterator pattern typedef int (*osdistro_iter)(osdistro_t *distro, void *opaque); int os_distro_iterate(osinfo_t, osdistro_iter iter, void *opaque); > > Imagine virt-manager building a 2 level hiearchy. For the first level > > it wants to use this method > > > >- Get a list of uninque vendor names among all known OS distros > > Pruning duplicates is surely something done in the client. It > really doesn't need library help... I was thinking that it may well be more efficient to let the library do it. That said, this is an optimization we can add later if observed to be neccessary in real world Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| ___ et-mgmt-tools mailing list et-mgmt-tools@redhat.com https://www.redhat.com/mailman/listinfo/et-mgmt-tools
[et-mgmt-tools] Re: RFC: libosinfo: Library for virt OS/distro metadata 3
On Mon, Jun 15, 2009 at 04:27:49PM +0100, Daniel P. Berrange wrote: > > > - Find OS distros, matching a specific set of properties > > > > What uses do you have in mind for this? Being able to say e.g. 'all > > distros that support xen PV for IA64'? > > > > How do you think an API call would look? > > Just a function callback for doing the filtering. If the callback > returned 1 the distro object is kept, otherwise it is discarded. > > typedef int (*osfilter-t)(osdistro_t *distro, void *opaque); > osdistro_t **os_distro_find(osinfo_t, osfilter_t filter, void *opaque); I don't really like the idea of filtering. Instead it should just be a callback API full stop. It's up to the client if they want to filter, or find a specific entry, or whatever. The lifetime rules are such that an entry exists whilst the handle is open, so there's no issues with borrowing pointers in the callback etc. > Imagine virt-manager building a 2 level hiearchy. For the first level > it wants to use this method > >- Get a list of uninque vendor names among all known OS distros Pruning duplicates is surely something done in the client. It really doesn't need library help... regards john ___ et-mgmt-tools mailing list et-mgmt-tools@redhat.com https://www.redhat.com/mailman/listinfo/et-mgmt-tools
[et-mgmt-tools] Re: RFC: libosinfo: Library for virt OS/distro metadata 3
On Mon, Jun 15, 2009 at 11:09:56AM -0400, Cole Robinson wrote:
> Daniel P. Berrange wrote:
> > On Sun, Jun 14, 2009 at 06:50:02PM -0400, Cole Robinson wrote:
> > For OS distros I think we need APIs to:
> >
> > - List all OS distros
> > - Find OS distros, matching a specific set of properties
>
> What uses do you have in mind for this? Being able to say e.g. 'all
> distros that support xen PV for IA64'?
>
> How do you think an API call would look?
Just a function callback for doing the filtering. If the callback
returned 1 the distro object is kept, otherwise it is discarded.
typedef int (*osfilter-t)(osdistro_t *distro, void *opaque);
osdistro_t **os_distro_find(osinfo_t, osfilter_t filter, void *opaque);
> > - Read a property from an OS distro
> > - Read all properties from an OS distro
>
> Why would an API user want to do this? I don't see why we would need to
> enable this specifically, rather than make the user do this iteratively.
>
> > - List unique values for a property across all distros
> >
>
> Not sure I fully understand this. Why would a user want this?
Imagine virt-manager building a 2 level hiearchy. For the first level
it wants to use this method
- Get a list of uninque vendor names among all known OS distros
eg
char **vendors = os_info_unique_properties(osinfo, "vendor");
Upon selecting a vendor, it then wants the earlier method
- Get a list of all OS distros for that vendor
eg
int vendorfilter(osdistro_t distro, void *opaque) {
char *wantvendor = opaque;
char *gotvendor = os_distro_get_prop(distro, "vendor");
if (gotvendor && STREQ(wantvendor, gotvendor)
return 1;
return 0;
}
osdistro_t *distros = os_distro_find(osinfo, vendorfilter, "Red Hat");
> >>
> >> - OS being installed
> >> - Virt type ('hvm' vs. 'xen')
> >> - Guest Architecture (i386, x86_64, ...)
> >> - Hypervisor (kvm, qemu, xen, vbox, ...)
> >> - Hypervisor version
> >> - Libvirt version
> >>
> >> We would need to find the intersection of what the OS, the hypervisor,
> >> and libvirt support, and return what we decide is the best choice.
> >>
> >> How to expose this in the API? We could simply have one long function
> >>
> >> os_lookup_device_value(char *os_id, char *virt_type, char *arch, ...)
> >>
> >> It works, but its pretty tedious, and I'm afraid that we would need
> >> even more info to make a correct choice in the future, and the above
> >> isn't flexible. We may also need some of the above info for other values
> >> (ACPI/APIC settings, returning a proper install url may depend on arch).
> >> Any suggestions?
> >
> > The more I think about this, the more I think we should avoid any specific
> > named attributes in the API. Supported devices are just other types of
> > property we can associated with a distro, in addition to ones I already
> > listed earlier. This could be useful in the UI too, for example, if you
> > know the hypervisor requires support for 'Xen paravirt disk', then when
> > browsing OS, you can filter on this property just as you would with the
> > others.
> >
>
> If the app already knows their hypervisor requires 'Xen paravirt disk',
> the above is fine, and we should facilitate filtering like that. However
> I would like osinfo to save the user from having to know those details:
> they should just be able to say 'I'm using xenpv on i386 for xen 1.2.3
> and libvirt 4.5.6 with distro fedora10' and osinfo can return the
> required info. So I still don't see how to solve the above problem.
>
> Let's say we are installing winxp on qemu via libvirt: we want to know
> the recommended sound device model:
>
> winxp prefers ac97, es1370
> qemu supports es1370, if >= 0.10.0, supports ac97
> libvirt supports es1370, if >= 0.6.0 supports ac97
>
> How do we solve this via the API?
The latter 2 questions really say that libvirt needs to export more info
about what a driver has.
For the former, the OS info database needs to somehow provide a list of
drivers available for each OS. Obviously you can take the intersection,
but perhaps also define that they are listed in preferred order, "best"
first.
Perhaps we should be really ambitious and for each OS distro allow for a
full list of PCI device IDs it supports. For linux you can auto-generate
that from the kernel module metadata. You could even imagine that the
distro provide an XML file in our format with this info in their release
media/trees.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
___
et-mgmt-tools mailing list
et-mgmt-tools@redhat.com
https://www.redhat.com/mailman/listinfo/e
[et-mgmt-tools] Re: RFC: libosinfo: Library for virt OS/distro metadata 3
Daniel P. Berrange wrote:
> On Sun, Jun 14, 2009 at 06:50:02PM -0400, Cole Robinson wrote:
>> The public API looks like:
>>
>> /**
>> * Values stored in the OS dictionary
>> */
>> enum _os_value_type {
>> OS_VALUE_NAME = 1, /** Human readable family/distro... name */
>> OS_VALUE_MEDIA_INSTALL_URL, /** URL to an install tree */
>> };
>> typedef enum _os_value_type os_value_t;
>>
>> int os_init();
>> voidos_close();
>>
>> int os_find_families(char ***list);
>> int os_find_distros (const char *parent_id, char ***list);
>> int os_find_releases(const char *parent_id, char ***list);
>> int os_find_updates (const char *parent_id, char ***list);
>>
>> int os_lookup_value (os_value_t value_type,
>> const char *os_id,
>> char **value);
>>
>> The unique identifier for each distro is its 'id', which is a simple human
>> readable string, similar to values we use for virt-install --os-variant
>> today.
>
> As John suggested, I think we'd be safer having opaque structs for the
> conceptual objects. One for the library itself, and another for an OS
> distro.
>
> Perhaps have an
>
>'os_info_t'as a handle for a library itself returned by os_init
>'os_distro_t' as a handle for a single OS distro instance
>
>
> os_info_t os_info_new()
> os_info_init(os_info_t *info, char *uri); /* loads the XML data */
>
Sounds good, though why have a separate os_info_new()? And I'd rather
have a separate API for initializing from a file, since that should be
uncommon enough that we don't need to force uri=NULL on most users.
int os_info_init(os_info_t **info)
int os_info_init_from_uri(os_info_t **info, char *uri)
Though it's a minor distinction for now.
> For OS distros I think we need APIs to:
>
> - List all OS distros
> - Find OS distros, matching a specific set of properties
What uses do you have in mind for this? Being able to say e.g. 'all
distros that support xen PV for IA64'?
How do you think an API call would look?
> - Read a property from an OS distro
> - Read all properties from an OS distro
Why would an API user want to do this? I don't see why we would need to
enable this specifically, rather than make the user do this iteratively.
> - List unique values for a property across all distros
>
Not sure I fully understand this. Why would a user want this?
>> The user will ask the API for available families/distros/releases/updates,
>> which will return a list of ids. We then pass an id to os_lookup_value to
>> actually retrieve data. The family/distro/... separation will likely be
>> removed pretty soon, in favor of an arbitrary hierarchy, where every OS
>> can have child OSes: no doubt hardcoding the family/distro/... split would
>> come back to bite us in the ass.
>
> I agree, the fixed hierarchy I describe really doesn't seem very nice
> looking back on it. The names I gave them are rather contrived and only
> really map nicely onto RHEL/Fedora release process. I think we're better
> off being more flexible and allowing for arbitrary relationships in the
> data files and API. I don't think we neccessarily want to force a single
> rooted tree structure here. The key important factor with the hierarchy
> is the concept of sharing metadata.
>
> I think we should take a hint from the way RDF works and define the API
> and XML format as a flat list, but allow relationships to be defined,
> and also allow tagging.
>
> - Flat list of OS distros with their full name, as defined by their
>vendor/distributor
>
> "Red Hat Enterprise Linux 4.7"
> "Red Hat Enterprise Linux 5.0"
> "Fedora 10"
> "Fedora 10"
> "Debian Sarge"
>
> - A 'derived' property. Allows derived distros to declare
> they should inherit metdata (eg Scientific Linux derives from
> RHEL)
>
> - A 'clone' property. Allows functionally identical rebuilds
>to declare they use exactly same metadata. (eg CentOS / RHEL)
>
> - A 'upgrades' property. Allows to indicate 'Fedora 11' is the
>release following on from 'Fedora 10'.
>
> - A 'publisher' property to give name of entity producing the
>distro eg 'Fedora Project', 'Red Hat', 'Microsoft'
>
> - A 'kernel type' and 'kernel version' property, eg 'linux'
>and '2.6.26'.
>
>
> Application UI might simulate a hierarchy by using the 'publisher'
> property at first level, and then filtering the flat list of OS
> distros at the 2nd level according to selected publisher. This
> satisfies the key 'UI' reason for the hierarchy. The 'derived'
> and 'clone' allow for inheritance of metadata.
>
I like this idea: certainly will give more flexibility.
>> So, things that I'm interested in feedback on:
>>
>> - How do we expect apps to list OS choices? Currently, virt-manager lists
>> type (linux, windows, unix, etc.) and associated distros (Fedora 8, RHEL4,
>> Debian Lenn
[et-mgmt-tools] Re: RFC: libosinfo: Library for virt OS/distro metadata 3
On Sun, Jun 14, 2009 at 06:50:02PM -0400, Cole Robinson wrote:
> The public API looks like:
>
> /**
> * Values stored in the OS dictionary
> */
> enum _os_value_type {
> OS_VALUE_NAME = 1, /** Human readable family/distro... name */
> OS_VALUE_MEDIA_INSTALL_URL, /** URL to an install tree */
> };
> typedef enum _os_value_type os_value_t;
>
> int os_init();
> voidos_close();
>
> int os_find_families(char ***list);
> int os_find_distros (const char *parent_id, char ***list);
> int os_find_releases(const char *parent_id, char ***list);
> int os_find_updates (const char *parent_id, char ***list);
>
> int os_lookup_value (os_value_t value_type,
> const char *os_id,
> char **value);
>
> The unique identifier for each distro is its 'id', which is a simple human
> readable string, similar to values we use for virt-install --os-variant today.
As John suggested, I think we'd be safer having opaque structs for the
conceptual objects. One for the library itself, and another for an OS
distro.
Perhaps have an
'os_info_t'as a handle for a library itself returned by os_init
'os_distro_t' as a handle for a single OS distro instance
os_info_t os_info_new()
os_info_init(os_info_t *info, char *uri); /* loads the XML data */
For OS distros I think we need APIs to:
- List all OS distros
- Find OS distros, matching a specific set of properties
- Read a property from an OS distro
- Read all properties from an OS distro
- List unique values for a property across all distros
> The user will ask the API for available families/distros/releases/updates,
> which will return a list of ids. We then pass an id to os_lookup_value to
> actually retrieve data. The family/distro/... separation will likely be
> removed pretty soon, in favor of an arbitrary hierarchy, where every OS
> can have child OSes: no doubt hardcoding the family/distro/... split would
> come back to bite us in the ass.
I agree, the fixed hierarchy I describe really doesn't seem very nice
looking back on it. The names I gave them are rather contrived and only
really map nicely onto RHEL/Fedora release process. I think we're better
off being more flexible and allowing for arbitrary relationships in the
data files and API. I don't think we neccessarily want to force a single
rooted tree structure here. The key important factor with the hierarchy
is the concept of sharing metadata.
I think we should take a hint from the way RDF works and define the API
and XML format as a flat list, but allow relationships to be defined,
and also allow tagging.
- Flat list of OS distros with their full name, as defined by their
vendor/distributor
"Red Hat Enterprise Linux 4.7"
"Red Hat Enterprise Linux 5.0"
"Fedora 10"
"Fedora 10"
"Debian Sarge"
- A 'derived' property. Allows derived distros to declare
they should inherit metdata (eg Scientific Linux derives from
RHEL)
- A 'clone' property. Allows functionally identical rebuilds
to declare they use exactly same metadata. (eg CentOS / RHEL)
- A 'upgrades' property. Allows to indicate 'Fedora 11' is the
release following on from 'Fedora 10'.
- A 'publisher' property to give name of entity producing the
distro eg 'Fedora Project', 'Red Hat', 'Microsoft'
- A 'kernel type' and 'kernel version' property, eg 'linux'
and '2.6.26'.
Application UI might simulate a hierarchy by using the 'publisher'
property at first level, and then filtering the flat list of OS
distros at the 2nd level according to selected publisher. This
satisfies the key 'UI' reason for the hierarchy. The 'derived'
and 'clone' allow for inheritance of metadata.
>
> So, things that I'm interested in feedback on:
>
> - How do we expect apps to list OS choices? Currently, virt-manager lists
> type (linux, windows, unix, etc.) and associated distros (Fedora 8, RHEL4,
> Debian Lenny, etc.). The linux/windows/unix info isn't represented in the
> xml (should it be?) so the best way seems to be:
>
> Distro
> |
> --> Release
> |
> --> Update
>
> Ex.
>
> RHEL
> |
> -> RHEL5
> |
> -> 5.0
>5.1
>5.2
>
> If we do away with the family/distro/... distinction, the user won't have
> much choice in the matter, but the 'family' concept (e.g. value of
> 'Red Hat') isn't very useful to expose to a user.
We should try to avoid forcing one representation onto apps. I think the
flat OS list + sets of properties will allow apps to build a variety of
UI models for this, either search based, tree based or filter based.
> - How should we handle derivatives like Scientific Linux + CentOS: should we
> expect users to understand they are based on RHEL, or give them explicit
> IDs?
They need explicit IDs, since they have unique download URLs that
