[openstack-dev] [Nova] pci stats format and functional tests

2015-03-05 Thread Murray, Paul (HP Cloud)
Hi All,

I know Yunhong Jiang and Daniel Berrange have been involved in the following, 
but I thought it worth sending to the list for visibility.

While writing code to convert the resource tracker to use the ComputeNode 
object realized that the api samples used in the functional tests are not the 
same as the format as the PciDevicePool object. For example: 
hypervisor-pci-detail-resp.json has something like this:

os-pci:pci_stats: [
{
count: 5,
extra_info: {
key1: value1,
phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]]
},
keya: valuea,
product_id: 1520,
vendor_id: 8086
}
],

My understanding from interactions with yjiang5 in the past leads me to think 
that something like this is what is actually expected:

os-pci:pci_stats: [
{
count: 5,
key1: value1,
phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]],
keya: valuea,
product_id: 1520,
vendor_id: 8086
}
],

This is the way the PciDevicePool object expects the data structure to be and 
is also the way the libvirt virt driver creates pci device information (i.e. 
without the extra_info key). Other than that (which is actually pretty clear) 
I couldn't find anything to tell me definitively if my interpretation is 
correct and I don't want to change the functional tests without being sure they 
are wrong. So if anyone can give some guidance here I would appreciate it.

I separated this stuff out into a patch with a couple of other minor cleanups 
in preparation for the ComputeNode change, see: 
https://review.openstack.org/#/c/161843

Let me know if I am on the right track,

Cheers,
Paul


Paul Murray
Nova Technical Lead, HP Cloud
+44 117 316 2527

Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12 1HN 
Registered No: 690597 England. The contents of this message and any attachments 
to it are confidential and may be legally privileged. If you have received this 
message in error, you should delete it from your system immediately and advise 
the sender. To any recipient of this message within HP, unless otherwise stated 
you should consider this message and attachments as HP CONFIDENTIAL.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] pci stats format and functional tests

2015-03-05 Thread Daniel P. Berrange
On Thu, Mar 05, 2015 at 07:39:23PM +, Murray, Paul (HP Cloud) wrote:
 Hi All,
 
 I know Yunhong Jiang and Daniel Berrange have been involved in the following, 
 but I thought it worth sending to the list for visibility.
 
 While writing code to convert the resource tracker to use the ComputeNode 
 object realized that the api samples used in the functional tests are not the 
 same as the format as the PciDevicePool object. For example: 
 hypervisor-pci-detail-resp.json has something like this:
 
 os-pci:pci_stats: [
 {
 count: 5,
 extra_info: {
 key1: value1,
 phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]]
 },
 keya: valuea,
 product_id: 1520,
 vendor_id: 8086
 }
 ],
 
 My understanding from interactions with yjiang5 in the past leads me to think 
 that something like this is what is actually expected:
 
 os-pci:pci_stats: [
 {
 count: 5,
 key1: value1,
 phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]],
 keya: valuea,
 product_id: 1520,
 vendor_id: 8086
 }
 ],
 
 This is the way the PciDevicePool object expects the data structure to be and 
 is also the way the libvirt virt driver creates pci device information (i.e. 
 without the extra_info key). Other than that (which is actually pretty 
 clear) I couldn't find anything to tell me definitively if my interpretation 
 is correct and I don't want to change the functional tests without being sure 
 they are wrong. So if anyone can give some guidance here I would appreciate 
 it.

I'm afraid I've not actually done any work with the PCI support at the
API level - only looked at the data reported by the hypervisors in the
get_available_resource method which isn't API sensitive as its internal
code. So probably need someone else to answer these questions.


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] pci stats format and functional tests

2015-03-05 Thread Jiang, Yunhong
Paul, you are right that the 'extra_info' should not be in the 
os-pci:pci_stats, since it's not part of 'pool-keys' anymore, but I'm not sure 
if both 'key1' and 'phys_function' will be part of the pci_stats.

Thanks
--jyh

From: Murray, Paul (HP Cloud) [mailto:pmur...@hp.com]
Sent: Thursday, March 5, 2015 11:39 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Nova] pci stats format and functional tests

Hi All,

I know Yunhong Jiang and Daniel Berrange have been involved in the following, 
but I thought it worth sending to the list for visibility.

While writing code to convert the resource tracker to use the ComputeNode 
object realized that the api samples used in the functional tests are not the 
same as the format as the PciDevicePool object. For example: 
hypervisor-pci-detail-resp.json has something like this:

os-pci:pci_stats: [
{
count: 5,
extra_info: {
key1: value1,
phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]]
},
keya: valuea,
product_id: 1520,
vendor_id: 8086
}
],

My understanding from interactions with yjiang5 in the past leads me to think 
that something like this is what is actually expected:

os-pci:pci_stats: [
{
count: 5,
key1: value1,
phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]],
keya: valuea,
product_id: 1520,
vendor_id: 8086
}
],

This is the way the PciDevicePool object expects the data structure to be and 
is also the way the libvirt virt driver creates pci device information (i.e. 
without the extra_info key). Other than that (which is actually pretty clear) 
I couldn't find anything to tell me definitively if my interpretation is 
correct and I don't want to change the functional tests without being sure they 
are wrong. So if anyone can give some guidance here I would appreciate it.

I separated this stuff out into a patch with a couple of other minor cleanups 
in preparation for the ComputeNode change, see: 
https://review.openstack.org/#/c/161843

Let me know if I am on the right track,

Cheers,
Paul


Paul Murray
Nova Technical Lead, HP Cloud
+44 117 316 2527

Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12 1HN 
Registered No: 690597 England. The contents of this message and any attachments 
to it are confidential and may be legally privileged. If you have received this 
message in error, you should delete it from your system immediately and advise 
the sender. To any recipient of this message within HP, unless otherwise stated 
you should consider this message and attachments as HP CONFIDENTIAL.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] pci stats format and functional tests

2015-03-05 Thread Robert Li (baoli)
“extra_info” is no longer a key in the stats pool, nor the “physical_function”. 
If you check pci/stats.py, the keys are pool_keys = ['product_id', 
'vendor_id', 'numa_node’] plus whatever tags are used in the whitelist. So I 
believe it’s something like this:
os-pci:pci_stats: [
{
count: 5,
key1: value1”,
…
“keyn: valuen,
product_id: 1520,
vendor_id: “8086”,
“numa_node
},
],

And each stats entry may have different keys.

thanks,
—Robert

On 3/5/15, 5:16 PM, Jiang, Yunhong 
yunhong.ji...@intel.commailto:yunhong.ji...@intel.com wrote:

Paul, you are right that the ‘extra_info’ should not be in the 
os-pci:pci_stats, since it’s not part of ‘pool-keys’ anymore, but I’m not sure 
if both ‘key1’ and ‘phys_function’ will be part of the pci_stats.

Thanks
--jyh

From: Murray, Paul (HP Cloud) [mailto:pmur...@hp.com]
Sent: Thursday, March 5, 2015 11:39 AM
To: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Nova] pci stats format and functional tests

Hi All,

I know Yunhong Jiang and Daniel Berrange have been involved in the following, 
but I thought it worth sending to the list for visibility.

While writing code to convert the resource tracker to use the ComputeNode 
object realized that the api samples used in the functional tests are not the 
same as the format as the PciDevicePool object. For example: 
hypervisor-pci-detail-resp.json has something like this:

os-pci:pci_stats: [
{
count: 5,
extra_info: {
key1: value1,
phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]]”
},
keya: valuea,
product_id: 1520,
vendor_id: 8086
}
],

My understanding from interactions with yjiang5 in the past leads me to think 
that something like this is what is actually expected:

os-pci:pci_stats: [
{
count: 5,
key1: value1,
phys_function: [[\0x\, \0x04\, \0x00\, \0x1\]]”,
keya: valuea,
product_id: 1520,
vendor_id: 8086
}
],

This is the way the PciDevicePool object expects the data structure to be and 
is also the way the libvirt virt driver creates pci device information (i.e. 
without the “extra_info” key). Other than that (which is actually pretty clear) 
I couldn’t find anything to tell me definitively if my interpretation is 
correct and I don’t want to change the functional tests without being sure they 
are wrong. So if anyone can give some guidance here I would appreciate it.

I separated this stuff out into a patch with a couple of other minor cleanups 
in preparation for the ComputeNode change, see: 
https://review.openstack.org/#/c/161843

Let me know if I am on the right track,

Cheers,
Paul


Paul Murray
Nova Technical Lead, HP Cloud
+44 117 316 2527

Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12 1HN 
Registered No: 690597 England. The contents of this message and any attachments 
to it are confidential and may be legally privileged. If you have received this 
message in error, you should delete it from your system immediately and advise 
the sender. To any recipient of this message within HP, unless otherwise stated 
you should consider this message and attachments as HP CONFIDENTIAL.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev