Re: [openstack-dev] [nova] Can all virt drivers provide a disk 'id' for the diagnostics API?

2016-09-26 Thread Daniel P. Berrange 
On Mon, Sep 26, 2016 at 09:31:39PM +0800, Alex Xu wrote:
> 2016-09-23 20:38 GMT+08:00 Daniel P. Berrange :
> 
> > On Fri, Sep 23, 2016 at 07:32:36AM -0500, Matt Riedemann wrote:
> > > On 9/23/2016 3:54 AM, Daniel P. Berrange wrote:
> > > > On Thu, Sep 22, 2016 at 01:54:21PM -0500, Matt Riedemann wrote:
> > > > > Sergey is working on a spec to use the standardized virt driver
> > instance
> > > > > diagnostics in the os-diagnostics API. A question came up during
> > review of
> > > > > the spec about how to define a disk 'id':
> > > > >
> > > > > https://review.openstack.org/#/c/357884/2/specs/ocata/
> > approved/restore-vm-diagnostics.rst@140
> > > > >
> > > > > The existing diagnostics code doesn't set a disk id in the list of
> > disk
> > > > > dicts, but I think with at least libvirt we can set that to the
> > target
> > > > > device from the disk device xml.
> > > > >
> > > > > The xenapi code for getting this info is a bit confusing for me at
> > least,
> > > > > but it looks like it's possible to get the disks, but the id might
> > need to
> > > > > be parsed out (as a side note, it looks like the cpu/memory/disk
> > diagnostics
> > > > > are not even populated in the get_instance_diagnostics method for
> > xen).
> > > > >
> > > > > vmware is in the same boat as xen, it's not fully implemented:
> > > > >
> > > > > https://github.com/openstack/nova/blob/
> > 64cbd7c51a5a82b965dab53eccfaecba45be9c27/nova/virt/
> > vmwareapi/vmops.py#L1561
> > > > >
> > > > > Hyper-v and Ironic virt drivers haven't implemented
> > get_instance_diagnostics
> > > > > yet.
> > > >
> > > > The key value of this field (which we should call "device_name", not
> > "id"),
> > > > is to allow the stats data to be correlated with the entries in the
> > block
> > > > device mapping list used to configure storage when bootin the VM. As
> > such
> > > > we should declare its value to match the corresponding field in BDM.
> > > >
> > > > Regards,
> > > > Daniel
> > > >
> > >
> > > Well, except that we don't want people specifying a device name in the
> > block
> > > device list when creating a server, and the libvirt driver ignores that
> > > altogether. In fact, I think Dan Smith was planning on adding a
> > microversion
> > > in Ocata to remove that field from the server create request since we
> > can't
> > > guarantee it's what you'll end up with for all virt drivers.
> >
> > We don't want people specifying it, but we should report the auto-allocated
> > names back when you query the data after instance creation, don't we ? If
> > we don't, then there's no way for users to correlate the disks that they
> > requested with the instance diagnostic stats, which severely limits their
> > usefulness.
> >
> 
> So what use-case for this API? I thought it is used by admin user to
> diagnose the cloud. If that is the right use-case, we can expose the disk
> image path in the API for admin user to correlate the disks. In the
> libvirt, it would looks like
> "/opt/stack/data/nova/instances/cbc7985c-434d-4ec3-8d96-d99ad6afb618/disk".
> As this is admin-only API, and for diagnostics, this info is safe to expose
> in this API.

You can't assume that all disks have a local path in the filesystem.
Any disks using a QEMU built-in network client (eg rbd) will not
appear there.


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Can all virt drivers provide a disk 'id' for the diagnostics API?

2016-09-26 Thread Alex Xu 
2016-09-23 20:38 GMT+08:00 Daniel P. Berrange :

> On Fri, Sep 23, 2016 at 07:32:36AM -0500, Matt Riedemann wrote:
> > On 9/23/2016 3:54 AM, Daniel P. Berrange wrote:
> > > On Thu, Sep 22, 2016 at 01:54:21PM -0500, Matt Riedemann wrote:
> > > > Sergey is working on a spec to use the standardized virt driver
> instance
> > > > diagnostics in the os-diagnostics API. A question came up during
> review of
> > > > the spec about how to define a disk 'id':
> > > >
> > > > https://review.openstack.org/#/c/357884/2/specs/ocata/
> approved/restore-vm-diagnostics.rst@140
> > > >
> > > > The existing diagnostics code doesn't set a disk id in the list of
> disk
> > > > dicts, but I think with at least libvirt we can set that to the
> target
> > > > device from the disk device xml.
> > > >
> > > > The xenapi code for getting this info is a bit confusing for me at
> least,
> > > > but it looks like it's possible to get the disks, but the id might
> need to
> > > > be parsed out (as a side note, it looks like the cpu/memory/disk
> diagnostics
> > > > are not even populated in the get_instance_diagnostics method for
> xen).
> > > >
> > > > vmware is in the same boat as xen, it's not fully implemented:
> > > >
> > > > https://github.com/openstack/nova/blob/
> 64cbd7c51a5a82b965dab53eccfaecba45be9c27/nova/virt/
> vmwareapi/vmops.py#L1561
> > > >
> > > > Hyper-v and Ironic virt drivers haven't implemented
> get_instance_diagnostics
> > > > yet.
> > >
> > > The key value of this field (which we should call "device_name", not
> "id"),
> > > is to allow the stats data to be correlated with the entries in the
> block
> > > device mapping list used to configure storage when bootin the VM. As
> such
> > > we should declare its value to match the corresponding field in BDM.
> > >
> > > Regards,
> > > Daniel
> > >
> >
> > Well, except that we don't want people specifying a device name in the
> block
> > device list when creating a server, and the libvirt driver ignores that
> > altogether. In fact, I think Dan Smith was planning on adding a
> microversion
> > in Ocata to remove that field from the server create request since we
> can't
> > guarantee it's what you'll end up with for all virt drivers.
>
> We don't want people specifying it, but we should report the auto-allocated
> names back when you query the data after instance creation, don't we ? If
> we don't, then there's no way for users to correlate the disks that they
> requested with the instance diagnostic stats, which severely limits their
> usefulness.
>

So what use-case for this API? I thought it is used by admin user to
diagnose the cloud. If that is the right use-case, we can expose the disk
image path in the API for admin user to correlate the disks. In the
libvirt, it would looks like
"/opt/stack/data/nova/instances/cbc7985c-434d-4ec3-8d96-d99ad6afb618/disk".
As this is admin-only API, and for diagnostics, this info is safe to expose
in this API.



>
> > I'm fine with calling the field device_name though.
>
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/
> :|
> |: http://libvirt.org  -o- http://virt-manager.org
> :|
> |: http://autobuild.org   -o- http://search.cpan.org/~danberr/
> :|
> |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc
> :|
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Can all virt drivers provide a disk 'id' for the diagnostics API?

2016-09-23 Thread Daniel P. Berrange 
On Fri, Sep 23, 2016 at 07:32:36AM -0500, Matt Riedemann wrote:
> On 9/23/2016 3:54 AM, Daniel P. Berrange wrote:
> > On Thu, Sep 22, 2016 at 01:54:21PM -0500, Matt Riedemann wrote:
> > > Sergey is working on a spec to use the standardized virt driver instance
> > > diagnostics in the os-diagnostics API. A question came up during review of
> > > the spec about how to define a disk 'id':
> > > 
> > > https://review.openstack.org/#/c/357884/2/specs/ocata/approved/restore-vm-diagnostics.rst@140
> > > 
> > > The existing diagnostics code doesn't set a disk id in the list of disk
> > > dicts, but I think with at least libvirt we can set that to the target
> > > device from the disk device xml.
> > > 
> > > The xenapi code for getting this info is a bit confusing for me at least,
> > > but it looks like it's possible to get the disks, but the id might need to
> > > be parsed out (as a side note, it looks like the cpu/memory/disk 
> > > diagnostics
> > > are not even populated in the get_instance_diagnostics method for xen).
> > > 
> > > vmware is in the same boat as xen, it's not fully implemented:
> > > 
> > > https://github.com/openstack/nova/blob/64cbd7c51a5a82b965dab53eccfaecba45be9c27/nova/virt/vmwareapi/vmops.py#L1561
> > > 
> > > Hyper-v and Ironic virt drivers haven't implemented 
> > > get_instance_diagnostics
> > > yet.
> > 
> > The key value of this field (which we should call "device_name", not "id"),
> > is to allow the stats data to be correlated with the entries in the block
> > device mapping list used to configure storage when bootin the VM. As such
> > we should declare its value to match the corresponding field in BDM.
> > 
> > Regards,
> > Daniel
> > 
> 
> Well, except that we don't want people specifying a device name in the block
> device list when creating a server, and the libvirt driver ignores that
> altogether. In fact, I think Dan Smith was planning on adding a microversion
> in Ocata to remove that field from the server create request since we can't
> guarantee it's what you'll end up with for all virt drivers.

We don't want people specifying it, but we should report the auto-allocated
names back when you query the data after instance creation, don't we ? If
we don't, then there's no way for users to correlate the disks that they
requested with the instance diagnostic stats, which severely limits their
usefulness.

> I'm fine with calling the field device_name though.


Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Can all virt drivers provide a disk 'id' for the diagnostics API?

2016-09-23 Thread Matt Riedemann 

On 9/23/2016 3:54 AM, Daniel P. Berrange wrote:

On Thu, Sep 22, 2016 at 01:54:21PM -0500, Matt Riedemann wrote:

Sergey is working on a spec to use the standardized virt driver instance
diagnostics in the os-diagnostics API. A question came up during review of
the spec about how to define a disk 'id':

https://review.openstack.org/#/c/357884/2/specs/ocata/approved/restore-vm-diagnostics.rst@140

The existing diagnostics code doesn't set a disk id in the list of disk
dicts, but I think with at least libvirt we can set that to the target
device from the disk device xml.

The xenapi code for getting this info is a bit confusing for me at least,
but it looks like it's possible to get the disks, but the id might need to
be parsed out (as a side note, it looks like the cpu/memory/disk diagnostics
are not even populated in the get_instance_diagnostics method for xen).

vmware is in the same boat as xen, it's not fully implemented:

https://github.com/openstack/nova/blob/64cbd7c51a5a82b965dab53eccfaecba45be9c27/nova/virt/vmwareapi/vmops.py#L1561

Hyper-v and Ironic virt drivers haven't implemented get_instance_diagnostics
yet.


The key value of this field (which we should call "device_name", not "id"),
is to allow the stats data to be correlated with the entries in the block
device mapping list used to configure storage when bootin the VM. As such
we should declare its value to match the corresponding field in BDM.

Regards,
Daniel



Well, except that we don't want people specifying a device name in the 
block device list when creating a server, and the libvirt driver ignores 
that altogether. In fact, I think Dan Smith was planning on adding a 
microversion in Ocata to remove that field from the server create 
request since we can't guarantee it's what you'll end up with for all 
virt drivers.


I'm fine with calling the field device_name though.

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Can all virt drivers provide a disk 'id' for the diagnostics API?

2016-09-23 Thread Daniel P. Berrange 
On Thu, Sep 22, 2016 at 01:54:21PM -0500, Matt Riedemann wrote:
> Sergey is working on a spec to use the standardized virt driver instance
> diagnostics in the os-diagnostics API. A question came up during review of
> the spec about how to define a disk 'id':
> 
> https://review.openstack.org/#/c/357884/2/specs/ocata/approved/restore-vm-diagnostics.rst@140
> 
> The existing diagnostics code doesn't set a disk id in the list of disk
> dicts, but I think with at least libvirt we can set that to the target
> device from the disk device xml.
> 
> The xenapi code for getting this info is a bit confusing for me at least,
> but it looks like it's possible to get the disks, but the id might need to
> be parsed out (as a side note, it looks like the cpu/memory/disk diagnostics
> are not even populated in the get_instance_diagnostics method for xen).
> 
> vmware is in the same boat as xen, it's not fully implemented:
> 
> https://github.com/openstack/nova/blob/64cbd7c51a5a82b965dab53eccfaecba45be9c27/nova/virt/vmwareapi/vmops.py#L1561
> 
> Hyper-v and Ironic virt drivers haven't implemented get_instance_diagnostics
> yet.

The key value of this field (which we should call "device_name", not "id"),
is to allow the stats data to be correlated with the entries in the block
device mapping list used to configure storage when bootin the VM. As such
we should declare its value to match the corresponding field in BDM.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev