Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) allow one driver to call another driver's public method?

2016-03-25 Thread Fox, Kevin M 
Sounds like a reasonable optimization, but may not always be possible. Take, 
for example, the ceph backend. For other backends that are appliances, they 
would have to be able to implement a ceph client that matches the version of 
the ceph servers in use. That might be very difficult to support.

So maybe allow it, but have some automatic fall back mechanism if it fails?

Thanks,
Kevin

From: liuxinguo [liuxin...@huawei.com]
Sent: Thursday, March 24, 2016 6:44 PM
To: OpenStack Development Mailing List (not for usage questions)
Cc: Zhangli (ISSP); Luozhen
Subject: Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder 
team) allow one driver to call another driver's public method?

Hi Jon,

My basic idea is that, when we need to migrate a volume from my (vendor of) 
storage array to another backend (storage array), we will firstly call the 
other backend's create_volume() to create a volume on the other backend,
and then we will take my backend (storage array) as a "host" and attach other 
backend's volume to our backend (storage array). And then we can migrate the 
volume from our storage array to other storage array directly and very 
efficiently.
And at last when the migration finished, we call the other backend's 
terminate_connection() to detach the volume from our storage array.

Of course, the precondition is that our backend array is connected (can copy 
data) to other backend array so we can migrate volume directly between them.

--
Wilson Liu

-邮件原件-
发件人: Jon Bernard [mailto:jbern...@tuxion.com]
发送时间: 2016年3月23日 4:44
收件人: OpenStack Development Mailing List (not for usage questions)
抄送: Luozhen
主题: Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) 
allow one driver to call another driver's public method?

* liuxinguo <liuxin...@huawei.com> wrote:
> Hi Cinder team,
>
> We are going to implement storage-assisted volume migrate in our
> driver between different backend storage array or even different array
> of different vendors.  This is really high-efficiency than the
> host-copy migration between different array of different vendors.

Could you elaborate more on this?  During a volume migration operation we give 
the driver an opportunity to more-intelligently relocate the volume's data.  
This is done through the migrate_volume() method defined in the driver itself.  
If this method exists, it will be called before falling back to a byte-for-byte 
copy approach - and if it succeeds the volume is considered migrated and the 
operation returns.

Is this what you were looking for, or did you have something different in mind?

--
Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) allow one driver to call another driver's public method?

2016-03-24 Thread liuxinguo 
Hi Jon,

My basic idea is that, when we need to migrate a volume from my (vendor of) 
storage array to another backend (storage array), we will firstly call the 
other backend's create_volume() to create a volume on the other backend, 
and then we will take my backend (storage array) as a "host" and attach other 
backend's volume to our backend (storage array). And then we can migrate the 
volume from our storage array to other storage array directly and very 
efficiently.
And at last when the migration finished, we call the other backend's 
terminate_connection() to detach the volume from our storage array.

Of course, the precondition is that our backend array is connected (can copy 
data) to other backend array so we can migrate volume directly between them.

--
Wilson Liu

-邮件原件-
发件人: Jon Bernard [mailto:jbern...@tuxion.com] 
发送时间: 2016年3月23日 4:44
收件人: OpenStack Development Mailing List (not for usage questions)
抄送: Luozhen
主题: Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) 
allow one driver to call another driver's public method?

* liuxinguo <liuxin...@huawei.com> wrote:
> Hi Cinder team,
> 
> We are going to implement storage-assisted volume migrate in our 
> driver between different backend storage array or even different array 
> of different vendors.  This is really high-efficiency than the 
> host-copy migration between different array of different vendors.

Could you elaborate more on this?  During a volume migration operation we give 
the driver an opportunity to more-intelligently relocate the volume's data.  
This is done through the migrate_volume() method defined in the driver itself.  
If this method exists, it will be called before falling back to a byte-for-byte 
copy approach - and if it succeeds the volume is considered migrated and the 
operation returns.

Is this what you were looking for, or did you have something different in mind?

--
Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) allow one driver to call another driver's public method?

2016-03-22 Thread Jon Bernard 
* liuxinguo  wrote:
> Hi Cinder team,
> 
> We are going to implement storage-assisted volume migrate in our
> driver between different backend storage array or even different array
> of different vendors.  This is really high-efficiency than the
> host-copy migration between different array of different vendors.

Could you elaborate more on this?  During a volume migration operation
we give the driver an opportunity to more-intelligently relocate the
volume's data.  This is done through the migrate_volume() method defined
in the driver itself.  If this method exists, it will be called before
falling back to a byte-for-byte copy approach - and if it succeeds the
volume is considered migrated and the operation returns.

Is this what you were looking for, or did you have something different
in mind?

-- 
Jon

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) allow one driver to call another driver's public method?

2016-03-19 Thread Sean McGinnis 
On Fri, Mar 18, 2016 at 04:05:34AM +, liuxinguo wrote:
> Hi Cinder team,
> 
> We are going to implement storage-assisted volume migrate in our driver 
> between different backend storage array or even different array of different 
> vendors.
> This is really high-efficiency than the host-copy migration between different 
> array of different vendors.
> 
> To implement this, we need to call other backend's method like 
> create_volume() or initialize_connection(). We can call them like the 
> cinder/volume/manage.py:
> 
> rpcapi.create_volume(ctxt, new_volume, host['host'],
>  None, None, allow_reschedule=False)
> 
> or
> conn = rpcapi.initialize_connection(ctxt, volume, properties)
> 
> And my question is: Does the OpenStack community(or Cinder team) allow driver 
> to call rpcapi in order to call other driver's method like create_volume() or 
> initialize_connection()?
> 

This is an interesting question. I have thought in the past we may be
able to do some interesting things, particularly with more involved
replication or migration scenarios.

We do not currently do this. Ideally I think we would want the other
driver instance passed in to the source driver so each driver would not
need to do something special to look it up.

You do have the option today of optimizing migrate for your driver [1].
But I think especially in cross-vendor migrations, there are things that
need to be done outside the scope of a driver that are currently handled
by Cinder.

There could be a valid use case for driver to driver interfaces, but I
think as it is now, what I think you are looking for is something that
is a little more involved and would need a little more design (and a lot
more discussion) to support.

[1]
https://github.com/openstack/cinder/blob/master/cinder/volume/driver.py#L1552

> 
> Thanks for any input!
> --
> Wilson Liu

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] Does the OpenStack community(or Cinder team) allow one driver to call another driver's public method?

2016-03-19 Thread liuxinguo 
Hi Cinder team,

We are going to implement storage-assisted volume migrate in our driver between 
different backend storage array or even different array of different vendors.
This is really high-efficiency than the host-copy migration between different 
array of different vendors.

To implement this, we need to call other backend's method like create_volume() 
or initialize_connection(). We can call them like the cinder/volume/manage.py:

rpcapi.create_volume(ctxt, new_volume, host['host'],
 None, None, allow_reschedule=False)

or
conn = rpcapi.initialize_connection(ctxt, volume, properties)

And my question is: Does the OpenStack community(or Cinder team) allow driver 
to call rpcapi in order to call other driver's method like create_volume() or 
initialize_connection()?


Thanks for any input!
--
Wilson Liu
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) allow one driver to call another driver's public method?

2016-03-19 Thread John Griffith 
On Thu, Mar 17, 2016 at 10:05 PM, liuxinguo  wrote:

> Hi Cinder team,
>
>
>
> We are going to implement storage-assisted volume migrate in our driver
> between different backend storage array or even different array of
> different vendors.
>
> This is really high-efficiency than the host-copy migration between
> different array of different vendors.
>
>
>
> To implement this, we need to call other backend’s method like
> create_volume() or initialize_connection(). We can call them like the
> cinder/volume/manage.py:
>
>
>
> rpcapi.create_volume(ctxt, new_volume, host[*'host'*],
>
>  None, None, allow_reschedule=False)
>
>
>
> or
>
> conn = rpcapi.initialize_connection(ctxt, volume, properties)
>
>
>
> And my question is: Does the OpenStack community(or Cinder team) allow
> driver to call rpcapi in order to call other driver’s method like
> create_volume() or initialize_connection()?
>
>
>
>
>
> Thanks for any input!
>
> --
>
> Wilson Liu
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> ​Hi Wilson,

We don't have a direct version of the model you describe in place
currently.  We do have something similar by coordinating through the api
and scheduler layers, there's even some existing code in the migrate code
that does some similar things in manager.py.

It would be good to see if you could leverage some of the design that's
already in place.  I don't know that there should be an objection to having
a driver call another driver but it really depends on how in depth it ends
up being and how all the details around context and quota are dealt with.

I'd be curious to see what you've got going.  Might be something that helps
make the migrate code we already have better?

Thanks,
John​
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Does the OpenStack community(or Cinder team) allow one driver to call another driver's public method?

2016-03-19 Thread Eric Harney 
On 03/18/2016 10:01 AM, Sean McGinnis wrote:
> On Fri, Mar 18, 2016 at 04:05:34AM +, liuxinguo wrote:
>> Hi Cinder team,
>>
>> We are going to implement storage-assisted volume migrate in our driver 
>> between different backend storage array or even different array of different 
>> vendors.
>> This is really high-efficiency than the host-copy migration between 
>> different array of different vendors.
>>
>> To implement this, we need to call other backend's method like 
>> create_volume() or initialize_connection(). We can call them like the 
>> cinder/volume/manage.py:
>>
>> rpcapi.create_volume(ctxt, new_volume, host['host'],
>>  None, None, allow_reschedule=False)
>>
>> or
>> conn = rpcapi.initialize_connection(ctxt, volume, properties)
>>
>> And my question is: Does the OpenStack community(or Cinder team) allow 
>> driver to call rpcapi in order to call other driver's method like 
>> create_volume() or initialize_connection()?
>>
> 
> This is an interesting question. I have thought in the past we may be
> able to do some interesting things, particularly with more involved
> replication or migration scenarios.
> 
> We do not currently do this. Ideally I think we would want the other
> driver instance passed in to the source driver so each driver would not
> need to do something special to look it up.
> 

I believe Jon Bernard researched this same idea a bit while implementing
generic volume migration [2] and found that there were a handful of
reasons that it doesn't really work.

[2] https://review.openstack.org/#/c/187270/

> You do have the option today of optimizing migrate for your driver [1].
> But I think especially in cross-vendor migrations, there are things that
> need to be done outside the scope of a driver that are currently handled
> by Cinder.
> 
> There could be a valid use case for driver to driver interfaces, but I
> think as it is now, what I think you are looking for is something that
> is a little more involved and would need a little more design (and a lot
> more discussion) to support.
> 
> [1]
> https://github.com/openstack/cinder/blob/master/cinder/volume/driver.py#L1552
> 
>>
>> Thanks for any input!
>> --
>> Wilson Liu
> 




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev