Re: [openstack-dev] [glance] Does glance_store swift driver support range requests ?

2017-11-15 Thread John Dickinson 


On 15 Nov 2017, at 7:40, Jay Pipes wrote:

> On 11/15/2017 06:28 AM, Duncan Thomas wrote:
>> On 15 November 2017 at 11:15, Matt Keenan  wrote:
>>> On 13/11/17 22:51, Nikhil Komawar wrote:
>>>
>>> I think it will a rather hard problem to solve. As swift store can be
>>> configured to store objects in different configurations. I guess the next
>>> question would be, what is your underlying problem -- multiple build
>>> requests or is this for retry for a single download?
>>>
>>> If the image is in image cache and you are hitting the glance node with
>>> cached image (which is quite possible for tiny deployments), this feature
>>> will be relatively easier.
>>>
>>>
>>> So the specific image stored in glance is a Unified Archive
>>> (https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).
>>>
>>> During a UAR deployment the archive UUID is required and it is contained in
>>> the first 33 characters of the UAR image, thus a range request for this
>>> portion is required when initiating the deployment. Then the rest of the
>>> archive is extracted and deployed.
>>
>> Given the range you want is always at the beginning, is a range
>> request any different to doing a full get request and dripping the
>> connection when you've got the bytes you want?
>
> Or just store the UAR UUID in the image metadata...
>
> -jay

Swift supports range requests (and multiple ranges at the same time)

eg: "Range: bytes=1-34, 100-1024"


>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Does glance_store swift driver support range requests ?

2017-11-15 Thread Jay Pipes 

On 11/15/2017 06:28 AM, Duncan Thomas wrote:

On 15 November 2017 at 11:15, Matt Keenan  wrote:

On 13/11/17 22:51, Nikhil Komawar wrote:

I think it will a rather hard problem to solve. As swift store can be
configured to store objects in different configurations. I guess the next
question would be, what is your underlying problem -- multiple build
requests or is this for retry for a single download?

If the image is in image cache and you are hitting the glance node with
cached image (which is quite possible for tiny deployments), this feature
will be relatively easier.


So the specific image stored in glance is a Unified Archive
(https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).

During a UAR deployment the archive UUID is required and it is contained in
the first 33 characters of the UAR image, thus a range request for this
portion is required when initiating the deployment. Then the rest of the
archive is extracted and deployed.


Given the range you want is always at the beginning, is a range
request any different to doing a full get request and dripping the
connection when you've got the bytes you want?


Or just store the UAR UUID in the image metadata...

-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Does glance_store swift driver support range requests ?

2017-11-15 Thread Duncan Thomas 
On 15 November 2017 at 11:15, Matt Keenan  wrote:
> On 13/11/17 22:51, Nikhil Komawar wrote:
>
> I think it will a rather hard problem to solve. As swift store can be
> configured to store objects in different configurations. I guess the next
> question would be, what is your underlying problem -- multiple build
> requests or is this for retry for a single download?
>
> If the image is in image cache and you are hitting the glance node with
> cached image (which is quite possible for tiny deployments), this feature
> will be relatively easier.
>
>
> So the specific image stored in glance is a Unified Archive
> (https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).
>
> During a UAR deployment the archive UUID is required and it is contained in
> the first 33 characters of the UAR image, thus a range request for this
> portion is required when initiating the deployment. Then the rest of the
> archive is extracted and deployed.

Given the range you want is always at the beginning, is a range
request any different to doing a full get request and dripping the
connection when you've got the bytes you want?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Does glance_store swift driver support range requests ?

2017-11-15 Thread Matt Keenan 

On 13/11/17 22:51, Nikhil Komawar wrote:
I think it will a rather hard problem to solve. As swift store can be 
configured to store objects in different configurations. I guess the 
next question would be, what is your underlying problem -- multiple 
build requests or is this for retry for a single download?


If the image is in image cache and you are hitting the glance node 
with cached image (which is quite possible for tiny deployments), this 
feature will be relatively easier.




So the specific image stored in glance is a Unified Archive 
(https://docs.oracle.com/cd/E36784_01/html/E38524/gmrlo.html).


During a UAR deployment the archive UUID is required and it is contained 
in the first 33 characters of the UAR image, thus a range request for 
this portion is required when initiating the deployment. Then the rest 
of the archive is extracted and deployed.


I just want to know whether this capability is possible with swift ?

If I change the default_store in glance_api.conf  to say "file" e.g. 
default_store = file. restart devstack@g-api and then upload an archive. 
The curl request succeeds, so file based range requests are working, 
just the default swift setup is failing and I though it might be some 
conf file setting was required to enable the capability. Pretty sure 
cinder works (well it used to work last time I tried this which was with 
way back with Mitaka :) ).


Anyhow if it's not supported then I can work around at least initially 
using file store.


thanks

Matt




On Mon, Nov 13, 2017 at 6:47 AM, Matt Keenan > wrote:


Hi,

 Just configured devstack on Fedora 26, and by default
glance_store uses swift for image storage. When attempting to get
a specific range from a glance stored image, it's reporting range
requests are not supported e.g.:

    $ curl -i -X GET -r 0-32 -H "X-Auth-Token: $auth_token"
http://10.169.104.255/image/v2/images/29b7aa

    5e-3ec2-49b5-ab6b-d6cc5099f46c/file
    HTTP/1.1 400 Bad Request
    Date: Mon, 13 Nov 2017 10:43:23 GMT
    Server: Apache/2.4.27 (Fedora) OpenSSL/1.1.0f-fips
mod_wsgi/4.5.15 Python/2.7
    Content-Length: 205
    Content-Type: text/html; charset=UTF-8
    x-openstack-request-id: req-5ed2239f-165b-406f-969b-5cc4ab8c632d
    Connection: close

    
 
  400 Bad Request
 
 
  400 Bad Request
  Getting images randomly from this store is not supported.
Offset: 0, length: 33
 

Upon investigation, glance-api log is emitting:

    Nov 13 10:45:31 devstack@g-api.service[22783]:
#033[01;31mERROR glance.location [#033[01;36mNone
req-ad6da3f0-ead1-486a-a873-d301f02b0888 #033[00;36mdemo
demo#033[01;31m]
#033[01;35m#033[0│·1;31mGlance tried all
active locations to get data for image
29b7aa5e-3ec2-49b5-ab6b-d6cc5099f46c but all have failed.#033[00m:
StoreRandomGetNotSupported: Getting images randomly from this
store is notMDg4OCAjMDMzWzAwOzM2bW supported. Offset: 0, length: 33

The exception StoreRandomGetNotSupported is emitted by
glance_store from glance_store/capabilities.py:

    op_exec_map = {
    'get': (exceptions.StoreRandomGetNotSupported
    if kwargs.get('offset') or
kwargs.get('chunk_size') else
    exceptions.StoreGetNotSupported),
    'add': exceptions.StoreAddDisabled,
    'delete': exceptions.StoreDeleteNotSupported}

Looking at _driver/swift/store.py I think range requests are
supported, it I've be unsuccessful in configuring it.

Does the glance_store swift driver support range requests ?

Can it be configured within a conf file, by somehow adding a
capability ?

thanks

Matt

-- 



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





--
--
Thanks,
Nikhil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
HHGS : http://www.hh-gs.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Does glance_store swift driver support range requests ?

2017-11-13 Thread Nikhil Komawar 
I think it will a rather hard problem to solve. As swift store can be
configured to store objects in different configurations. I guess the next
question would be, what is your underlying problem -- multiple build
requests or is this for retry for a single download?

If the image is in image cache and you are hitting the glance node with
cached image (which is quite possible for tiny deployments), this feature
will be relatively easier.

On Mon, Nov 13, 2017 at 6:47 AM, Matt Keenan  wrote:

> Hi,
>
>  Just configured devstack on Fedora 26, and by default glance_store uses
> swift for image storage. When attempting to get a specific range from a
> glance stored image, it's reporting range requests are not supported e.g.:
>
> $ curl -i -X GET -r 0-32 -H "X-Auth-Token: $auth_token"
> http://10.169.104.255/image/v2/images/29b7aa
> 5e-3ec2-49b5-ab6b-d6cc5099f46c/file
> HTTP/1.1 400 Bad Request
> Date: Mon, 13 Nov 2017 10:43:23 GMT
> Server: Apache/2.4.27 (Fedora) OpenSSL/1.1.0f-fips mod_wsgi/4.5.15
> Python/2.7
> Content-Length: 205
> Content-Type: text/html; charset=UTF-8
> x-openstack-request-id: req-5ed2239f-165b-406f-969b-5cc4ab8c632d
> Connection: close
>
> 
>  
>   400 Bad Request
>  
>  
>   400 Bad Request
>   Getting images randomly from this store is not supported. Offset: 0,
> length: 33
>  
>
> Upon investigation, glance-api log is emitting:
>
> Nov 13 10:45:31 devstack@g-api.service[22783]: #033[01;31mERROR
> glance.location [#033[01;36mNone req-ad6da3f0-ead1-486a-a873-d301f02b0888
> #033[00;36mdemo demo#033[01;31m] 
> #033[01;35m#033[0│·1;31mGlance
> tried all active locations to get data for image
> 29b7aa5e-3ec2-49b5-ab6b-d6cc5099f46c but all have failed.#033[00m:
> StoreRandomGetNotSupported: Getting images randomly from this store is
> notMDg4OCAjMDMzWzAwOzM2bW supported. Offset: 0, length: 33
>
> The exception StoreRandomGetNotSupported is emitted by glance_store from
> glance_store/capabilities.py:
>
> op_exec_map = {
> 'get': (exceptions.StoreRandomGetNotSupported
> if kwargs.get('offset') or kwargs.get('chunk_size') else
> exceptions.StoreGetNotSupported),
> 'add': exceptions.StoreAddDisabled,
> 'delete': exceptions.StoreDeleteNotSupported}
>
> Looking at _driver/swift/store.py I think range requests are supported, it
> I've be unsuccessful in configuring it.
>
> Does the glance_store swift driver support range requests ?
>
> Can it be configured within a conf file, by somehow adding a capability ?
>
> thanks
>
> Matt
>
> --
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
--
Thanks,
Nikhil
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev