Re: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

[Brad Pokorny]

Sorry guys, I'm a little late responding to this topic.

>From a Horizon perspective, it would help if there were an API to discover
whether custom locations are off or on in a Glance v2 installation. We
could then expose it to the user only if it's enabled.

What we've done to prevent this from being a blocker to v2 adoption so far
is add a config value in Horizon that allows an operator to turn it off
explicitly [0], but that requires operators to turn it on/off in 2
services separately.

[0] https://review.openstack.org/#/c/320039/

Thanks,
Brad

On 7/27/16, 5:08 AM, "Flavio Percoco"  wrote:

>On 26/07/16 14:32 +0300, Mikhail Fedosin wrote:
>>Hello!
>>
>>As you may know glance v1 is going to be deprecated in Newton cycle.
>>Almost
>>all projects support glance v2 at this moment, Nova uses it by default.
>>Only one thing that blocks us from complete adoption is a possibility to
>>set custom locations to images. In v1 any user can set a location to his
>>image, but in v2 this functionality is not allowed by default, which
>>prevents v2 adoption in services like Horizon or Heat.
>>
>>It all happens because of differences between v1 and v2 locations. In v1
>>it
>>is pretty easy - user specifies an url and send a request, glance adds
>>this
>>url to the image and activates it.
>>In v2 things are more complicated: v2 supports multiple locations per
>>image, which means that when user wants to download image file glance
>>will
>>choose the best one from the list of locations. It leads to some
>>inconsistencies: user can add or delete locations from his image even if
>>it
>>is active.
>>
>>To enable adding custom locations operator has to set True to config
>>option
>>'show_multiple_locations'. After that any user will be able to add or
>>remove his image locations, update locations metadata, and finally see
>>locations of all images even if they were uploaded to local storage. All
>>this things are not desired if glance v2 has public interface, because it
>>exposes inner cloud architecture. It leads to the fact that Heat and
>>Horizon and Nova in some cases and other services that used to set custom
>>locations in glance v1 won't be able to adopt glance v2. Unfortunately,
>>removing this behavior in v2 isn't easy, because it requires serious
>>architecture changes and breaks API. Moreover, many vendors use these
>>features in their clouds for private glance deployments and they really
>>won't like if we break anything.
>>
>>So, I want to hear opinions from Glance community and other involved
>>people.
>
>I agree the current situation is not ideal but I don't think there's a
>perfect
>solution that will let other services magically use the location's
>implementation in v2. The API itself is different and it requires a
>different
>call.
>
>With that in mind, I think the right thing to do here is to get rid of
>that
>option[0] and let operators manage this through poilicies. This does not
>mean
>the policies available are perfect.
>
>I'm not an expert on service tokens but I think we said that we could
>probably
>just use service tokens to allow for this feature to be used by other
>services
>instead of keeping it wide open everywhere.
>
>While I don't think the current situation is ideal, I think it's better
>than
>keeping it wide open.
>
>Hope the above helps,
>Flavio
>
>[0] https://review.openstack.org/#/c/313936/
>
>>
>>Best regards,
>>Mikhail Fedosin
>
>>_
>>_
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: 
>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>-- 
>@flaper87
>Flavio Percoco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

[Flavio Percoco]

On 26/07/16 14:32 +0300, Mikhail Fedosin wrote:

Hello!

As you may know glance v1 is going to be deprecated in Newton cycle. Almost
all projects support glance v2 at this moment, Nova uses it by default.
Only one thing that blocks us from complete adoption is a possibility to
set custom locations to images. In v1 any user can set a location to his
image, but in v2 this functionality is not allowed by default, which
prevents v2 adoption in services like Horizon or Heat.

It all happens because of differences between v1 and v2 locations. In v1 it
is pretty easy - user specifies an url and send a request, glance adds this
url to the image and activates it.
In v2 things are more complicated: v2 supports multiple locations per
image, which means that when user wants to download image file glance will
choose the best one from the list of locations. It leads to some
inconsistencies: user can add or delete locations from his image even if it
is active.

To enable adding custom locations operator has to set True to config option
'show_multiple_locations'. After that any user will be able to add or
remove his image locations, update locations metadata, and finally see
locations of all images even if they were uploaded to local storage. All
this things are not desired if glance v2 has public interface, because it
exposes inner cloud architecture. It leads to the fact that Heat and
Horizon and Nova in some cases and other services that used to set custom
locations in glance v1 won't be able to adopt glance v2. Unfortunately,
removing this behavior in v2 isn't easy, because it requires serious
architecture changes and breaks API. Moreover, many vendors use these
features in their clouds for private glance deployments and they really
won't like if we break anything.

So, I want to hear opinions from Glance community and other involved people.


I agree the current situation is not ideal but I don't think there's a perfect
solution that will let other services magically use the location's
implementation in v2. The API itself is different and it requires a different
call.

With that in mind, I think the right thing to do here is to get rid of that
option[0] and let operators manage this through poilicies. This does not mean
the policies available are perfect.

I'm not an expert on service tokens but I think we said that we could probably
just use service tokens to allow for this feature to be used by other services
instead of keeping it wide open everywhere.

While I don't think the current situation is ideal, I think it's better than
keeping it wide open.

Hope the above helps,
Flavio

[0] https://review.openstack.org/#/c/313936/



Best regards,
Mikhail Fedosin



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

[Flavio Percoco]

On 26/07/16 15:45 +, Fox, Kevin M wrote:

The app catalog has suffered this change too. We had to force v1 in our 
suggested download cli lines to make it work when newer clients defaulted to v2 
and the previously working command line switches suddenly vanished.

As I understand, v2 had a solution to it, but that solution too was deprecated. 
I've heard rumor of a new suggested way of doing it, but I haven't been able to 
find it, so I guess its still cooking.

I'd ask the Glance team to not deprecate v1 until this issue is resolved, as it 
is a very common use case for Glance. I understand the desire to sluff off the 
old and only support a single, new api. But the new api has a big gap in it 
that needs to be fixed first.


To be honest, I don't think this is a blocker for the v1 deprecation. The v1
deprecation has been postponed long enough and I don't think this is a real
blocker. There is indeed a way to allow for services to add locations, which is
enabling it in the config file. I don't mean to oversimplify the work of
changing config files and the cost this has from an operations perspective but
it's not blocking other projects. This option can be set to True in the gate if
necessary.

That being said, I do think we can do better here and that the current state is
not ideal to everyone. I've attempted several times to change this situation but
I believe it was not the right time *shrugs*

Flavio


Thanks,
Kevin

From: Mikhail Fedosin [mfedo...@mirantis.com]
Sent: Tuesday, July 26, 2016 4:32 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

Hello!

As you may know glance v1 is going to be deprecated in Newton cycle. Almost all 
projects support glance v2 at this moment, Nova uses it by default. Only one 
thing that blocks us from complete adoption is a possibility to set custom 
locations to images. In v1 any user can set a location to his image, but in v2 
this functionality is not allowed by default, which prevents v2 adoption in 
services like Horizon or Heat.

It all happens because of differences between v1 and v2 locations. In v1 it is 
pretty easy - user specifies an url and send a request, glance adds this url to 
the image and activates it.
In v2 things are more complicated: v2 supports multiple locations per image, 
which means that when user wants to download image file glance will choose the 
best one from the list of locations. It leads to some inconsistencies: user can 
add or delete locations from his image even if it is active.

To enable adding custom locations operator has to set True to config option 
'show_multiple_locations'. After that any user will be able to add or remove 
his image locations, update locations metadata, and finally see locations of 
all images even if they were uploaded to local storage. All this things are not 
desired if glance v2 has public interface, because it exposes inner cloud 
architecture. It leads to the fact that Heat and Horizon and Nova in some cases 
and other services that used to set custom locations in glance v1 won't be able 
to adopt glance v2. Unfortunately, removing this behavior in v2 isn't easy, 
because it requires serious architecture changes and breaks API. Moreover, many 
vendors use these features in their clouds for private glance deployments and 
they really won't like if we break anything.

So, I want to hear opinions from Glance community and other involved people.

Best regards,
Mikhail Fedosin



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

[Fox, Kevin M]

The app catalog has suffered this change too. We had to force v1 in our 
suggested download cli lines to make it work when newer clients defaulted to v2 
and the previously working command line switches suddenly vanished.

As I understand, v2 had a solution to it, but that solution too was deprecated. 
I've heard rumor of a new suggested way of doing it, but I haven't been able to 
find it, so I guess its still cooking.

I'd ask the Glance team to not deprecate v1 until this issue is resolved, as it 
is a very common use case for Glance. I understand the desire to sluff off the 
old and only support a single, new api. But the new api has a big gap in it 
that needs to be fixed first.

Thanks,
Kevin

From: Mikhail Fedosin [mfedo...@mirantis.com]
Sent: Tuesday, July 26, 2016 4:32 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

Hello!

As you may know glance v1 is going to be deprecated in Newton cycle. Almost all 
projects support glance v2 at this moment, Nova uses it by default. Only one 
thing that blocks us from complete adoption is a possibility to set custom 
locations to images. In v1 any user can set a location to his image, but in v2 
this functionality is not allowed by default, which prevents v2 adoption in 
services like Horizon or Heat.

It all happens because of differences between v1 and v2 locations. In v1 it is 
pretty easy - user specifies an url and send a request, glance adds this url to 
the image and activates it.
In v2 things are more complicated: v2 supports multiple locations per image, 
which means that when user wants to download image file glance will choose the 
best one from the list of locations. It leads to some inconsistencies: user can 
add or delete locations from his image even if it is active.

To enable adding custom locations operator has to set True to config option 
'show_multiple_locations'. After that any user will be able to add or remove 
his image locations, update locations metadata, and finally see locations of 
all images even if they were uploaded to local storage. All this things are not 
desired if glance v2 has public interface, because it exposes inner cloud 
architecture. It leads to the fact that Heat and Horizon and Nova in some cases 
and other services that used to set custom locations in glance v1 won't be able 
to adopt glance v2. Unfortunately, removing this behavior in v2 isn't easy, 
because it requires serious architecture changes and breaks API. Moreover, many 
vendors use these features in their clouds for private glance deployments and 
they really won't like if we break anything.

So, I want to hear opinions from Glance community and other involved people.

Best regards,
Mikhail Fedosin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

[Timur Sufiev]

Can we have a special way (i.e. restricted) of setting Image locations, to
be used by clients like Horizon (where an end-users create images)? It
would be hard to explain different set of Image Sources depending on their
adminness.

Anoher question that I'm asking myself about Horizon is how affordable
would be the loss of 'URL Image Source', since 'copy-from' is missing for
sure already. Are there many valuable use cases for URL source in absence
of 'copy-from' feature?

On Tue, Jul 26, 2016 at 5:33 PM  wrote:

> Hi Mike,
>
> Here's my $0.02 on setting locations directly:
>
> I think there will always be some (probably public cloud) operators who
> are wary of allowing regular users to do this. For this reason,
> and also because the API calls are backend specific (setting location
> isn't supported for a filesystem backend [1]), my guess is it's unlikely
> that
> API calls for setting locations directly will become part of DefCore.
>
> In general I'm not a huge fan of setting the image locations directly
> because:
>
> * Any time you allow setting the location directly you're risking the
> database and the image data getting out of sync. Eg an 'active' image's
> data can be deleted from under it.
> * Some ways of setting the locations directly give you images without
> a known checksum.
>
> In my mind, the feature is probably best restricted (by default at least)
> to power users (admins) and other services -- eg Cinder can do this
> to reduce copying data around in some cases. Existing policies can be
> used to restrict to admin; something like service tokens [1] could be
> used to allow other services (eg Cinder) to do this on behalf of users,
> while preventing users from doing it directly themselves.
>
> To keep things as inter-operable as possible it might be good to ensure
> that services are able to use Glance even if setting locations directly
> is completely disabled.
>
> -Stuart
>
> [1] https://review.openstack.org/#/c/141706
> [2]
> https://specs.openstack.org/openstack/keystone-specs/specs/keystonemiddleware/implemented/service-tokens.html
>
> > Hello!
> >
> > As you may know glance v1 is going to be deprecated in Newton cycle.
> Almost
> > all projects support glance v2 at this moment, Nova uses it by default.
> > Only one thing that blocks us from complete adoption is a possibility to
> > set custom locations to images. In v1 any user can set a location to his
> > image, but in v2 this functionality is not allowed by default, which
> > prevents v2 adoption in services like Horizon or Heat.
> >
> > It all happens because of differences between v1 and v2 locations. In v1
> it
> > is pretty easy - user specifies an url and send a request, glance adds
> this
> > url to the image and activates it.
> > In v2 things are more complicated: v2 supports multiple locations per
> > image, which means that when user wants to download image file glance
> will
> > choose the best one from the list of locations. It leads to some
> > inconsistencies: user can add or delete locations from his image even if
> it
> > is active.
> >
> > To enable adding custom locations operator has to set True to config
> option
> > 'show_multiple_locations'. After that any user will be able to add or
> > remove his image locations, update locations metadata, and finally see
> > locations of all images even if they were uploaded to local storage. All
> > this things are not desired if glance v2 has public interface, because it
> > exposes inner cloud architecture. It leads to the fact that Heat and
> > Horizon and Nova in some cases and other services that used to set custom
> > locations in glance v1 won't be able to adopt glance v2. Unfortunately,
> > removing this behavior in v2 isn't easy, because it requires serious
> > architecture changes and breaks API. Moreover, many vendors use these
> > features in their clouds for private glance deployments and they really
> > won't like if we break anything.
> >
> > So, I want to hear opinions from Glance community and other involved
> > people.
> >
> > Best regards,
> > Mikhail Fedosin
> > -- next part --
> > An HTML attachment was scrubbed...
> > URL:
> > <
> http://lists.openstack.org/pipermail/openstack-dev/attachments/20160726/b97af3b0/attachment-0001.html
> >
> >
> > --
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

[stuart . mclaren]

Hi Mike,

Here's my $0.02 on setting locations directly:

I think there will always be some (probably public cloud) operators who
are wary of allowing regular users to do this. For this reason,
and also because the API calls are backend specific (setting location
isn't supported for a filesystem backend [1]), my guess is it's unlikely that
API calls for setting locations directly will become part of DefCore.

In general I'm not a huge fan of setting the image locations directly 
because:


* Any time you allow setting the location directly you're risking the
database and the image data getting out of sync. Eg an 'active' image's
data can be deleted from under it.
* Some ways of setting the locations directly give you images without
a known checksum.

In my mind, the feature is probably best restricted (by default at least)
to power users (admins) and other services -- eg Cinder can do this
to reduce copying data around in some cases. Existing policies can be
used to restrict to admin; something like service tokens [1] could be
used to allow other services (eg Cinder) to do this on behalf of users,
while preventing users from doing it directly themselves.

To keep things as inter-operable as possible it might be good to ensure
that services are able to use Glance even if setting locations directly
is completely disabled.

-Stuart

[1] https://review.openstack.org/#/c/141706
[2] 
https://specs.openstack.org/openstack/keystone-specs/specs/keystonemiddleware/implemented/service-tokens.html


Hello!

As you may know glance v1 is going to be deprecated in Newton cycle. Almost
all projects support glance v2 at this moment, Nova uses it by default.
Only one thing that blocks us from complete adoption is a possibility to
set custom locations to images. In v1 any user can set a location to his
image, but in v2 this functionality is not allowed by default, which
prevents v2 adoption in services like Horizon or Heat.

It all happens because of differences between v1 and v2 locations. In v1 it
is pretty easy - user specifies an url and send a request, glance adds this
url to the image and activates it.
In v2 things are more complicated: v2 supports multiple locations per
image, which means that when user wants to download image file glance will
choose the best one from the list of locations. It leads to some
inconsistencies: user can add or delete locations from his image even if it
is active.

To enable adding custom locations operator has to set True to config option
'show_multiple_locations'. After that any user will be able to add or
remove his image locations, update locations metadata, and finally see
locations of all images even if they were uploaded to local storage. All
this things are not desired if glance v2 has public interface, because it
exposes inner cloud architecture. It leads to the fact that Heat and
Horizon and Nova in some cases and other services that used to set custom
locations in glance v1 won't be able to adopt glance v2. Unfortunately,
removing this behavior in v2 isn't easy, because it requires serious
architecture changes and breaks API. Moreover, many vendors use these
features in their clouds for private glance deployments and they really
won't like if we break anything.

So, I want to hear opinions from Glance community and other involved 
people.


Best regards,
Mikhail Fedosin
-- next part --
An HTML attachment was scrubbed...
URL: 



--



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Glance][Heat][Horizon] Glance v2 and custom locations

[Mikhail Fedosin]

Hello!

As you may know glance v1 is going to be deprecated in Newton cycle. Almost
all projects support glance v2 at this moment, Nova uses it by default.
Only one thing that blocks us from complete adoption is a possibility to
set custom locations to images. In v1 any user can set a location to his
image, but in v2 this functionality is not allowed by default, which
prevents v2 adoption in services like Horizon or Heat.

It all happens because of differences between v1 and v2 locations. In v1 it
is pretty easy - user specifies an url and send a request, glance adds this
url to the image and activates it.
In v2 things are more complicated: v2 supports multiple locations per
image, which means that when user wants to download image file glance will
choose the best one from the list of locations. It leads to some
inconsistencies: user can add or delete locations from his image even if it
is active.

To enable adding custom locations operator has to set True to config option
'show_multiple_locations'. After that any user will be able to add or
remove his image locations, update locations metadata, and finally see
locations of all images even if they were uploaded to local storage. All
this things are not desired if glance v2 has public interface, because it
exposes inner cloud architecture. It leads to the fact that Heat and
Horizon and Nova in some cases and other services that used to set custom
locations in glance v1 won't be able to adopt glance v2. Unfortunately,
removing this behavior in v2 isn't easy, because it requires serious
architecture changes and breaks API. Moreover, many vendors use these
features in their clouds for private glance deployments and they really
won't like if we break anything.

So, I want to hear opinions from Glance community and other involved people.

Best regards,
Mikhail Fedosin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev