Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Clint Byrum]

Excerpts from Ian Cordasco's message of 2017-02-13 15:15:12 -0500:
> -Original Message-
> From: Clint Byrum <cl...@fewbar.com>
> Reply: OpenStack Development Mailing List (not for usage questions)
> <openstack-dev@lists.openstack.org>
> Date: February 13, 2017 at 13:41:00
> To: openstack-dev <openstack-dev@lists.openstack.org>
> Subject:  Re: [openstack-dev] [tc][glance][glare][all]
> glance/glare/artifacts/images at the PTG
> 
> > Excerpts from Mikhail Fedosin's message of 2017-02-13 18:23:19 +0300:
> > > Hello!
> > >
> > >
> > > Let me quickly describe my vision of the problem. I asked this question to
> > > Brian last Friday, because it is evident that the projects have the
> > > intersection in functionality. For this reason, I proposed to bring Glare
> > > back and develop it as a new generation of Glance service. Perhaps such a
> > > solution would be more correct from my point of view.
> > >
> > > Moving away from Glance, let me remind you why we created Glare service.
> > >
> > > Almost every project works with some binary data and must store it
> > > somewhere, and almost always storage itself is not the part of the
> > > project's mission. This issue has often been neglected. For this reason
> > > there is no single recommended method for storing of binary data, which
> > > would have a unified public api and hide all the things of the internal
> > > storage infrastructure.
> > >
> >
> > We have an awesome service for storing binary data in a hierarchical
> > format in Swift. But it's so generic, it can't really just be the image
> > service. But something like Glare is just a way to scope it down and
> > give us a way to ask for "just the images" or "just the heat templates",
> > which I think is a natural thing for cloud users to want.
> >
> > > These questions were answered by Glare. First of all, the service allows 
> > > to
> > > use different storages for various types of artifacts - an operator can
> > > assign the storage of large files, such as virtual machine images, to
> > > Swift, and for relatively small ones, such as Heat templates, use a mysql
> > > database.
> > >
> >
> > Meh. Swift isn't exactly slow, or cumbersome for small files.
> >
> > > Then, we have to admit that data tends to change, so we added a versioning
> > > of artifacts and dependencies between them, that the user was convenient 
> > > to
> > > take the data of required version.
> > >
> >
> > Any attempt at versioning that is not git, will frustrate any git user.
> > This cat's already out of the bag, but I'd suggest adding git repositories
> > as a blob container type and finding a way to allow git to push/pull
> > to/from swift. That would be an amazing feature _for swift_ anyway
> > (maybe it already exists?) but it would allow Glare to piggy back on all
> > of the collective versioning capabilities in Git rather than having to
> > chase git.
> 
> So the versioning that's present will frustrate everyone. The
> reasoning for it is that the original Glare developers found a hack
> online to convert the version string into something that a database
> can sort (by turning it into one giant integer basically). (I'm
> certain that's not the only reason, but when challenged with several
> other options they said they couldn't find anyone who had already
> found a way to make it sortable on the version.)
> 
> That aside, I'm not sure anyone wants git (even git-lfs) managing 50GB
> images for them.
> 

Sounds like this was a high level solution that I don't fully understand,
so I'll stop bikeshedding it. But generally I'd say for most OpenStack
services you want to stay low-level whenever possible.

And of course the actual binaries would not be in git. But the metadata
about them would be, which would allow things like bisection,
annotation, etc.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Thomas Herve]

On Mon, Feb 13, 2017 at 8:39 PM, Clint Byrum  wrote:

[snip]

> Any attempt at versioning that is not git, will frustrate any git user.
> This cat's already out of the bag, but I'd suggest adding git repositories
> as a blob container type and finding a way to allow git to push/pull
> to/from swift. That would be an amazing feature _for swift_ anyway
> (maybe it already exists?) but it would allow Glare to piggy back on all
> of the collective versioning capabilities in Git rather than having to
> chase git.

That has been done:
https://blogs.rdoproject.org/6642/openstack-swift-as-backend-for-git-part-1

I don't know if it's maintained nowadays, but I suspect it could be
picked up if interest raises.

-- 
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Ian Cordasco]

-Original Message-
From: Clint Byrum <cl...@fewbar.com>
Reply: OpenStack Development Mailing List (not for usage questions)
<openstack-dev@lists.openstack.org>
Date: February 13, 2017 at 13:41:00
To: openstack-dev <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [tc][glance][glare][all]
glance/glare/artifacts/images at the PTG

> Excerpts from Mikhail Fedosin's message of 2017-02-13 18:23:19 +0300:
> > Hello!
> >
> >
> > Let me quickly describe my vision of the problem. I asked this question to
> > Brian last Friday, because it is evident that the projects have the
> > intersection in functionality. For this reason, I proposed to bring Glare
> > back and develop it as a new generation of Glance service. Perhaps such a
> > solution would be more correct from my point of view.
> >
> > Moving away from Glance, let me remind you why we created Glare service.
> >
> > Almost every project works with some binary data and must store it
> > somewhere, and almost always storage itself is not the part of the
> > project's mission. This issue has often been neglected. For this reason
> > there is no single recommended method for storing of binary data, which
> > would have a unified public api and hide all the things of the internal
> > storage infrastructure.
> >
>
> We have an awesome service for storing binary data in a hierarchical
> format in Swift. But it's so generic, it can't really just be the image
> service. But something like Glare is just a way to scope it down and
> give us a way to ask for "just the images" or "just the heat templates",
> which I think is a natural thing for cloud users to want.
>
> > These questions were answered by Glare. First of all, the service allows to
> > use different storages for various types of artifacts - an operator can
> > assign the storage of large files, such as virtual machine images, to
> > Swift, and for relatively small ones, such as Heat templates, use a mysql
> > database.
> >
>
> Meh. Swift isn't exactly slow, or cumbersome for small files.
>
> > Then, we have to admit that data tends to change, so we added a versioning
> > of artifacts and dependencies between them, that the user was convenient to
> > take the data of required version.
> >
>
> Any attempt at versioning that is not git, will frustrate any git user.
> This cat's already out of the bag, but I'd suggest adding git repositories
> as a blob container type and finding a way to allow git to push/pull
> to/from swift. That would be an amazing feature _for swift_ anyway
> (maybe it already exists?) but it would allow Glare to piggy back on all
> of the collective versioning capabilities in Git rather than having to
> chase git.

So the versioning that's present will frustrate everyone. The
reasoning for it is that the original Glare developers found a hack
online to convert the version string into something that a database
can sort (by turning it into one giant integer basically). (I'm
certain that's not the only reason, but when challenged with several
other options they said they couldn't find anyone who had already
found a way to make it sortable on the version.)

That aside, I'm not sure anyone wants git (even git-lfs) managing 50GB
images for them.

> > Often a "binary data" refers to more than one specific object, but a whole
> > lot of files. Therefore, we have implemented the ability to create
> > arbitrary nested folders per one artifact and store multiple files there.
> > And for sure users can receive any file with a single API request.
> >
>
> See above: IMO, use git for this as well and just teach Glare to
> understand git repos rather than having to implement folders in databases.
>
> > For validation and conversion of uploaded data Glare introduces the concept
> > of hooks for the operation. Thus the operator can extend the basic
> > functionality of the system and add integration with third-party systems
> > for each artifact type. For example, for Nokia we implemented integration
> > with custom TOSCA validator.
> >
>
> At first this set off my interop alarm, but it was a false alarm as long
> as this is always limited to 3rd party systems. What worries me is when
> somebody adds one of these for something already in OpenStack, and now
> suddenly a perfectly interoperable app only works right on that one
> special cloud.
>
> > This is just a small overview of the key features of the service. For sure,
> > at the moment Glare is able to do all that Glance can do (except maybe a
> > sharing of artifacts), on the other hand we have added a number of new
> > features, that w

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Clint Byrum]

Excerpts from Mikhail Fedosin's message of 2017-02-13 18:23:19 +0300:
> Hello!
> 
> 
> Let me quickly describe my vision of the problem. I asked this question to
> Brian last Friday, because it is evident that the projects have the
> intersection in functionality. For this reason, I proposed to bring Glare
> back and develop it as a new generation of Glance service. Perhaps such a
> solution would be more correct from my point of view.
> 
> Moving away from Glance, let me remind you why we created Glare service.
> 
> Almost every project works with some binary data and must store it
> somewhere, and almost always storage itself is not the part of the
> project's mission. This issue has often been neglected. For this reason
> there is no single recommended method for storing of binary data, which
> would have a unified public api and hide all the things of the internal
> storage infrastructure.
> 

We have an awesome service for storing binary data in a hierarchical
format in Swift. But it's so generic, it can't really just be the image
service. But something like Glare is just a way to scope it down and
give us a way to ask for "just the images" or "just the heat templates",
which I think is a natural thing for cloud users to want.

> These questions were answered by Glare. First of all, the service allows to
> use different storages for various types of artifacts - an operator can
> assign the storage of large files, such as virtual machine images, to
> Swift, and for relatively small ones, such as Heat templates, use a mysql
> database.
> 

Meh. Swift isn't exactly slow, or cumbersome for small files.

> Then, we have to admit that data tends to change, so we added a versioning
> of artifacts and dependencies between them, that the user was convenient to
> take the data of required version.
> 

Any attempt at versioning that is not git, will frustrate any git user.
This cat's already out of the bag, but I'd suggest adding git repositories
as a blob container type and finding a way to allow git to push/pull
to/from swift. That would be an amazing feature _for swift_ anyway
(maybe it already exists?) but it would allow Glare to piggy back on all
of the collective versioning capabilities in Git rather than having to
chase git.

> Often a "binary data" refers to more than one specific object, but a whole
> lot of files. Therefore, we have implemented the ability to create
> arbitrary nested folders per one artifact and store multiple files there.
> And for sure users can receive any file with a single API request.
> 

See above: IMO, use git for this as well and just teach Glare to
understand git repos rather than having to implement folders in databases.

> For validation and conversion of uploaded data Glare introduces the concept
> of hooks for the operation. Thus the operator can extend the basic
> functionality of the system and add integration with third-party systems
> for each artifact type. For example, for Nokia we implemented integration
> with custom TOSCA validator.
> 

At first this set off my interop alarm, but it was a false alarm as long
as this is always limited to 3rd party systems. What worries me is when
somebody adds one of these for something already in OpenStack, and now
suddenly a perfectly interoperable app only works right on that one
special cloud.

> This is just a small overview of the key features of the service. For sure,
> at the moment Glare is able to do all that Glance can do (except maybe a
> sharing of artifacts), on the other hand we have added a number of new
> features, that were requested by cloud operators for a long time.
> 
> Fyi, now we in Nokia are preparing additional API, which corresponds to the
> ETSI VNF Packaging Specification [1]. So support of Image v2 API is not an
> impossible task, and we may implement it as an alternative way of
> interaction with "Images" artifact type. In this case Nova and other
> services using Glance are absolutely indifferent to what service provides
> Image API.
> 

If you can make it 100% API compatible with Image v2, you'll go a long way
to helping users smoothly switch over.

> All tasks related to documentation and packaging are solvable. We’re
> working on them together with Nokia, so I can assure you that the documents
> and packages will be available this spring. The same story is for Ansible
> and Puppet.
> 
> Now back again to our question. What I'd like is that Glare will receive
> due recognition. Doing a project on the outskirts of OpenStack is not I
> really want to. Therefore, it would be nice to develop Glare as a natural
> evolution of Glance, associated with the requirements of operators and the
> market in general. For Glance team is a good chance to try something new
> and interesting, and of course gain new experience.
> 

I support you in your attempt to have a natural evolution. I think
it's going to be harder and harder the longer you're developing Glare's
features without pushing for a transition to 

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Monty Taylor]

On 02/13/2017 07:47 AM, Ian Cordasco wrote:
> -Original Message-
> From: Clint Byrum <cl...@fewbar.com>
> Reply: OpenStack Development Mailing List (not for usage questions)
> <openstack-dev@lists.openstack.org>
> Date: February 12, 2017 at 20:09:04
> To: openstack-dev <openstack-dev@lists.openstack.org>
> Subject:  Re: [openstack-dev] [tc][glance][glare][all]
> glance/glare/artifacts/images at the PTG
> 
>> Excerpts from Brian Rosmaita's message of 2017-02-10 12:39:11 -0500:
>>> I want to give all interested parties a heads up that I have scheduled a
>>> session in the Macon room from 9:30-10:30 a.m. on Thursday morning
>>> (February 23).
>>>
>>> Here's what we need to discuss. This is from my perspective as Glance
>>> PTL, so it's going to be Glance-centric. This is a quick narrative
>>> description; please go to the session etherpad [0] to turn this into a
>>> specific set of discussion items.
>>>
>>> Glance is the OpenStack image cataloging and delivery service. A few
>>> cycles ago (Juno?), someone noticed that maybe Glance could be
>>> generalized so that instead of storing image metadata and image data,
>>> Glance could store arbitrary digital "stuff" along with metadata
>>> describing the "stuff". Some people (like me) thought that this was an
>>> obvious direction for Glance to take, but others (maybe wiser, cooler
>>> heads) thought that Glance needed to focus on image cataloging and
>>> delivery and make sure it did a good job at that. Anyway, the Glance
>>> mission statement was changed to include artifacts, but the Glance
>>> community never embraced them 100%, and in Newton, Glare split off as
>>> its own project (which made sense to me, there was too much unclarity in
>>> Glance about how Glare fit in, and we were holding back development, and
>>> besides we needed to focus on images), and the Glance mission statement
>>> was re-amended specifically to exclude artifacts and focus on images and
>>> metadata definitions.
>>>
>>> OK, so the current situation is:
>>> - Glance "does" image cataloging and delivery and metadefs, and that's
>>> all it does.
>>> - Glare is an artifacts service (cataloging and delivery) that can also
>>> handle images.
>>>
>>> You can see that there's quite a bit of overlap. I gave you the history
>>> earlier because we did try to work as a single project, but it did not
>>> work out.
>>>
>>> So, now we are in 2017. The OpenStack development situation has been
>>> fragile since the second half of 2016, with several big OpenStack
>>> sponsors pulling way back on the amount of development resources being
>>> contributed to the community. This has left Glare in the position where
>>> it cannot qualify as a Bit Tent project, even though there is interest
>>> in artifacts.
>>>
>>> Mike Fedosin, the PTL for Glare, has asked me about Glare becoming part
>>> of the Glance project again. I will be completely honest, I am inclined
>>> to say "no". I have enough problems just getting Glance stuff done (for
>>> example, image import missed Ocata). But in addition to doing what's
>>> right for Glance, I want to do what's right for OpenStack. And I look
>>> at the overlap and think ...
>>>
>>> Well, what I think is that I don't want to go through the Juno-Newton
>>> cycles of argument again. And we have to do what is right for our users.
>>>
>>> The point of this session is to discuss:
>>> - What does the Glance community see as the future of Glance?
>>> - What does the wider OpenStack community (TC) see as the future of Glance?
>>> - Maybe, more importantly, what does the wider community see as the
>>> obligations of Glance?
>>> - Does Glare fit into this vision?
>>> - What kind of community support is there for Glare?
>>>
>>> My reading of Glance history is that while some people were on board
>>> with artifacts as the future of Glance, there was not a sufficient
>>> critical mass of the Glance community that endorsed this direction and
>>> that's why things unravelled in Newton. I don't want to see that happen
>>> again. Further, I don't think the Glance community got the word out to
>>> the broader OpenStack community about the artifacts project, and we got
>>> a lot of pushback along the lines of "WTF? Glance needs to do images"
>>> variety. And probably rightly so -

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Mikhail Fedosin]

Okay, it seems I did not express myself very well :) But then there is a
rhetorical question: if it's officially recommended to use the service with
the name that starts with "S", why does Nova use a service with the name
beginning with "G" to store its images?

As you may know Glare is a proxy to Swift, Ceph and other possible cloud
storages, and it provides an abstraction (artifact) upon them + several
additional features (like custom data validation and conversion) that Swift
doesn't and shouldn't have. And for sure I was talking about secure and
customizable catalog of binary data with its metadata, and not the concrete
storage implementation. Sorry again for this confusion :)

Best,
Mike

On Mon, Feb 13, 2017 at 8:04 PM, Ian Cordasco <sigmaviru...@gmail.com>
wrote:

> -Original Message-
> From: Jeremy Stanley <fu...@yuggoth.org>
> Reply: OpenStack Development Mailing List (not for usage questions)
> <openstack-dev@lists.openstack.org>
> Date: February 13, 2017 at 10:14:24
> To: OpenStack Development Mailing List (not for usage questions)
> <openstack-dev@lists.openstack.org>
> Subject:  Re: [openstack-dev] [tc][glance][glare][all]
> glance/glare/artifacts/images at the PTG
>
> > On 2017-02-13 18:23:19 +0300 (+0300), Mikhail Fedosin wrote:
> > [...]
> > > Almost every project works with some binary data and must store it
> > > somewhere, and almost always storage itself is not the part of the
> > > project's mission. This issue has often been neglected. For this reason
> > > there is no single recommended method for storing of binary data, which
> > > would have a unified public api and hide all the things of the internal
> > > storage infrastructure.
> > [...]
> >
> > If you'll forgive the sarcasm, it sounds like you're proposing that
> > OpenStack components should be able to rely on the existence of a
> > standard service suitable for generalized storage and retrieval of
> > arbitrary blobs of data through an API. Our trademark
> > interoperability requirements may even guarantee the presence of one
> > already in any compliant deployment; I'll have to check... ;)
>
> Well it's a storage service, so I hope the name doesn't start with "S". ;D
>
> --
> Ian Cordasco
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Ian Cordasco]

-Original Message-
From: Jeremy Stanley <fu...@yuggoth.org>
Reply: OpenStack Development Mailing List (not for usage questions)
<openstack-dev@lists.openstack.org>
Date: February 13, 2017 at 10:14:24
To: OpenStack Development Mailing List (not for usage questions)
<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [tc][glance][glare][all]
glance/glare/artifacts/images at the PTG

> On 2017-02-13 18:23:19 +0300 (+0300), Mikhail Fedosin wrote:
> [...]
> > Almost every project works with some binary data and must store it
> > somewhere, and almost always storage itself is not the part of the
> > project's mission. This issue has often been neglected. For this reason
> > there is no single recommended method for storing of binary data, which
> > would have a unified public api and hide all the things of the internal
> > storage infrastructure.
> [...]
>
> If you'll forgive the sarcasm, it sounds like you're proposing that
> OpenStack components should be able to rely on the existence of a
> standard service suitable for generalized storage and retrieval of
> arbitrary blobs of data through an API. Our trademark
> interoperability requirements may even guarantee the presence of one
> already in any compliant deployment; I'll have to check... ;)

Well it's a storage service, so I hope the name doesn't start with "S". ;D

--
Ian Cordasco

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Jeremy Stanley]

On 2017-02-13 18:23:19 +0300 (+0300), Mikhail Fedosin wrote:
[...]
> Almost every project works with some binary data and must store it
> somewhere, and almost always storage itself is not the part of the
> project's mission. This issue has often been neglected. For this reason
> there is no single recommended method for storing of binary data, which
> would have a unified public api and hide all the things of the internal
> storage infrastructure.
[...]

If you'll forgive the sarcasm, it sounds like you're proposing that
OpenStack components should be able to rely on the existence of a
standard service suitable for generalized storage and retrieval of
arbitrary blobs of data through an API. Our trademark
interoperability requirements may even guarantee the presence of one
already in any compliant deployment; I'll have to check... ;)
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Mikhail Fedosin]

Hello!


Let me quickly describe my vision of the problem. I asked this question to
Brian last Friday, because it is evident that the projects have the
intersection in functionality. For this reason, I proposed to bring Glare
back and develop it as a new generation of Glance service. Perhaps such a
solution would be more correct from my point of view.

Moving away from Glance, let me remind you why we created Glare service.

Almost every project works with some binary data and must store it
somewhere, and almost always storage itself is not the part of the
project's mission. This issue has often been neglected. For this reason
there is no single recommended method for storing of binary data, which
would have a unified public api and hide all the things of the internal
storage infrastructure.

These questions were answered by Glare. First of all, the service allows to
use different storages for various types of artifacts - an operator can
assign the storage of large files, such as virtual machine images, to
Swift, and for relatively small ones, such as Heat templates, use a mysql
database.

Then, we have to admit that data tends to change, so we added a versioning
of artifacts and dependencies between them, that the user was convenient to
take the data of required version.

Often a "binary data" refers to more than one specific object, but a whole
lot of files. Therefore, we have implemented the ability to create
arbitrary nested folders per one artifact and store multiple files there.
And for sure users can receive any file with a single API request.

For validation and conversion of uploaded data Glare introduces the concept
of hooks for the operation. Thus the operator can extend the basic
functionality of the system and add integration with third-party systems
for each artifact type. For example, for Nokia we implemented integration
with custom TOSCA validator.

This is just a small overview of the key features of the service. For sure,
at the moment Glare is able to do all that Glance can do (except maybe a
sharing of artifacts), on the other hand we have added a number of new
features, that were requested by cloud operators for a long time.

Fyi, now we in Nokia are preparing additional API, which corresponds to the
ETSI VNF Packaging Specification [1]. So support of Image v2 API is not an
impossible task, and we may implement it as an alternative way of
interaction with "Images" artifact type. In this case Nova and other
services using Glance are absolutely indifferent to what service provides
Image API.

All tasks related to documentation and packaging are solvable. We’re
working on them together with Nokia, so I can assure you that the documents
and packages will be available this spring. The same story is for Ansible
and Puppet.

Now back again to our question. What I'd like is that Glare will receive
due recognition. Doing a project on the outskirts of OpenStack is not I
really want to. Therefore, it would be nice to develop Glare as a natural
evolution of Glance, associated with the requirements of operators and the
market in general. For Glance team is a good chance to try something new
and interesting, and of course gain new experience.

I am ready to discuss all these questions in this thread, and at PTG, as
long as necessary.

Best,

Mike

[1]
http://www.etsi.org/deliver/etsi_gs/NFV-IFA/001_099/011/02.01.01_60/gs_NFV-IFA011v020101p.pdf

On Fri, Feb 10, 2017 at 8:39 PM, Brian Rosmaita 
wrote:

> I want to give all interested parties a heads up that I have scheduled a
> session in the Macon room from 9:30-10:30 a.m. on Thursday morning
> (February 23).
>
> Here's what we need to discuss.  This is from my perspective as Glance
> PTL, so it's going to be Glance-centric.  This is a quick narrative
> description; please go to the session etherpad [0] to turn this into a
> specific set of discussion items.
>
> Glance is the OpenStack image cataloging and delivery service.  A few
> cycles ago (Juno?), someone noticed that maybe Glance could be
> generalized so that instead of storing image metadata and image data,
> Glance could store arbitrary digital "stuff" along with metadata
> describing the "stuff".  Some people (like me) thought that this was an
> obvious direction for Glance to take, but others (maybe wiser, cooler
> heads) thought that Glance needed to focus on image cataloging and
> delivery and make sure it did a good job at that.  Anyway, the Glance
> mission statement was changed to include artifacts, but the Glance
> community never embraced them 100%, and in Newton, Glare split off as
> its own project (which made sense to me, there was too much unclarity in
> Glance about how Glare fit in, and we were holding back development, and
> besides we needed to focus on images), and the Glance mission statement
> was re-amended specifically to exclude artifacts and focus on images and
> metadata definitions.
>
> OK, so the current situation is:
> - Glance "does" 

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Flavio Percoco]

On 13/02/17 14:37 +0100, Thierry Carrez wrote:

Brian Rosmaita wrote:

I want to give all interested parties a heads up that I have scheduled a
session in the Macon room from 9:30-10:30 a.m. on Thursday morning
(February 23).
[...]


Thanks for setting this meeting up. I'll be there!


Ditto! I'm looking forward to this discussion and I'll hold any comments back
until it happens next week.

Cheers,
Flavio

--
@flaper87
Flavio Percoco


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Ian Cordasco]

-Original Message-
From: Clint Byrum <cl...@fewbar.com>
Reply: OpenStack Development Mailing List (not for usage questions)
<openstack-dev@lists.openstack.org>
Date: February 12, 2017 at 20:09:04
To: openstack-dev <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [tc][glance][glare][all]
glance/glare/artifacts/images at the PTG

> Excerpts from Brian Rosmaita's message of 2017-02-10 12:39:11 -0500:
> > I want to give all interested parties a heads up that I have scheduled a
> > session in the Macon room from 9:30-10:30 a.m. on Thursday morning
> > (February 23).
> >
> > Here's what we need to discuss. This is from my perspective as Glance
> > PTL, so it's going to be Glance-centric. This is a quick narrative
> > description; please go to the session etherpad [0] to turn this into a
> > specific set of discussion items.
> >
> > Glance is the OpenStack image cataloging and delivery service. A few
> > cycles ago (Juno?), someone noticed that maybe Glance could be
> > generalized so that instead of storing image metadata and image data,
> > Glance could store arbitrary digital "stuff" along with metadata
> > describing the "stuff". Some people (like me) thought that this was an
> > obvious direction for Glance to take, but others (maybe wiser, cooler
> > heads) thought that Glance needed to focus on image cataloging and
> > delivery and make sure it did a good job at that. Anyway, the Glance
> > mission statement was changed to include artifacts, but the Glance
> > community never embraced them 100%, and in Newton, Glare split off as
> > its own project (which made sense to me, there was too much unclarity in
> > Glance about how Glare fit in, and we were holding back development, and
> > besides we needed to focus on images), and the Glance mission statement
> > was re-amended specifically to exclude artifacts and focus on images and
> > metadata definitions.
> >
> > OK, so the current situation is:
> > - Glance "does" image cataloging and delivery and metadefs, and that's
> > all it does.
> > - Glare is an artifacts service (cataloging and delivery) that can also
> > handle images.
> >
> > You can see that there's quite a bit of overlap. I gave you the history
> > earlier because we did try to work as a single project, but it did not
> > work out.
> >
> > So, now we are in 2017. The OpenStack development situation has been
> > fragile since the second half of 2016, with several big OpenStack
> > sponsors pulling way back on the amount of development resources being
> > contributed to the community. This has left Glare in the position where
> > it cannot qualify as a Bit Tent project, even though there is interest
> > in artifacts.
> >
> > Mike Fedosin, the PTL for Glare, has asked me about Glare becoming part
> > of the Glance project again. I will be completely honest, I am inclined
> > to say "no". I have enough problems just getting Glance stuff done (for
> > example, image import missed Ocata). But in addition to doing what's
> > right for Glance, I want to do what's right for OpenStack. And I look
> > at the overlap and think ...
> >
> > Well, what I think is that I don't want to go through the Juno-Newton
> > cycles of argument again. And we have to do what is right for our users.
> >
> > The point of this session is to discuss:
> > - What does the Glance community see as the future of Glance?
> > - What does the wider OpenStack community (TC) see as the future of Glance?
> > - Maybe, more importantly, what does the wider community see as the
> > obligations of Glance?
> > - Does Glare fit into this vision?
> > - What kind of community support is there for Glare?
> >
> > My reading of Glance history is that while some people were on board
> > with artifacts as the future of Glance, there was not a sufficient
> > critical mass of the Glance community that endorsed this direction and
> > that's why things unravelled in Newton. I don't want to see that happen
> > again. Further, I don't think the Glance community got the word out to
> > the broader OpenStack community about the artifacts project, and we got
> > a lot of pushback along the lines of "WTF? Glance needs to do images"
> > variety. And probably rightly so -- Glance needs to do images. My
> > point is that I don't want Glance to take Glare back unless it fits in
> > with what the community sees as the appropriate direction for Glance.
> > And I certainly don't want to take it back if the entire Glance
> > communit

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Thierry Carrez]

Brian Rosmaita wrote:
> I want to give all interested parties a heads up that I have scheduled a
> session in the Macon room from 9:30-10:30 a.m. on Thursday morning
> (February 23).
> [...]

Thanks for setting this meeting up. I'll be there!

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Clint Byrum]

Excerpts from Brian Rosmaita's message of 2017-02-10 12:39:11 -0500:
> I want to give all interested parties a heads up that I have scheduled a
> session in the Macon room from 9:30-10:30 a.m. on Thursday morning
> (February 23).
> 
> Here's what we need to discuss.  This is from my perspective as Glance
> PTL, so it's going to be Glance-centric.  This is a quick narrative
> description; please go to the session etherpad [0] to turn this into a
> specific set of discussion items.
> 
> Glance is the OpenStack image cataloging and delivery service.  A few
> cycles ago (Juno?), someone noticed that maybe Glance could be
> generalized so that instead of storing image metadata and image data,
> Glance could store arbitrary digital "stuff" along with metadata
> describing the "stuff".  Some people (like me) thought that this was an
> obvious direction for Glance to take, but others (maybe wiser, cooler
> heads) thought that Glance needed to focus on image cataloging and
> delivery and make sure it did a good job at that.  Anyway, the Glance
> mission statement was changed to include artifacts, but the Glance
> community never embraced them 100%, and in Newton, Glare split off as
> its own project (which made sense to me, there was too much unclarity in
> Glance about how Glare fit in, and we were holding back development, and
> besides we needed to focus on images), and the Glance mission statement
> was re-amended specifically to exclude artifacts and focus on images and
> metadata definitions.
> 
> OK, so the current situation is:
> - Glance "does" image cataloging and delivery and metadefs, and that's
> all it does.
> - Glare is an artifacts service (cataloging and delivery) that can also
> handle images.
> 
> You can see that there's quite a bit of overlap.  I gave you the history
> earlier because we did try to work as a single project, but it did not
> work out.
> 
> So, now we are in 2017.  The OpenStack development situation has been
> fragile since the second half of 2016, with several big OpenStack
> sponsors pulling way back on the amount of development resources being
> contributed to the community.  This has left Glare in the position where
> it cannot qualify as a Bit Tent project, even though there is interest
> in artifacts.
> 
> Mike Fedosin, the PTL for Glare, has asked me about Glare becoming part
> of the Glance project again.  I will be completely honest, I am inclined
> to say "no".  I have enough problems just getting Glance stuff done (for
> example, image import missed Ocata).  But in addition to doing what's
> right for Glance, I want to do what's right for OpenStack.  And I look
> at the overlap and think ...
> 
> Well, what I think is that I don't want to go through the Juno-Newton
> cycles of argument again.  And we have to do what is right for our users.
> 
> The point of this session is to discuss:
> - What does the Glance community see as the future of Glance?
> - What does the wider OpenStack community (TC) see as the future of Glance?
> - Maybe, more importantly, what does the wider community see as the
> obligations of Glance?
> - Does Glare fit into this vision?
> - What kind of community support is there for Glare?
> 
> My reading of Glance history is that while some people were on board
> with artifacts as the future of Glance, there was not a sufficient
> critical mass of the Glance community that endorsed this direction and
> that's why things unravelled in Newton.  I don't want to see that happen
> again.  Further, I don't think the Glance community got the word out to
> the broader OpenStack community about the artifacts project, and we got
> a lot of pushback along the lines of "WTF? Glance needs to do images"
> variety.  And probably rightly so -- Glance needs to do images.  My
> point is that I don't want Glance to take Glare back unless it fits in
> with what the community sees as the appropriate direction for Glance.
> And I certainly don't want to take it back if the entire Glance
> community is not on board.
> 
> Anyway, that's what we're going to discuss.  I've booked one of the
> fishbowl rooms so we can get input from people beyond just the Glance
> and Glare projects.
> 

Does anybody else feel like this is deja vu of Neutron's inception?

While I understand sometimes there are just incompatibilities in groups,
I think we should probably try again. Unfortunately, it sounds like
Glare already did the Neutron thing of starting from scratch and sort
of overlapping in functionality, instead of the Cinder thing where you
forklift the code from one project into a new one and pay close attention
to the peaceful transition of users. But we've been here before and we
can do better. :)

Given that, I'm hoping some folks who helped with the Neutron transition
can attend this session and share what Neutron learned so that if Glare
does take over for images, we don't end up in a multi-cycle quagmire
where Glance is frozen and Glare is moving too fast for Glance devs to

Re: [openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Tim Bell]

Although there has not been much discussion on this point on the mailing list, 
I feel we do need to find the right level of granularity for ‘mainstream’ 
projects:

For CERN, we look for the following before offering a project to our end users:

- Distro packaging (in our case RPMs through RDO)
- Puppet modules
- Openstack client support (which brings Kerberos/X.509 authentication)
- Install, admin and user docs
- Project diversity for long term sustainability

We have many use cases of ‘resellers’ where one project provides a deliverable 
for others to consume, some degree of community image sharing is arriving and 
these are the same problems to face for artefacts and application catalogues 
(such as Heat and Magnum).

For me, which project provides this for images and/or artefacts is a choice for 
the technical community but consistent semantics would be greatly appreciated 
for those discussions with our end users such as “I need a Heat template for X 
but this needs community image Y and the visibility rules means that one needs 
to be shared in advance, the other I need to subscribe to” are difficult 
discussion which discourages uptake.

A cloud user should be able to click on community offered ‘R-as-a-Service’ in 
the application catalog GUI, and that’s all.

Tim

On 10.02.17, 18:39, "Brian Rosmaita"  wrote:

I want to give all interested parties a heads up that I have scheduled a
session in the Macon room from 9:30-10:30 a.m. on Thursday morning
(February 23).

Here's what we need to discuss.  This is from my perspective as Glance
PTL, so it's going to be Glance-centric.  This is a quick narrative
description; please go to the session etherpad [0] to turn this into a
specific set of discussion items.

Glance is the OpenStack image cataloging and delivery service.  A few
cycles ago (Juno?), someone noticed that maybe Glance could be
generalized so that instead of storing image metadata and image data,
Glance could store arbitrary digital "stuff" along with metadata
describing the "stuff".  Some people (like me) thought that this was an
obvious direction for Glance to take, but others (maybe wiser, cooler
heads) thought that Glance needed to focus on image cataloging and
delivery and make sure it did a good job at that.  Anyway, the Glance
mission statement was changed to include artifacts, but the Glance
community never embraced them 100%, and in Newton, Glare split off as
its own project (which made sense to me, there was too much unclarity in
Glance about how Glare fit in, and we were holding back development, and
besides we needed to focus on images), and the Glance mission statement
was re-amended specifically to exclude artifacts and focus on images and
metadata definitions.

OK, so the current situation is:
- Glance "does" image cataloging and delivery and metadefs, and that's
all it does.
- Glare is an artifacts service (cataloging and delivery) that can also
handle images.

You can see that there's quite a bit of overlap.  I gave you the history
earlier because we did try to work as a single project, but it did not
work out.

So, now we are in 2017.  The OpenStack development situation has been
fragile since the second half of 2016, with several big OpenStack
sponsors pulling way back on the amount of development resources being
contributed to the community.  This has left Glare in the position where
it cannot qualify as a Bit Tent project, even though there is interest
in artifacts.

Mike Fedosin, the PTL for Glare, has asked me about Glare becoming part
of the Glance project again.  I will be completely honest, I am inclined
to say "no".  I have enough problems just getting Glance stuff done (for
example, image import missed Ocata).  But in addition to doing what's
right for Glance, I want to do what's right for OpenStack.  And I look
at the overlap and think ...

Well, what I think is that I don't want to go through the Juno-Newton
cycles of argument again.  And we have to do what is right for our users.

The point of this session is to discuss:
- What does the Glance community see as the future of Glance?
- What does the wider OpenStack community (TC) see as the future of Glance?
- Maybe, more importantly, what does the wider community see as the
obligations of Glance?
- Does Glare fit into this vision?
- What kind of community support is there for Glare?

My reading of Glance history is that while some people were on board
with artifacts as the future of Glance, there was not a sufficient
critical mass of the Glance community that endorsed this direction and
that's why things unravelled in Newton.  I don't want to see that happen
again.  Further, I don't think the Glance community got the word out to
 

[openstack-dev] [tc][glance][glare][all] glance/glare/artifacts/images at the PTG

[Brian Rosmaita]

I want to give all interested parties a heads up that I have scheduled a
session in the Macon room from 9:30-10:30 a.m. on Thursday morning
(February 23).

Here's what we need to discuss.  This is from my perspective as Glance
PTL, so it's going to be Glance-centric.  This is a quick narrative
description; please go to the session etherpad [0] to turn this into a
specific set of discussion items.

Glance is the OpenStack image cataloging and delivery service.  A few
cycles ago (Juno?), someone noticed that maybe Glance could be
generalized so that instead of storing image metadata and image data,
Glance could store arbitrary digital "stuff" along with metadata
describing the "stuff".  Some people (like me) thought that this was an
obvious direction for Glance to take, but others (maybe wiser, cooler
heads) thought that Glance needed to focus on image cataloging and
delivery and make sure it did a good job at that.  Anyway, the Glance
mission statement was changed to include artifacts, but the Glance
community never embraced them 100%, and in Newton, Glare split off as
its own project (which made sense to me, there was too much unclarity in
Glance about how Glare fit in, and we were holding back development, and
besides we needed to focus on images), and the Glance mission statement
was re-amended specifically to exclude artifacts and focus on images and
metadata definitions.

OK, so the current situation is:
- Glance "does" image cataloging and delivery and metadefs, and that's
all it does.
- Glare is an artifacts service (cataloging and delivery) that can also
handle images.

You can see that there's quite a bit of overlap.  I gave you the history
earlier because we did try to work as a single project, but it did not
work out.

So, now we are in 2017.  The OpenStack development situation has been
fragile since the second half of 2016, with several big OpenStack
sponsors pulling way back on the amount of development resources being
contributed to the community.  This has left Glare in the position where
it cannot qualify as a Bit Tent project, even though there is interest
in artifacts.

Mike Fedosin, the PTL for Glare, has asked me about Glare becoming part
of the Glance project again.  I will be completely honest, I am inclined
to say "no".  I have enough problems just getting Glance stuff done (for
example, image import missed Ocata).  But in addition to doing what's
right for Glance, I want to do what's right for OpenStack.  And I look
at the overlap and think ...

Well, what I think is that I don't want to go through the Juno-Newton
cycles of argument again.  And we have to do what is right for our users.

The point of this session is to discuss:
- What does the Glance community see as the future of Glance?
- What does the wider OpenStack community (TC) see as the future of Glance?
- Maybe, more importantly, what does the wider community see as the
obligations of Glance?
- Does Glare fit into this vision?
- What kind of community support is there for Glare?

My reading of Glance history is that while some people were on board
with artifacts as the future of Glance, there was not a sufficient
critical mass of the Glance community that endorsed this direction and
that's why things unravelled in Newton.  I don't want to see that happen
again.  Further, I don't think the Glance community got the word out to
the broader OpenStack community about the artifacts project, and we got
a lot of pushback along the lines of "WTF? Glance needs to do images"
variety.  And probably rightly so -- Glance needs to do images.  My
point is that I don't want Glance to take Glare back unless it fits in
with what the community sees as the appropriate direction for Glance.
And I certainly don't want to take it back if the entire Glance
community is not on board.

Anyway, that's what we're going to discuss.  I've booked one of the
fishbowl rooms so we can get input from people beyond just the Glance
and Glare projects.

cheers,
brian

[0] https://etherpad.openstack.org/p/pike-glance-glare-discussion


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev