Doug, I totally agree with your findings on the policy module. Neutron already has some "customizations" there and we already have a few contributors working on syncing it back with oslo-incubator during the Kilo release cycle.
However, my query was about the quota module. >From what I gather it seems not a lot of projects use it: $ find . -name openstack-common.conf | xargs grep quota $ Salvatore On 15 October 2014 00:34, Doug Hellmann <d...@doughellmann.com> wrote: > > On Oct 14, 2014, at 12:31 PM, Salvatore Orlando <sorla...@nicira.com> > wrote: > > Hi Doug, > > do you know if the existing quota oslo-incubator module has already some > active consumers? > In the meanwhile I've pushed a spec to neutron-specs for improving quota > management there [1] > > > It looks like a lot of projects are syncing the module: > > $ grep policy */openstack-common.conf > > barbican/openstack-common.conf:modules=gettextutils,jsonutils,log,local,timeutils,importutils,policy > ceilometer/openstack-common.conf:module=policy > cinder/openstack-common.conf:module=policy > designate/openstack-common.conf:module=policy > gantt/openstack-common.conf:module=policy > glance/openstack-common.conf:module=policy > heat/openstack-common.conf:module=policy > horizon/openstack-common.conf:module=policy > ironic/openstack-common.conf:module=policy > keystone/openstack-common.conf:module=policy > manila/openstack-common.conf:module=policy > neutron/openstack-common.conf:module=policy > nova/openstack-common.conf:module=policy > trove/openstack-common.conf:module=policy > tuskar/openstack-common.conf:module=policy > > I’m not sure how many are actively using it, but I wouldn’t expect them to > copy it in if they weren’t using it at all. > > > Now, I can either work on the oslo-incubator module and leverage it in > Neutron, or develop the quota module in Neutron, and move it to > oslo-incubator once we validate it with Neutron. The latter approach seems > easier from a workflow perspective - as it avoid the intermediate steps of > moving code from oslo-incubator to neutron. On the other hand it will delay > adoption in oslo-incubator. > > > The policy module is up for graduation this cycle. It may end up in its > own library, to allow us to build a review team for the code more easily > than if we put it in with some of the other semi-related modules like the > server code. We’re still working that out [1], and if you expect to make a > lot of incompatible changes we should delay graduation to make that simpler. > > Either way, since we have so many consumers, I think it would be easier to > have the work happen in Oslo somewhere so we can ensure those changes are > useful to and usable by all of the existing consumers. > > Doug > > [1] https://etherpad.openstack.org/p/kilo-oslo-library-proposals > > > What's your opinion? > > Regards, > Salvatore > > [1] https://review.openstack.org/#/c/128318/ > > On 8 October 2014 18:52, Doug Hellmann <d...@doughellmann.com> wrote: > >> >> On Oct 8, 2014, at 7:03 AM, Davanum Srinivas <dava...@gmail.com> wrote: >> >> > Salvatore, Joe, >> > >> > We do have this at the moment: >> > >> > >> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/quota.py >> > >> > — dims >> >> If someone wants to drive creating a useful library during kilo, please >> consider adding the topic to the etherpad we’re using to plan summit >> sessions and then come participate in the Oslo meeting this Friday 16:00 >> UTC. >> >> https://etherpad.openstack.org/p/kilo-oslo-summit-topics >> >> Doug >> >> > >> > On Wed, Oct 8, 2014 at 2:29 AM, Salvatore Orlando <sorla...@nicira.com> >> wrote: >> >> >> >> On 8 October 2014 04:13, Joe Gordon <joe.gord...@gmail.com> wrote: >> >>> >> >>> >> >>> On Fri, Oct 3, 2014 at 10:47 AM, Morgan Fainberg >> >>> <morgan.fainb...@gmail.com> wrote: >> >>>> >> >>>> Keeping the enforcement local (same way policy works today) helps >> limit >> >>>> the fragility, big +1 there. >> >>>> >> >>>> I also agree with Vish, we need a uniform way to talk about quota >> >>>> enforcement similar to how we have a uniform policy language / >> enforcement >> >>>> model (yes I know it's not perfect, but it's far closer to uniform >> than >> >>>> quota management is). >> >>> >> >>> >> >>> It sounds like maybe we should have an oslo library for quotas? >> Somewhere >> >>> where we can share the code,but keep the operations local to each >> service. >> >> >> >> >> >> This is what I had in mind as well. A simple library for quota >> enforcement >> >> which can be used regardless of where and how you do it, which might >> depend >> >> on the application business logic, the WSGI framework in use, or other >> >> factors. >> >> >> >>> >> >>>> >> >>>> >> >>>> If there is still interest of placing quota in keystone, let's talk >> about >> >>>> how that will work and what will be needed from Keystone . The >> previous >> >>>> attempt didn't get much traction and stalled out early in >> implementation. If >> >>>> we want to revisit this lets make sure we have the resources needed >> and >> >>>> spec(s) in progress / info on etherpads (similar to how the >> multitenancy >> >>>> stuff was handled at the last summit) as early as possible. >> >>> >> >>> >> >>> Why not centralize quota management via the python-openstackclient, >> what >> >>> is the benefit of getting keystone involved? >> >> >> >> >> >> Providing this through the openstack client in my opinion has the >> >> disadvantage that users which either use the REST API direct or write >> their >> >> own clients won't leverage it. I don't think it's a reasonable >> assumption >> >> that everybody will use python-openstackclient, is it? >> >> >> >> Said that, storing quotas in keystone poses a further challenge to the >> >> scalability of the system, which we shall perhaps address by using >> >> appropriate caching strategies and leveraging keystone notifications. >> Until >> >> we get that, I think that the openstack client will be the best way of >> >> getting a unified quota management experience. >> >> >> >> Salvatore >> >> >> >> >> >>>> >> >>>> Cheers, >> >>>> Morgan >> >>>> >> >>>> Sent via mobile >> >>>> >> >>>> >> >>>> On Friday, October 3, 2014, Salvatore Orlando <sorla...@nicira.com> >> >>>> wrote: >> >>>>> >> >>>>> Thanks Vish, >> >>>>> >> >>>>> this seems a very reasonable first step as well - and since most >> >>>>> projects would be enforcing quotas in the same way, the shared >> library would >> >>>>> be the logical next step. >> >>>>> After all this is quite the same thing we do with authZ. >> >>>>> >> >>>>> Duncan is expressing valid concerns which in my opinion can be >> addressed >> >>>>> with an appropriate design - and a decent implementation. >> >>>>> >> >>>>> Salvatore >> >>>>> >> >>>>> On 3 October 2014 18:25, Vishvananda Ishaya <vishvana...@gmail.com> >> >>>>> wrote: >> >>>>>> >> >>>>>> The proposal in the past was to keep quota enforcement local, but >> to >> >>>>>> put the resource limits into keystone. This seems like an obvious >> first >> >>>>>> step to me. Then a shared library for enforcing quotas with decent >> >>>>>> performance should be next. The quota calls in nova are extremely >> >>>>>> inefficient right now and it will only get worse when we try to add >> >>>>>> hierarchical projects and quotas. >> >>>>>> >> >>>>>> Vish >> >>>>>> >> >>>>>> On Oct 3, 2014, at 7:53 AM, Duncan Thomas <duncan.tho...@gmail.com >> > >> >>>>>> wrote: >> >>>>>> >> >>>>>>> Taking quota out of the service / adding remote calls for quota >> >>>>>>> management is going to make things fragile - you've somehow got to >> >>>>>>> deal with the cases where your quota manager is slow, goes away, >> >>>>>>> hiccups, drops connections etc. You'll also need some way of >> >>>>>>> reconciling actual usage against quota usage periodically, to >> detect >> >>>>>>> problems. >> >>>>>>> >> >>>>>>> On 3 October 2014 15:03, Salvatore Orlando <sorla...@nicira.com> >> >>>>>>> wrote: >> >>>>>>>> Hi, >> >>>>>>>> >> >>>>>>>> Quota management is currently one of those things where every >> >>>>>>>> openstack >> >>>>>>>> project does its own thing. While quotas are obviously managed >> in a >> >>>>>>>> similar >> >>>>>>>> way for each project, there are subtle differences which >> ultimately >> >>>>>>>> result >> >>>>>>>> in lack of usability. >> >>>>>>>> >> >>>>>>>> I recall that in the past there have been several calls for >> unifying >> >>>>>>>> quota >> >>>>>>>> management. The blueprint [1] for instance, hints at the >> possibility >> >>>>>>>> of >> >>>>>>>> storing quotas in keystone. >> >>>>>>>> On the other hand, the blazar project [2, 3] seems to aim at >> solving >> >>>>>>>> this >> >>>>>>>> problem for good enabling resource reservation and therefore >> >>>>>>>> potentially >> >>>>>>>> freeing openstack projects from managing and enforcing quotas. >> >>>>>>>> >> >>>>>>>> While Blazar is definetely a good thing to have, I'm not entirely >> >>>>>>>> sure we >> >>>>>>>> want to make it a "required" component for every deployment. >> Perhaps >> >>>>>>>> single >> >>>>>>>> projects should still be able to enforce quota. On the other >> hand, >> >>>>>>>> at least >> >>>>>>>> on paper, the idea of making Keystone "THE" endpoint for managing >> >>>>>>>> quotas, >> >>>>>>>> and then letting the various project enforce them, sounds >> promising >> >>>>>>>> - is >> >>>>>>>> there any reason for which this blueprint is stalled to the point >> >>>>>>>> that it >> >>>>>>>> seems forgotten now? >> >>>>>>>> >> >>>>>>>> I'm coming to the mailing list with these random questions about >> >>>>>>>> quota >> >>>>>>>> management, for two reasons: >> >>>>>>>> 1) despite developing and using openstack on a daily basis I'm >> still >> >>>>>>>> confused by quotas >> >>>>>>>> 2) I've found a race condition in neutron quotas and the fix is >> not >> >>>>>>>> trivial. >> >>>>>>>> So, rather than start coding right away, it might probably make >> more >> >>>>>>>> sense >> >>>>>>>> to ask the community if there is already a known better approach >> to >> >>>>>>>> quota >> >>>>>>>> management - and obviously enforcement. >> >>>>>>>> >> >>>>>>>> Thanks in advance, >> >>>>>>>> Salvatore >> >>>>>>>> >> >>>>>>>> [1] >> https://blueprints.launchpad.net/keystone/+spec/service-metadata >> >>>>>>>> [2] https://wiki.openstack.org/wiki/Blazar >> >>>>>>>> [3] >> https://review.openstack.org/#/q/project:stackforge/blazar,n,z >> >>>>>>>> >> >>>>>>>> _______________________________________________ >> >>>>>>>> OpenStack-dev mailing list >> >>>>>>>> OpenStack-dev@lists.openstack.org >> >>>>>>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> -- >> >>>>>>> Duncan Thomas >> >>>>>>> >> >>>>>>> _______________________________________________ >> >>>>>>> OpenStack-dev mailing list >> >>>>>>> OpenStack-dev@lists.openstack.org >> >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>>>>> >> >>>>>> >> >>>>>> _______________________________________________ >> >>>>>> OpenStack-dev mailing list >> >>>>>> OpenStack-dev@lists.openstack.org >> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>>>>> >> >>>>> >> >>>> >> >>>> _______________________________________________ >> >>>> OpenStack-dev mailing list >> >>>> OpenStack-dev@lists.openstack.org >> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>>> >> >>> >> >>> >> >>> _______________________________________________ >> >>> OpenStack-dev mailing list >> >>> OpenStack-dev@lists.openstack.org >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >> >> >> >> >> _______________________________________________ >> >> OpenStack-dev mailing list >> >> OpenStack-dev@lists.openstack.org >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> > >> > >> > >> > -- >> > Davanum Srinivas :: https://twitter.com/dims >> > >> > _______________________________________________ >> > OpenStack-dev mailing list >> > OpenStack-dev@lists.openstack.org >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev