Thank you Morgan for quick fixes proposed! On Tue, Apr 12, 2016 at 6:19 PM, Morgan Fainberg <morgan.fainb...@gmail.com> wrote:
> Sorry Missed the copy/paste of links: > > https://bugs.launchpad.net/keystone/+bug/1567403 [0] > https://bugs.launchpad.net/keystone/+bug/1567413 [1] > > [0] > https://review.openstack.org/#/q/I4857cfe1e62d54c3c89a0206ffc895c4cf681ce5,n,z > [1] https://review.openstack.org/#/c/304688/ > > --Morgan > > On Tue, Apr 12, 2016 at 8:16 AM, Morgan Fainberg < > morgan.fainb...@gmail.com> wrote: > >> Fixes have been proposed for both of these bugs. >> >> Cheers, >> --Morgan >> >> On Tue, Apr 12, 2016 at 12:38 AM, Dina Belova <dbel...@mirantis.com> >> wrote: >> >>> Matt, >>> >>> Thanks for sharing the information about your benchmark. Indeed we need >>> to follow up on this topic (I'll attend the summit). Let's try to collect >>> as much information as possible prior Austin to have more facts to operate. >>> I'll try to figure out why local context cache did not work at least on my >>> environment, and, due to the results, most probably it did not act as >>> supposed during your benchmarking as well. >>> >>> Cheers, >>> Dina >>> >>> On Mon, Apr 11, 2016 at 10:57 PM, Matt Fischer <m...@mattfischer.com> >>> wrote: >>> >>>> On Mon, Apr 11, 2016 at 8:11 AM, Dina Belova <dbel...@mirantis.com> >>>> wrote: >>>> >>>>> Hey, openstackers! >>>>> >>>>> Recently I was trying to profile Keystone (OpenStack Liberty vs >>>>> Mitaka) using this set of changes >>>>> <https://review.openstack.org/#/q/topic:osprofiler-support-in-keystone+status:open> >>>>> (that's >>>>> currently on review - some final steps are required there to finish the >>>>> work) and OSprofiler. >>>>> >>>>> Some preliminary results (all in one OpenStack node) can be found here >>>>> <http://docs.openstack.org/developer/performance-docs/test_results/keystone/all-in-one/index.html> >>>>> (raw >>>>> OSprofiler reports are not yet merged to some place and can be found >>>>> here <https://review.openstack.org/#/c/299514/>). The full plan >>>>> <http://docs.openstack.org/developer/performance-docs/test_plans/keystone/plan.html#keystone-performance> >>>>> of >>>>> what's going to be tested can be found in the docs as well. In short I >>>>> wanted to take a look how does Keystone changed its DB/Cache usage from >>>>> Liberty to Mitaka, keeping in mind that there were several changes >>>>> introduced: >>>>> >>>>> - federation support was added (and made DB scheme a bit more >>>>> complex) >>>>> - Keystone moved to oslo.cache usage >>>>> - local context cache was introduced during Mitaka >>>>> >>>>> First of all - *good job on making Keystone less DB-extensive in case >>>>> of cache turned on*! If Keystone caching is turned on, number of DB >>>>> queries done to Keystone DB in Mitaka is averagely twice less than in >>>>> Liberty, comparing the same requests and topologies. Thanks Keystone >>>>> community to make it happen :) >>>>> >>>>> Although, I faced *two strange issues* during my experiments, and I'm >>>>> kindly asking you, folks, to help me here: >>>>> >>>>> - I've created #1567403 >>>>> <https://bugs.launchpad.net/keystone/+bug/1567403> bug to share >>>>> information - when I turned caching on, local context cache should >>>>> cache >>>>> identical per API requests function calls not to ping Memcache too >>>>> often. >>>>> Although I faced such calls, Keystone still used Memcache to gather >>>>> this >>>>> information. May someone take a look on this and help me figure out >>>>> what am >>>>> I observing? At the first sight local context cache should work ok, >>>>> but for >>>>> some reason I do not see it's being used. >>>>> - One more filed bug - #1567413 >>>>> <https://bugs.launchpad.net/keystone/+bug/1567413> - is about a >>>>> bit opposite situation :) When I turned cache off explicitly in the >>>>> keystone.conf file, I still observed some of the values being fetched >>>>> from >>>>> Memcache... Your help is very appreciated! >>>>> >>>>> Thanks in advance and sorry for a long email :) >>>>> >>>>> Cheers, >>>>> Dina >>>>> >>>>> >>>> Dina, >>>> >>>> Thanks for starting this conversation. I had some weird perf results >>>> comparing L to an RC release of Mitaka, but I was holding them until >>>> someone else confirmed what I saw. I'm testing token creation and >>>> validation. From what I saw, token validation slowed down in Mitaka. After >>>> doing my benchmark runs, the traffic to memcache was 8x in Mitaka from what >>>> it was in Liberty. That implies more caching but 8x is a lot and even >>>> memcache references are not free. >>>> >>>> I know some of the Keystone folks are looking into this so it will be >>>> good to follow-up on it. Maybe we could talk about this at the summit? >>>> >>>> >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> >>> -- >>> >>> Best regards, >>> >>> Dina Belova >>> >>> Software Engineer >>> >>> Mirantis Inc. >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Best regards, Dina Belova Software Engineer Mirantis Inc.
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev