We had a mini tutorial today in #openstack-infra, were Clark Boylan explained how one can bring up an environment to debug logstash-2.0. This is tangential to "debugging the gate", but still could be useful to better understand logstash/es pipeline.
I did condense the conversation into the doc, excuse any grammar / punctuation that I missed: http://paste.openstack.org/show/479346/ -- Mikhail Medvedev On Tue, Nov 10, 2015, at 10:21, Sean Dague wrote: > There is also a neutron issue which is causing a 35% failure rate in > neutron jobs - > http://tinyurl.com/ne3ex4v > > That one still needs resolution. > > On 11/10/2015 10:54 AM, Davanum Srinivas wrote: > > Took about 35 mins or so :) > > > > -- Dims > > > > On Tue, Nov 10, 2015 at 10:45 AM, Matt Riedemann > > <[email protected]> wrote: > >> > >> > >> On 11/9/2015 3:54 PM, Anita Kuno wrote: > >>> > >>> On 11/05/2015 07:45 PM, Anita Kuno wrote: > >>>> > >>>> On 11/03/2015 05:30 PM, Anita Kuno wrote: > >>>>> > >>>>> On 11/02/2015 12:39 PM, Anita Kuno wrote: > >>>>>> > >>>>>> On 10/29/2015 10:42 PM, Anita Kuno wrote: > >>>>>>> > >>>>>>> On 10/29/2015 08:27 AM, Anita Kuno wrote: > >>>>>>>> > >>>>>>>> On 10/28/2015 12:14 AM, Matt Riedemann wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 10/27/2015 4:08 AM, Anita Kuno wrote: > >>>>>>>>>> > >>>>>>>>>> Learning how to debug the gate was identified as a theme at the > >>>>>>>>>> "Establish Key Themes for the Mitaka Cycle" cross-project session: > >>>>>>>>>> https://etherpad.openstack.org/p/mitaka-crossproject-themes > >>>>>>>>>> > >>>>>>>>>> I agreed to take on this item and facilitate the process. > >>>>>>>>>> > >>>>>>>>>> Part one of the conversation includes referencing this video > >>>>>>>>>> created by > >>>>>>>>>> Sean Dague and Dan Smith: > >>>>>>>>>> https://www.youtube.com/watch?v=fowBDdLGBlU > >>>>>>>>>> > >>>>>>>>>> Please consume this as you are able. > >>>>>>>>>> > >>>>>>>>>> Other suggestions for how to build on this resource were mentioned > >>>>>>>>>> and > >>>>>>>>>> will be coming in the future but this was an easy, actionable first > >>>>>>>>>> step. > >>>>>>>>>> > >>>>>>>>>> Thank you, > >>>>>>>>>> Anita. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> __________________________________________________________________________ > >>>>>>>>>> > >>>>>>>>>> OpenStack Development Mailing List (not for usage questions) > >>>>>>>>>> Unsubscribe: > >>>>>>>>>> [email protected]?subject:unsubscribe > >>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/tales-from-the-gate-how-debugging-the-gate-helps-your-enterprise > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> The source for the definition of "the gate": > >>>>>>>> > >>>>>>>> http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n34 > >>>>>>>> > >>>>>>>> Thanks for following along, > >>>>>>>> Anita. > >>>>>>>> > >>>>>>> > >>>>>>> This is the status page showing the status of our running jobs, > >>>>>>> including patches in the gate pipeline: > >>>>>>> http://status.openstack.org/zuul/ > >>>>>>> > >>>>>>> Thank you, > >>>>>>> Anita. > >>>>>>> > >>>>>> > >>>>>> This is a simulation of how the gate tests patches: > >>>>>> http://docs.openstack.org/infra/publications/zuul/#%2818%29 > >>>>>> > >>>>>> Click in the browser window to advance the simulation. > >>>>>> > >>>>>> Thank you, > >>>>>> Anita. > >>>>>> > >>>>> > >>>>> Here is a presentation that uses the slide deck linked above, I > >>>>> recommend watching: https://www.youtube.com/watch?v=WDoSCGPiFDQ > >>>>> > >>>>> Thank you, > >>>>> Anita. > >>>>> > >>>> > >>>> Three links in this edition of Learning to Debug the Gate: > >>>> > >>>> The view that tracks our top bugs: > >>>> http://status.openstack.org/elastic-recheck/ > >>>> > >>>> The logstash queries that create the above view: > >>>> > >>>> http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/queries > >>>> > >>>> Logstash itself, where you too can practice creating queries: > >>>> http://logstash.openstack.org > >>>> > >>>> Note: in logstash the query is the transferable piece of information. > >>>> Filters can help you create a query, they do not populate a query. The > >>>> information that is in the query bar is what is important here. > >>>> > >>>> Practice making some queries of your own. > >>>> > >>>> Thanks for reading, > >>>> Anita. > >>>> > >>> > >>> Today the elastic search cluster was upgraded to 1.7.3: > >>> > >>> http://lists.openstack.org/pipermail/openstack-dev/2015-November/078314.html > >>> > >>> Go over the 1.7 elastic search docs: > >>> https://www.elastic.co/guide/en/elasticsearch/reference/1.7/index.html > >>> and try a few queries. > >>> > >>> Thanks for following along, > >>> Anita. > >>> > >>> __________________________________________________________________________ > >>> OpenStack Development Mailing List (not for usage questions) > >>> Unsubscribe: [email protected]?subject:unsubscribe > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> > >> > >> If you want a fun little play by play of a critical blocking gate issue for > >> nova, it's in the nova irc logs starting here: > >> > >> http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2015-11-10.log.html#t2015-11-10T14:59:27 > >> > >> Which led to a block of oslo.messaging 2.8.0 in g-r and a revert in the o.m > >> repo. The point isn't the outcome, but the process in which we figured out > >> what the regression was and quickly resolved it using logstash and seeing > >> what recent dependent library releases impacted nova. > >> > >> -- > >> > >> Thanks, > >> > >> Matt Riedemann > >> > >> > >> __________________________________________________________________________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: [email protected]?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > -- > Sean Dague > http://dague.net > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
