Hi, Alex Schultz <[email protected]> writes:
> On Wed, Jun 7, 2017 at 5:20 AM, Sofer Athlan-Guyot <[email protected]> > wrote: >> Hi, >> >> Emilien Macchi <[email protected]> writes: >> >>> On Wed, Jun 7, 2017 at 12:45 PM, Sofer Athlan-Guyot <[email protected]> >>> wrote: >>>> Hi, >>>> >>>> I don't think we have such a job in place. Basically that would check >>>> that re-running the "openstack deploy ..." command won't do anything. I've had a look at openstack-infra/tripleo-ci. Should I test it in with ovb/quickstart or tripleo.sh. Both way are fine by me, but I may be lacking context about which one is more relevant. >>>> We had such an error by the past[1], but I'm not sure this has been >>>> captured by an associated job. >>>> >>>> WDYT ? >>> >>> It would be interesting to measure how much time does it take to run >>> it again. >> >> Could you point out how such an experiment could be done ? >> >>> If it's short, we could add it to all our scenarios + ovb >>> jobs. If it's long, maybe we need an additional job, but it would >>> take more resources, so maybe we could run it in periodic pipeline >>> (note that periodic jobs are not optimal since we could break >>> something quite easily). >> >> Just adding as context that the issue was already raised[1]. Beside >> time constraint, it was pointed out that we would also need to parse the >> log to find out if anything was restarted. But it could be a second >> step. For parsing, this code was pointed out[2]. >> > > There's a few things that would need to be enabled in order to reuse > some of this work. We'll need to add the ability to generate a report > on the puppet run[0]. And then we'll need to be able to capture it[1] > somewhere that we could then use that parsing code on. From there, > just rerunning the installation would be a simple start to the > idempotency check. In fuel, we had hacked in a special flag[2] that > we used in testing to actually rerun the task immediately to find when > a specific task was not idempotent in addition to also rerunning the > entire deployment. For tripleo a similar concept would be to rerun the > steps twice but that's usually not where the issues crop us for us. So > rerunning the entire installation deployment would be better as we > tend to have issues with configuration items between steps > conflicting. Maybe we could go with something equivalent to: ts="$(date '+%F %T')" ... re-run deploy command ... sudo journalctl --since="${ts}" | egrep 'Stopping|Starting' | grep -v 'user.*slice' > restarted.log wc -l restarted.log This should be 0 on every overcloud nodes. This is simpler to implement and should catch any unwanted service restart. WDYT ? > > Thanks, > -Alex > > [0] https://review.openstack.org/#/c/273740/4/mcagents/puppetd.rb@204 > [1] https://review.openstack.org/#/c/273740/4/mcagents/puppetd.rb@102 > [2] https://review.openstack.org/#/c/273737/ > >> [1] http://lists.openstack.org/pipermail/openstack-dev/2017-March/114836.html >> [2] >> https://review.openstack.org/#/c/279271/9/fuelweb_test/helpers/astute_log_parser.py@212 >> >>> >>>> [1] https://bugs.launchpad.net/tripleo/+bug/1664650 >>>> -- >>>> Sofer Athlan-Guyot >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: [email protected]?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> -- >>> Emilien Macchi >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: [email protected]?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> -- >> Sofer Athlan-Guyot >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Thanks, -- Sofer Athlan-Guyot __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
