Re: [Openstack-operators] Best kernel options for openvswitch on network nodes on a large setup
Jean-Philippe Méthot writes: > This particular message makes it sound as if openvswitch is getting > overloaded. > Sep 23 03:54:08 network1 ovsdb-server: > ovs|01253|reconnect|ERR|tcp:127.0.0.1:50814: no response to inactivity probe > after 5.01 seconds, disconnecting We get these as well :-( > A lot of those keep appear, and openvswitch always reconnects almost > instantly though. I’ve done some research about that particular > message, but it didn’t give me anything I can use to fix it. Would be interested in solutions as well. But I'm sceptical whether kernel settings can help here, because the timeout/slowness seems to be located in the user-space/control-plane parts of Open vSwitch, i.e. OVSDB. -- Simon. > Jean-Philippe Méthot > Openstack system administrator > Administrateur système Openstack > PlanetHoster inc. > Le 25 sept. 2018 à 19:37, Erik McCormick a > écrit : > Ate you getting any particular log messages that lead you to conclude your > issue lies with OVS? I've hit lots of kernel limits under those conditions > before OVS itself ever > noticed. Anything in dmesg, journal or neutron logs of interest? > On Tue, Sep 25, 2018, 7:27 PM Jean-Philippe Méthot > wrote: > Hi, > Are there some recommendations regarding kernel settings configuration for > openvswitch? We’ve just been hit by what we believe may be an attack of some > kind we > have never seen before and we’re wondering if there’s a way to optimize our > network nodes kernel for openvswitch operation and thus minimize the impact > of such an > attack, or whatever it was. > Best regards, > Jean-Philippe Méthot > Openstack system administrator > Administrateur système Openstack > PlanetHoster inc. > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > ___ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] [openstack-dev] [nova] Default scheduler filters survey
Artom Lifshitz writes: > To that end, we'd like to know what filters operators are enabling in > their deployment. If you can, please reply to this email with your > [filter_scheduler]/enabled_filters (or > [DEFAULT]/scheduler_default_filters if you're using an older version) > option from nova.conf. Any other comments are welcome as well :) We have the following enabled on our semi-public (academic community) cloud, which runs on Newton: AggregateInstanceExtraSpecsFilter AvailabilityZoneFilter ComputeCapabilitiesFilter ComputeFilter ImagePropertiesFilter PciPassthroughFilter RamFilter RetryFilter ServerGroupAffinityFilter ServerGroupAntiAffinityFilter (sorted alphabetically) Recently we've also been trying AggregateImagePropertiesIsolation ...but it looks like we'll replace it with our own because it's a bit awkward to use for our purpose (scheduling Windows instance to licensed compute nodes). -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] How are you handling billing/chargeback?
Lars Kellogg-Stedman writes: > I'm curious what folks out there are using for chargeback/billing in > your OpenStack environment. We use a homegrown billing system that periodically samples utilization of billable resources. (We turned off Ceilometer a few years ago because we weren't really using it and found that it caused us trouble.) -- Simon. > Are you doing any sort of chargeback (or showback)? Are you using (or > have you tried) CloudKitty? Or some other existing project? Have you > rolled your own instead? > I ask because I am helping out some folks get a handle on the > operational side of their existing OpenStack environment, and they are > interested in but have not yet deployed some sort of reporting > mechanism. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] Milan Ops Midcycle - "Do not do..." session [CORRECTED]
Here's the initial Etherpad for the Wednesday afternoon session "DO NOT DO... UNDER ANY CIRCUMSTANCES": https://etherpad.openstack.org/p/MIL-ops-do-not-do [Corrected link - thanks Maxime Guyot for pointing out the error!] Please add your war stories and "lessons learned". Thanks & looking forward to the discussion in Milan! -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] Milan Ops Midcycle - "Do not do..." session
Here's the initial Etherpad for the Wednesday afternoon session "DO NOT DO... UNDER ANY CIRCUMSTANCES": https://etherpad.openstack.org/p/MIL-ops-scaling-and-tuning Please add your war stories and "lessons learned". Thanks & looking forward to the discussion in Milan! -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
[Openstack-operators] Milan Ops Midcycle - Scaling & Tuning session
Here's the initial Etherpad for the Wednesday afternoon session on SCALING & TUNING: https://etherpad.openstack.org/p/MIL-ops-scaling-and-tuning Please contribute your tuning tips (and questions), scaling woes (and solutions), etc. Thanks & looking forward to the discussion in Milan! -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] [User-committee] Seeking feedback: Active User Contributor (AUC) eligibility requirements
Shamail Tahir writes: > The AUC Recognition WG has been hard at work on milestone-4 of our > plan which is to identify the eligibility criteria for each community > contributor role that is covered by AUC. > [...] Thank you in advance for your feedback! +1 Looks good, thanks all for the work figuring these out. -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] How do you handle purge of database tables ?
Nick Jones writes: > Another vote of confidence for the script that Tim has mentioned with > regards to clearing down Nova’s DB. I blogged a bit about the process > and the results here: > http://dischord.org/2015/12/30/archiving-data-in-nova-s-database/ Unfortunately the nova/openstack_db_archive.sh seems to be broken by https://review.openstack.org/#/c/269530/3 ; I have created a bug: https://bugs.launchpad.net/osops/+bug/1596193 Can anyone confirm? I have a fix locally and will push a review soon. -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Upgrade OpenStack Juno to Mitaka
Michael Stang writes: > Is this one the actual guid for upgrades, and is it valid for every > upgrade or ony for specific versions?: > http://docs.openstack.org/ops-guide/ops_upgrades.html Yes, that's part of the official Operations Guide. It is not version-specific. The examples are based on Ubuntu as the underlying OS distribution. But the approach and recommendations are general. -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
Re: [Openstack-operators] Ephemeral instances in RBD issue
Abel Lopez writes: I saw this last commit to jdurgin's nova fork which solves the issue ( https://github.com/jdurgin/nova/commit/ea4b5369e4bec4dd7a0ce9f68769600329cda6c6 ) now a snapshot happens in seconds. The problem that we've introduced however, is that about 15-20m after we do a snapshot, the VM is powered off. Every time. Ouch! Have you checked the logs (nova-compute and maybe libvirtd's)? I can start the instance back up with `nova start`, but I am leery of pushing this out to prod and having to tell users to expect a shutdown after a snapshot. Understood. Anyone else using this in Havana? Not me, but I'm sympathetic with your worries, and want this resolved as well. We're using Icehouse with RBD, currently without the ephemeral patches, but we would really like to (re-) activate that part of the integration soon. It's maybe worth asking on #ceph or posting to one of the CEPH mailing lists, too. Good luck, -- Simon. ___ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
