Re: [openstack-dev] [TripleO] os-refresh-config run frequency
On Thursday 26 June 2014 12:20:30 Clint Byrum wrote: Excerpts from Macdonald-Wallace, Matthew's message of 2014-06-26 04:13:31 -0700: Hi all, I've been working more and more with TripleO recently and whilst it does seem to solve a number of problems well, I have found a couple of idiosyncrasies that I feel would be easy to address. My primary concern lies in the fact that os-refresh-config does not run on every boot/reboot of a system. Surely a reboot *is* a configuration change and therefore we should ensure that the box has come up in the expected state with the correct config? This is easily fixed through the addition of an @reboot entry in /etc/crontab to run o-r-c or (less easily) by re-designing o-r-c to run as a service. My secondary concern is that through not running os-refresh-config on a regular basis by default (i.e. every 15 minutes or something in the same style as chef/cfengine/puppet), we leave ourselves exposed to someone trying to make a quick fix to a production node and taking that node offline the next time it reboots because the config was still left as broken owing to a lack of updates to HEAT (I'm thinking a quick change to allow root access via SSH during a major incident that is then left unchanged for months because no-one updated HEAT). There are a number of options to fix this including Modifying os-collect-config to auto-run os-refresh-config on a regular basis or setting os-refresh-config to be its own service running via upstart or similar that triggers every 15 minutes I'm sure there are other solutions to these problems, however I know from experience that claiming this is solved through education of users or (more severely!) via HR is not a sensible approach to take as by the time you realise that your configuration has been changed for the last 24 hours it's often too late! So I see two problems highlighted above. 1) We don't re-assert ephemeral state set by o-r-c scripts. You're right, and we've been talking about it for a while. The right thing to do is have os-collect-config re-run its command on boot. I don't think a cron job is the right way to go, we should just have a file in /var/run that is placed there only on a successful run of the command. If that file does not exist, then we run the command. I've just opened this bug in response: https://bugs.launchpad.net/os-collect-config/+bug/1334804 I have been looking into bug #1334804 and I have a review up to resolve it. I want to highlight something. Currently on a reboot we start all services via upstart (on debian anyways) and there have been quite a lot of issues around this - missing upstart scripts and timing issues. I don't know the issues on fedora. So with a fix to #1334804, on a reboot upstart will start all the services first (with potentially out-of-date configuration), then o-c-c will start o-r- c and will now configure all services and restart them or start them if upstart isn't configured properly. I would like to turn off all boot scripts for services we configure and leave all this to o-r-c. I think this will simplify things and put us in control of starting services. I believe that it will also narrow the gap between fedora and debian or debian and debian so what works on one should work on the other and make it easier for developers. Having the ability to service nova-api stop|start|restart is very handy but this will be a manually thing and I intend to leave that there. What do people think and how best do I push this forward. I feel that this leads into the the re-assert-system-state spec but mainly I think this is a bug and doesn't require a spec. I will be at the tripleo mid-cycle meetup next and willing to discuss this with anyone interested in this and put together the necessary bits to make this happen. Michael ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Use MariaDB by default on Fedora
I propose making mysql an abstract element and user must choose either percona or mariadb-rpm element.CI must be setup correctly Michael On Monday 30 June 2014 12:02:09 Clint Byrum wrote: Excerpts from Michael Kerrin's message of 2014-06-30 02:16:07 -0700: I am trying to finish off https://review.openstack.org/#/c/90134 - percona xtradb cluster for debian based system. I have read into this thread that I can error out on Redhat systems when trying to install percona and tell them to use mariadb instead, percona isn't support here. Is this correct? Probably. But if CI for Fedora breaks as a result you'll need a solution first. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Use MariaDB by default on Fedora
I am trying to finish off https://review.openstack.org/#/c/90134 - percona xtradb cluster for debian based system. I have read into this thread that I can error out on Redhat systems when trying to install percona and tell them to use mariadb instead, percona isn't support here. Is this correct? Michael On Friday 27 June 2014 16:49:58 James Slagle wrote: On Fri, Jun 27, 2014 at 4:13 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from James Slagle's message of 2014-06-27 12:59:36 -0700: Things are a bit confusing right now, especially with what's been proposed. Let me try and clarify (even if just for my own sake). Currently the choices offered are: 1. mysql percona with the percona tarball Percona Xtradb Cluster, not mysql percona 2. mariadb galera with mariadb.org packages 3. mariadb galera with rdo packages And, we're proposing to add 4. mysql percona with percona packages: https://review.openstack.org/#/c/90134 5. mariadb galera with fedora packages https://review.openstack.org/#/c/102815/ 4 replaces 1, but only for Ubuntu/Debian, it doesn't work on Fedora/RH 5 replaces 3 (neither of which work on Ubuntu/Debian, obviously) Do we still need 1? Fedora/RH + percona tarball. I personally don't think so. Do we still need 2? Fedora/RH or Ubuntu/Debian with galera packages from maraidb.org. For the Fedora/RH case, I doubt it, people will just use 5. 3 will be gone (replaced by 5). So, yes, I'd like to see 5 as the default for Fedora/RH and 4 as the default for Ubuntu/Debian, and both those tested in CI. And get rid of (or deprecate) 1-3. I'm actually more confused now than before I read this. The use of numbers is just making my head spin. There are 5 choices, some of which are not totally clear. Hence the need to clean things up. It can be stated this way I think: On RPM systems, use MariaDB Galera packages. If packages are in the distro, use distro packages. If packages are not in the distro, use RDO packages. There won't be a need to install from the RDO repositories. Mariadb galera packages are in the main Fedora package repositories, and for RHEL, they're in the epel repositories. On DEB systems, use Percona XtraDB Cluster packages. If packages are in the distro, use distro packages. If packages are not in the distro, use upstream packages. If anything doesn't match those principles, it is a bug. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] olso.config error on running Devstack
I have been seeing this problem also. My problem is actually with oslo.sphinx. I ran sudo pip install -r test-requirements.txt in cinder so that I could run the tests there, which installed oslo.sphinx. Strange thing is that the oslo.sphinx installed a directory called oslo in /usr/local/lib/python2.7/dist-packages with no __init__.py file. With this package installed like so I get the same error you get with oslo.config. I don't need oslo.sphinx so I just went and manually deleted the oslo directory and the oslo.sphinx* files in /usr/local/lib/python2.7/dist-packages. Everything worked fine after that. Not sure what to do about this, but that is my story Michael On Mon 23 Dec 2013 14:18:11 Sean Dague wrote: On 12/23/2013 11:52 AM, Ben Nemec wrote: On 2013-12-18 09:26, Sayali Lunkad wrote: Hello, I get the following error when I run stack.sh on Devstack Traceback (most recent call last): File /usr/local/bin/ceilometer-dbsync, line 6, in module from ceilometer.storage import dbsync File /opt/stack/ceilometer/ceilometer/storage/__init__.py, line 23, in module from oslo.config import cfg ImportError: No module named config ++ failed ++ local r=1 +++ jobs -p ++ kill ++ set +o xtrace Search gives me olso.config is installed. Please let me know of any solution. Devstack pulls oslo.config from git, so if you have it installed on the system through pip or something it could cause problems. If you can verify that it's only in /opt/stack/oslo.config, you might try deleting that directory and rerunning devstack to pull down a fresh copy. I don't know for sure what the problem is, but those are a couple of things to try. We actually try to resolve that here: https://github.com/openstack-dev/devstack/blob/master/lib/oslo#L43 However, have I said how terrible python packaging is recently? Basically you can very easily get yourself in a situation where *just enough* of the distro package is left behind that pip thinks its there, so won't install it, but the python loader doesn't, so won't work. Then much sadness. If anyone has a more fool proof way to fix this, suggestions appreciated. -Sean ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] swift keystone failing in devstack
Hi there,
we raised a bug https://bugs.launchpad.net/devstack/+bug/1193112 where the
default configuration of swift in devstack interacts badly with keystone. We
end up with the following exception iwth swift-proxy-server:
$ cd /opt/stack/swift /opt/stack/swift/bin/swift-proxy-server
/etc/swift/proxy-server.conf -v || touch /opt/stack/status/stack/s-
proxy.failure
proxy-server Starting keystone auth_token middleware
proxy-server Using /var/cache/swift as cache directory for signing certificate
proxy-server signing_dir mode is 0755 instead of 0700
proxy-server Starting the S3 Token Authentication component
Traceback (most recent call last):
File /opt/stack/swift/bin/swift-proxy-server, line 22, in module
run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
File /opt/stack/swift/swift/common/wsgi.py, line 249, in run_wsgi
loadapp(conf_path, global_conf={'log_name': log_name})
File /opt/stack/swift/swift/common/wsgi.py, line 100, in wrapper
return f(conf_uri, *args, **kwargs)
File /usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py, line 247,
in loadapp
return loadobj(APP, uri, name=name, **kw)
File /usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py, line 272,
in loadobj
return context.create()
File /usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py, line 710,
in create
return self.object_type.invoke(self)
File /usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py, line 207,
in invoke
app = filter(app)
File /opt/stack/keystone/keystone/middleware/s3_token.py, line 215, in
auth_filter
return S3Token(app, conf)
File /opt/stack/keystone/keystone/middleware/s3_token.py, line 65, in
__init__
self.http_client_class = environment.httplib.HTTPConnection
AttributeError: 'NoneType' object has no attribute 'HTTPConnection'
I raised a work around here https://review.openstack.org/#/c/34207/ not in the
hope that this code will get committed but in the hope that someone will see
this and know what to do. That review is also a work around for anyone hit by
this bug.
So if anyone knows what to do for this bug please help,
Michael
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
