On 03.04.2017 21:01, Dan Prince wrote: > On Mon, 2017-04-03 at 16:15 +0200, Bogdan Dobrelya wrote: >> Let's please re-evaluate configuration of containerized non- >> openstack, >> like database, message queue, key value, web, services for tripleo >> heat >> templates and Kolla. Here is an example containerized etcd patch [0]. >> >> tl;dr use kolla images and bootsrap OR upstream images with direct >> commands: >> >> .. code-block:: yaml >> kolla_config: >> /var/lib/kolla/config_files/foo.json >> command: /usr/bin/foo >> >> vs >> >> .. code-block:: yaml >> foo: >> image: upstream/foo:latest >> command: /usr/bin/foo >> >> Note, tht already doesn't use configs [1] copied into the images by >> kolla build time. The next and logical step might be to omit kolla's >> bootstrap, where applicable, as well. > > The kolla config file copying proved to be a bit pedantic. So we > removed it. A good example of this would be how this played out for the > keystone service: > > http://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/? > id=332e8ec10345ad5c8bf10a532f6f6003da682b68 > >> >> There is a two options: >> * use kolla images and bootstrap as t-h-t does now for all services >> being containerized > > We do not use KOLLA_BOOTSTRAP for all services now. Only 4 services use > it currently I think. I think the general consensus is we should not be > using it unless there is a functional requirement to do so. Services > like Mysql and Rabbitmq have some extra initialization that needs to > execute in-container before the services startup. We could duplicate > this in tripleo-heat-templates, run it with docker-cmd perhaps and do > it that way but we initially made exception for just a few services. > > Other services like Glance are using it, but that is just historical. > There is already a patch to remove the use of KOLLA_BOOTSTRAP for this > service: > > https://review.openstack.org/#/c/440884/1 > > So in summary the consensus is we'd prefer not to be using the > KOLLA_BOOTSTRAP environment variables because in some cases there is a > 'Kolla' flavor to these things that don't match how TripleO deploys > things. > > It is worth pointing out that while we aren't using KOLLA_BOOTSTRAP we > are using the kolla startup systems in many cases. This gives some > features around file permissions, extra sudoers files, etc. We may be > able to stop using this for some services but I also think we are > getting value out of the interfaces today. They aren't nearly as > verbose as the Kolla config copying stuff so we could go either way.
That's interesting, there is related topic with hostpath mounted persistent log dirs [0]. The issue is: [tl;dr] kolla_config is a bad fit for logs, let's run containers' steps as 'user: root' in a user namespace remapped [1] for a tripleocontainers system user. Thoughts? And would that work for stock centos7/rhel7 kernels? So, the issue is when log/data dirs created by host_prep_tasks, we must configure host permissions in a simple non-opinionated way. What I can see for t-h-t to cover it on its own: * Custom entrypoints for containers adds complexity and head ache. * Mount logs world writable is a non starter. * Running containers as a user root is a security issue. What I can see for Kolla to accomplish that task: * the extended start does not cover logs and there are nuances with recursive chowning and permissions of new files [2]. * log directories doesn't fit well into the kolla_config's volumes and permissions defined in services' config.json. Using those for log volumes would look like a dirty hack. This also would undo the recent changes with using hostpath mounts instead of the kolla_config volumes. That's why I came to the option with 'user: root' and remapped user namespace for docker daemon. Please share your thoughts. [0] https://review.openstack.org/#/c/442603/ [1] https://goo.gl/HtDQYk [2] https://review.openstack.org/#/c/452742/ > >> pros: same way to template everything, kolla build/start just works. >> risks: non-openstack services, eventually, may stop being supported >> by >> Kolla for number of reasons. Kolla bootstrap changes aren't tested in >> tripleo CI and might be breaking >> cons: locking in to the kolla opinionated bootstrap entry points and >> kolla_config's config.json and command. >> >> * if applicable to the service, use upstream images (etcd example >> [2]) >> w/o any kolla parts. >> pros: less moving parts like custom entry points, no locking in >> onto opinionated Kolla config/bootstrap >> risks: Upstream image changes aren't tested in tripleo CI and might >> be >> breaking >> cons: different ways to template openstack/non-openstack services, >> kolla >> build/start doesn't work for the latter. >> >> [0] https://review.openstack.org/#/c/447627/ >> [1] https://review.openstack.org/#/c/451366 >> [2] >> https://review.openstack.org/#/c/445883/2/contrib/overcloud_container >> s.yaml >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Best regards, Bogdan Dobrelya, Irc #bogdando __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev