Hi Sergey, We are using Kolla Ansible with monasca log API, and have added support for customising the fluentd configuration [1][2]. Doug Szumski (dougsz) made some changes in Queens to try to standardise the log message format. I think Kolla Ansible would benefit from some better documentation on what this format is.
On the monasca log API side there is support for transforming logs using logstash. [1] https://docs.openstack.org/kolla-ansible/latest/reference/central-logging-guide.html#custom-log-forwarding [2] https://docs.openstack.org/kolla-ansible/latest/reference/central-logging-guide.html#custom-log-filtering Cheers, Mark On 14 July 2018 at 14:29, Sergey Glazyrin <[email protected]> wrote: > Hello guys! > We are migrating our product to kolla-ansible and as far as probably you > know, it uses fluentd to control logs, etc. In non containerized openstack > we use rsyslog to send data to logstash. We get data from syslog events. It > looks like it's impossible to use syslog in kolla-ansible. Unfortunately > external_syslog_server option doesn't work. Is there anyone who was able to > use it ? But, nevermind, we may use fluentd BUT.. we have one problem - > different data format for each service/container. > > So, probably the most optimal solution is to use default logging idea in > kolla-ansible. (to be honest, I am not sure... but I've no found better > option). But even with default logging idea in kolla - ansible we have one > serious problem. Fluentd has different data format for each service, for > instance, you may see this link with explanation how its designed in > kolla-ansible > https://github.com/openstack/kolla-ansible/commit/ > 3026cef7cfd1828a27e565d4211692f0ab0ce22e > there are grok patterns which parses log messages, etc > > so, we managed to put data to elasticsearch but we need to solve two > problems: > 1. unify data format for log events. We may solve it using logstash to > unify it before putting it to elasticsearch (or should we change fluentd > configs in our own version of kolla-ansible repository ? ) > For instance, we may do it using this logstash plugin > https://www.elastic.co/guide/en/logstash/2.4/plugins- > filters-mutate.html#plugins-filters-mutate-rename > > What's your suggestion ? > > > -- > Best, Sergey > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
