On Tue, Jul 11, 2017 at 9:00 AM, Alex Wauck <[email protected]> wrote: > Last I checked (OpenShift Origin 1.2), fluentd was just slurping up the log > files produced by Docker. It can do that because the pods it runs in have > access to the host filesystem. > > On Tue, Jul 11, 2017 at 6:12 AM, Stéphane Klein > <[email protected]> wrote: >> >> Hi, >> >> I see here >> https://github.com/openshift/origin-aggregated-logging/blob/master/fluentd/configs.d/input-post-forward-mux.conf#L2 >> that fluentd logging system use secure_forward input system. >> >> My question: what component forward log entries to fluentd input service ?
The "mux" service is a concentrator of sorts. Without the mux service, each fluentd pod runs on a host in an OpenShift cluster collecting logs and sending them to Elasticsearch directly. The collectors also have the responsibility of enhancing the logs collected with the metadata that describes which pod/container they came from. This requires connections to the API server to get that information. So in a large cluster, 200+ nodes, maybe less, maybe more, the API servers are overwhelmed by requests from all the fluentd pods. With the mux service, all the fluentd collections pods only talk to the mux service and DO NOT talk to the API server; they simply send the logs they collect to the mux fluentd instance. The mux fluentd instance in turns talks to the API service to enrich the logs with the pod/container metadata and then send along to Elasticsearch. This scales much better. -peter >> >> Best regards, >> Stéphane >> -- >> Stéphane Klein <[email protected]> >> blog: http://stephane-klein.info >> cv : http://cv.stephane-klein.info >> Twitter: http://twitter.com/klein_stephane >> >> _______________________________________________ >> users mailing list >> [email protected] >> http://lists.openshift.redhat.com/openshiftmm/listinfo/users >> > > > > -- > > Alex Wauck // Senior DevOps Engineer > > E X O S I T E > www.exosite.com > > Making Machines More Human. > > > _______________________________________________ > users mailing list > [email protected] > http://lists.openshift.redhat.com/openshiftmm/listinfo/users > _______________________________________________ users mailing list [email protected] http://lists.openshift.redhat.com/openshiftmm/listinfo/users
