Thanks. Those links are useful.
It looks to me like its a problem at the fluentd level. This is what I
see on on of the fluentd pods:
sh-4.2# cat /var/log/es-containers.log.pos
cat: /var/log/es-containers.log.pos: No such file or directory
sh-4.2# cat /var/log/journal.pos
s=52fdd277f90749b0a442c78739b1efa7;i=50d69;b=2a3f1736a1a1486d83f95db719fdc281;m=5465b53fd1;t=55cdac4738846;x=85596f3f5f5a27e4sh-4.2#
sh-4.2# journalctl -c `cat /var/log/journal.pos`
No journal files were found.
-- No entries --
Which might sort of explain why everything is running but no logs are
being processed.
This is based on a centos7 image with only the necessary openshift
packages installed and then openshift installed using ansible. The
logging setup in the inventory file is this:
openshift_hosted_logging_deployer_version=v3.6.0
openshift_hosted_logging_deploy=true
openshift_hosted_logging_storage_kind=nfs
openshift_hosted_logging_storage_access_modes=['ReadWriteOnce']
openshift_hosted_logging_storage_nfs_directory=/exports
openshift_hosted_logging_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_logging_storage_volume_name=logging
openshift_hosted_logging_storage_volume_size=10Gi
openshift_hosted_logging_storage_labels={'storage': 'logging'}
Tim
On 31/10/2017 16:37, Jeff Cantrill wrote:
Please provide additional information, logs, etc or post the output of
[1] someplace for review. Additionally, consider reviewing [2].
[1]
https://github.com/openshift/origin-aggregated-logging/blob/master/hack/logging-dump.sh
[2]
https://github.com/openshift/origin-aggregated-logging/blob/master/docs/checking-efk-health.md
On Tue, Oct 31, 2017 at 11:47 AM, Tim Dudgeon <tdudgeon...@gmail.com
<mailto:tdudgeon...@gmail.com>> wrote:
Hi All,
I've deployed logging using the ansible installer (v3.6.0) for a
fairly simple openshift setup and everything appears to running:
NAME READY STATUS RESTARTS AGE
logging-curator-1-gvh73 1/1 Running 24 3d
logging-es-data-master-xz0e7a0c-1-deploy 0/1 Error 0 3d
logging-es-data-master-xz0e7a0c-4-deploy 0/1 Error 0 3d
logging-es-data-master-xz0e7a0c-5-deploy 0/1 Error 0 3d
logging-es-data-master-xz0e7a0c-7-t4xpf 1/1 Running 0 3d
logging-fluentd-4rm2w 1/1 Running 0 3d
logging-fluentd-8h944 1/1 Running 0 3d
logging-fluentd-n00bn 1/1 Running 0 3d
logging-fluentd-vt8hh 1/1 Running 0 3d
logging-kibana-1-g7l4z 2/2 Running 0 3d
(the failed pods were related to getting elasticsearch running,
but that was resolved).
The problem is that I don't see any logs in Kibana. When I look in
the fluentd pod logs I see lots of stuff like this:
2017-10-31 13:53:15 +0000 [warn]: no patterns matched
tag="journal.system"
2017-10-31 13:58:02 +0000 [warn]: no patterns matched
tag="kubernetes.journal.container"
2017-10-31 14:02:18 +0000 [warn]: no patterns matched
tag="journal.system"
2017-10-31 14:07:15 +0000 [warn]: no patterns matched
tag="journal.system"
2017-10-31 14:11:20 +0000 [warn]: no patterns matched
tag="journal.system"
2017-10-31 14:15:16 +0000 [warn]: no patterns matched
tag="journal.system"
2017-10-31 14:19:58 +0000 [warn]: no patterns matched
tag="journal.system"
Is this the cause, and if so what is wrong?
If not how to debug this?
Tim
_______________________________________________
users mailing list
users@lists.openshift.redhat.com
<mailto:users@lists.openshift.redhat.com>
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
<http://lists.openshift.redhat.com/openshiftmm/listinfo/users>
--
--
Jeff Cantrill
Senior Software Engineer, Red Hat Engineering
OpenShift Integration Services
Red Hat, Inc.
*Office*: 703-748-4420 | 866-546-8970 ext. 8162420
jcant...@redhat.com <mailto:jcant...@redhat.com>
http://www.redhat.com
_______________________________________________
users mailing list
users@lists.openshift.redhat.com
http://lists.openshift.redhat.com/openshiftmm/listinfo/users
