We have a large amount of logs (mostly apache, log4j, syslog) from various sources that are collected in our DMZ. We pull these inside to our internal network through an rsync-over-ssh process (security is a bit tight - traffic originating from the DMZ is not allowed inward and only SSH with no tunnels is permitted from internal to the DMZ). I can get the current log files brought inside Graylog using Collector against the current log files, but the problem is our existing data set. In order to be usable for our use case, I need our existing data (going back several years) brought inside with their original time stamps.
I *can* bring these messages in using the Raw/Plaintext and netcat, but the time stamps are from the moment of import, not the original time stamp. I have tried using various methods found on this group and in the documentation (logstash, nxlog, fluentd and even a custom written utility with the GELF Ruby Gem). None of these are successful bringing messages into the Graylog server. I can see the packets between the internal server with the log files and the Graylog server with tcpdump, but no messages are being brought into Graylog/Elasticsearch. I do not even see the incoming messages/second in the System>Inputs view. It almost seems like Graylog is somehow discarding these. Any thoughts on how to debug this? Has anyone done something similar before? I've seen some similar messages posted in this group without conclusive answers. Thanks in advance, -Eli -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/56aeb3be-b25c-4e0f-bf56-0aad2fc81a9e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
