Hey Pete,
What is it you need to solve exactly?
If you have an unreliable link currently the best (reliable) solution is
a message broker like Kafka or AMQP. The former is a good choice for
high load but less flexibility, while AMQP is better if you need
flexible routing etc.
Logstash/NXLog allow to read from files, which Graylog doesn't support
yet, and pass the data on.
We have some pointers on the docs.graylog.org site to get started,
please let us know if we are missing details or entire topics. We'll
happily add more content to make it easier to get started.
Radio is deprecated mostly because there are problems around performance
and management. We will most likely move towards a solution that allows
Graylog servers to forward data, because of the flexibility this
provides. You can safely deploy it in the sense that we will provide
detailed migration steps before we stop supporting it, we just want to
discourage its use for purely performance reasons.
Load balancing TCP traffic across Graylog servers will yield _much_
better performance than radio has ever provided. In fact most
deployments we support have benefited greatly from moving to 1.0 and
replacing radio with mulitple servers.
We regularly see sustained >100k msg/sec setups with decent hardware
nowadays, something that message queue setups can also deliver but they
usually mean substantially increased complexity.
Disconnected/high latency deployments still benefit from message broker
setups and as such are a viable solution currently.
Feel free to contact me directly if you have more detailed questions.
Best regards,
Kay
On 21/03/15 23:31, Pete GS wrote:
I see what you're saying there... however is that a little too
complicated adding Logstash into the equation?
I was thinking Servers -> Kafka -> Graylog
Is that possible or am I missing something with Kafka that would
require something like Logstash?
Right now we're struggling to see the need to even add a second step
between the source servers and Graylog but we are being asked to cater
for this by other people in the business, so I need the simplest and
least effort solution.
Cheers, Pete
On Friday, 20 March 2015 18:12:32 UTC+10, Mathieu Grzybek wrote:
Hi,
To collect your logs from the DMZ, we use some RabbitMQ brokers :
servers === (syslog over UDP or TCP) ===> Logstash === (AMQP) ===>
RabbitMQ === (AMQP) ===> Logstash === (GELF) ===> Graylog
Mathieu
Le vendredi 20 mars 2015 05:51:11 UTC+1, Pete GS a écrit :
Hi all,
We're looking at adding message sources into our Graylog setup
from a couple of remote sites. There is the possibility of
temporary transit link outages so sending UDP packets would
result in lost messages. Using TCP will counter this to a
certain extent but may result in messages queuing on sources
running NXLog etc. which we would like to avoid.
I'm think it would be better to set up a message broker at the
remote site and having Graylog subscribe to this to retrieve
the messages instead once the link is available again.
Does this sound like a sensible idea or is there some better
way to achieve this?
Any recommendations on a preferred message broker etc.?
Any clues on how to get NXLog, Syslog, etc. to log to a
message broker instead of to Graylog directly?
Cheers, Pete
--
You received this message because you are subscribed to the Google
Groups "graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"graylog2" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
