Re: [Internet] [graylog2] Logstash to Graylog via GELF AMQP

2016-12-01 Thread cazy
Hi Mathieu,

I just tried it out. This works perfectly! Thank you!

Cheers,
Carlos

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/a4d38201-303a-4d82-842d-96f305fecd01%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [Internet] [graylog2] Logstash to Graylog via GELF AMQP

2016-12-01 Thread cazy
Hi Mathieu,

I just tried it out. This works perfectly! Thank you!

On Monday, November 28, 2016 at 3:33:28 PM UTC+1, Mathieu Grzybek wrote:
>
> Hi,
>
> You need to add some filters to create a GELF message :
>
> filter {
> mutate {
> join=> { "tags" => ", " }
> add_field=> { "version" => "1.1" }
> remove_field=> [ "@version", "@timestamp", "type" ]
> }
> }
>
> output {
> rabbitmq {
> host=> "{{ host_ip }}"
> port=> 5672
>
> user=> "{{ logstash_user }}"
> password=> "{{ logstash_pass }}"
>
> vhost=> "{{ logstash_vhost }}"
> exchange=> "{{ logstash_exchange }}"
> exchange_type=> topic
> durable=> true
>
> codec => json 
> }
> }
>
> Then, depending on your messages you can add some filtering rules or some 
> Graylog extractors.
>
> Mathieu
>
> On 11/28/16 13:27, cazy wrote:
>
> Hello everyone,
>
> we are trying to set up a log transport via GELF AMQP. At some stage of 
> the transport pipeline, there is Logstash, which should forward messages to 
> an AMQP broker such as RabbitMQ. Eventually, Graylog should receive 
> messages from the AMQP queues.
>
> "GELF AMQP" can be defined as Graylog input. However, there is only an 
> output plugin "rabbitmq" for Logstash; to my knowledge, it does not send 
> messages in the GELF format (like the GELF output plugin).
>
> So, is there a straightforward way to combine GELF with AMQP output? If 
> not, are there alternatives to achieve the same, e.g. with Kafka? Because 
> of architetural considerations, we need queueing at the log sources. And we 
> need GELF to retain meta information of log messages.
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to graylog2+u...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/graylog2/5ea90446-4f8b-4cc0-954b-a2983f4149b6%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> -- 
> Le capitaine Mathieu GRZYBEK
> COMSOPGN / STIG / BCQ
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/10e05b28-985e-43ce-af2b-bc5d3af532de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [Internet] [graylog2] Logstash to Graylog via GELF AMQP

2016-11-28 Thread Grzybek Mathieu CNE (BCQ STIG)

Hi,

You need to add some filters to create a GELF message :

filter {
mutate {
join=> { "tags" => ", " }
add_field=> { "version" => "1.1" }
remove_field=> [ "@version", "@timestamp", "type" ]
}
}

output {
rabbitmq {
host=> "{{ host_ip }}"
port=> 5672

user=> "{{ logstash_user }}"
password=> "{{ logstash_pass }}"

vhost=> "{{ logstash_vhost }}"
exchange=> "{{ logstash_exchange }}"
exchange_type=> topic
durable=> true

codec => json
}
}


Then, depending on your messages you can add some filtering rules or 
some Graylog extractors.


Mathieu

On 11/28/16 13:27, cazy wrote:

Hello everyone,

we are trying to set up a log transport via GELF AMQP. At some stage 
of the transport pipeline, there is Logstash, which should forward 
messages to an AMQP broker such as RabbitMQ. Eventually, Graylog 
should receive messages from the AMQP queues.


"GELF AMQP" can be defined as Graylog input. However, there is only an 
output plugin "rabbitmq" for Logstash; to my knowledge, it does not 
send messages in the GELF format (like the GELF output plugin).


So, is there a straightforward way to combine GELF with AMQP output? 
If not, are there alternatives to achieve the same, e.g. with Kafka? 
Because of architetural considerations, we need queueing at the log 
sources. And we need GELF to retain meta information of log messages.

--
You received this message because you are subscribed to the Google 
Groups "Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to graylog2+unsubscr...@googlegroups.com 
.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/5ea90446-4f8b-4cc0-954b-a2983f4149b6%40googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
Le capitaine Mathieu GRZYBEK
COMSOPGN / STIG / BCQ

--
You received this message because you are subscribed to the Google Groups "Graylog 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/4f613bc5-e9fa-e853-21fa-094aa1e47f95%40gendarmerie.interieur.gouv.fr.
For more options, visit https://groups.google.com/d/optout.
Ce message électronique et tous les fichiers attachés qu'il contient sont 
confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils 
sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à 
son émetteur. La publication, l'usage, la distribution, l'impression ou la 
copie non autorisée de ce message et des attachements qu'il contient sont 
strictement interdits.

En cas d'urgence, composez le 17 ou le 112.
Afin de contribuer au respect de l'environnement, merci de n'imprimer cet 
e-mail qu'en cas de necessite.

This e-mail and any files transmitted with it are confidential and intended 
solely for the use of the individual to whom it is addressed. If you have 
received this email in error please send it back to the person that sent it to 
you. Unauthorized publication, use, dissemination, forwarding, printing or 
copying of this email and its associated attachments is strictly prohibited.

In case of emergency, dial number 17 or 112.
To contribute to the environmental protection, please print this e-mail only if 
necessary. 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/4f613bc5-e9fa-e853-21fa-094aa1e47f95%40gendarmerie.interieur.gouv.fr.
For more options, visit https://groups.google.com/d/optout.