Re: [graylog2] Filebeats collector only one output

2016-09-22 Thread Marius Sturm 
Currently the idea goes like this: if you need routing inside the collector
use nxlog if not use filebeat.
I fear when we start to work around collectors inabilities we end up in
hairy ball of processes. In the end the Sidecar is
a configuration helper and not a feature compensator. And there is still
the option to simply contribute to the Fielbeat project and implement a
multi output feature, or ask the Elastic guys for it.

On 22 September 2016 at 17:54, Steve Kuntz  wrote:

> Thanks Marius,
>
> That's unfortunate, are there no plans to have the collector-sidecar
> service to run separate instance of filebeat for each output get around the
> limitations of filebeat?
>
> On Thursday, September 22, 2016 at 11:45:09 AM UTC-4, Marius Sturm wrote:
>>
>> Hi Steve,
>> Filebeat currently doesn't support multiple outputs in the way NXlog is
>> doing it. There is a ticket for it here: https://github.com/Graylog2/co
>> llector-sidecar/issues/57
>> In the end we will allow only one output per beat instance and show some
>> message in the web interface.
>>
>> Cheers,
>> Marius
>>
>>
>> On 22 September 2016 at 17:12, Steve Kuntz  wrote:
>>
>>> Good Morning/Afternoon/Evening/Night,
>>>
>>> I'm probably missing or misunderstanding something or missing some
>>> documentation that says this is not supported but here goes...
>>>
>>> When I configure multiple outputs in my Beats Collector Sidecar
>>> configuration in the web interface only one seems to make it to the
>>> filebeats.xml configuration file. Each prospector below is configured to
>>> forward to a different beats output. I've tried using one collector
>>> configuration for both files and separating the configurations into to 2
>>> collector configurations but it seams that both prospectors always makes it
>>> to the yml by only one output.
>>>
>>> filebeat:
>>>   prospectors:
>>>   - document_type: log
>>> fields:
>>>   gl2_source_collector: f48c9289-cc7c-4f40-84b8-38ba2d30b04c
>>> ignore_older: 0
>>> input_type: log
>>> paths:
>>> - /dev/shm/varnishncsa.log
>>> scan_frequency: 10s
>>> tail_files: true
>>>   - document_type: log
>>> fields:
>>>   gl2_source_collector: f48c9289-cc7c-4f40-84b8-38ba2d30b04c
>>> ignore_older: 0
>>> input_type: log
>>> paths:
>>> - /var/log/nginx/*.log
>>> scan_frequency: 10s
>>> tail_files: true
>>> output:
>>>   logstash:
>>> hosts:
>>> - graylogdmz.storm.dmz:5045
>>>
>>>
>>> Thanks
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Graylog Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to graylog2+u...@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/graylog2/b220fbcc-62e7-491d-bb54-c25d181da7f8%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Developer
>>
>> Tel.: +49 (0)40 609 452 077
>> Fax.: +49 (0)40 609 452 078
>>
>> TORCH GmbH - A Graylog Company
>> Poolstraße 21
>> 20335 Hamburg
>> Germany
>>
>> https://www.graylog.com 
>>
>> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
>> Geschäftsführer: Lennart Koopmann (CEO)
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/graylog2/930fd9d5-6e87-420d-9f2e-05156d9630b7%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Developer

Tel.: +49 (0)40 609 452 077
Fax.: +49 (0)40 609 452 078

TORCH GmbH - A Graylog Company
Poolstraße 21
20335 Hamburg
Germany

https://www.graylog.com 

Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
Geschäftsführer: Lennart Koopmann (CEO)

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAMqbBbK%3DYrUkP18oPQ-%2BPtifPPKtyZbozEC3yTomFAFW4DLEYg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Filebeats collector only one output

2016-09-22 Thread Steve Kuntz 
Thanks Marius,

That's unfortunate, are there no plans to have the collector-sidecar 
service to run separate instance of filebeat for each output get around the 
limitations of filebeat?

On Thursday, September 22, 2016 at 11:45:09 AM UTC-4, Marius Sturm wrote:
>
> Hi Steve,
> Filebeat currently doesn't support multiple outputs in the way NXlog is 
> doing it. There is a ticket for it here: 
> https://github.com/Graylog2/collector-sidecar/issues/57
> In the end we will allow only one output per beat instance and show some 
> message in the web interface.
>
> Cheers,
> Marius
>  
>
> On 22 September 2016 at 17:12, Steve Kuntz  > wrote:
>
>> Good Morning/Afternoon/Evening/Night,
>>
>> I'm probably missing or misunderstanding something or missing some 
>> documentation that says this is not supported but here goes...
>>
>> When I configure multiple outputs in my Beats Collector Sidecar 
>> configuration in the web interface only one seems to make it to the 
>> filebeats.xml configuration file. Each prospector below is configured to 
>> forward to a different beats output. I've tried using one collector 
>> configuration for both files and separating the configurations into to 2 
>> collector configurations but it seams that both prospectors always makes it 
>> to the yml by only one output.
>>
>> filebeat:
>>   prospectors:
>>   - document_type: log
>> fields:
>>   gl2_source_collector: f48c9289-cc7c-4f40-84b8-38ba2d30b04c
>> ignore_older: 0
>> input_type: log
>> paths:
>> - /dev/shm/varnishncsa.log
>> scan_frequency: 10s
>> tail_files: true
>>   - document_type: log
>> fields:
>>   gl2_source_collector: f48c9289-cc7c-4f40-84b8-38ba2d30b04c
>> ignore_older: 0
>> input_type: log
>> paths:
>> - /var/log/nginx/*.log
>> scan_frequency: 10s
>> tail_files: true
>> output:
>>   logstash:
>> hosts:
>> - graylogdmz.storm.dmz:5045
>>
>>
>> Thanks
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Graylog Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to graylog2+u...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/graylog2/b220fbcc-62e7-491d-bb54-c25d181da7f8%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Developer
>
> Tel.: +49 (0)40 609 452 077
> Fax.: +49 (0)40 609 452 078
>
> TORCH GmbH - A Graylog Company
> Poolstraße 21
> 20335 Hamburg
> Germany
>
> https://www.graylog.com 
>
> Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175
> Geschäftsführer: Lennart Koopmann (CEO)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/930fd9d5-6e87-420d-9f2e-05156d9630b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.