[graylog2] Re: Enabling geolocation

[Aykisn]

The GeoIP resolver automatically creates the ip_geolocation field on all 
the fields that are ip addresses. You don't need to do it manually.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/0a317029-b6fa-444c-a226-94a227ff0b6a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Enabling geolocation

[d3pr3cat3d]

Hello, I am trying to get geolocation working. 

# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)

# yum -y install geoip

# geoipupdate
MD5 Digest of installed database is 4cc97d426fbd0af868ae339aa9093061
/usr/share/GeoIP/GeoLiteCountry.dat is up to date, no updates required
GeoIP Database up to date
MD5 Digest of installed database is ac8d4ff284c73fd1120fb7980f8811b4
/usr/share/GeoIP/GeoLiteCity.dat is up to date, no updates required
GeoIP Database up to date

# geoiplookup -f /usr/share/GeoIP/GeoLiteCity.dat google.com
GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94043, 37.419201, 
-122.057404, 807, 650

I have configured /usr/share/GeoIP/GeoLiteCity.dat as the database path and 
GeoIP Resolver as the last message processor to run. Is it correct that if 
I append “_geolocation” to a grok pattern that is an IP this should start 
working?

Grok pattern for extractor

%{CISCOFW302013_302014_302015_302016}

Grok pattern

CISCOFW302013_302014_302015_302016 %{CISCO_ACTION:action}(?: 
%{CISCO_DIRECTION:direction})? %{WORD:protocol} connection %{INT:connection_id} 
for %{DATA:src_interface}:%{IP:src_ip}/%{INT:src_port}( 
\(%{IP:src_mapped_ip_geolocation}/%{INT:src_mapped_port}\))?(\(%{DATA:src_fwuser}\))?
 to %{DATA:dst_interface}:%{IP:dst_ip}/%{INT:dst_port}( 
\(%{IP:dst_mapped_ip_geolocation}/%{INT:dst_mapped_port}\))?(\(%{DATA:dst_fwuser}\))?(
 duration %{TIME:duration} bytes %{INT:bytes})?(?: %{CISCO_REASON:reason})?( 
\(%{DATA:user}\))?

Test message:

ASA %ASA-6-302013: Built outbound TCP connection 304484017 for 
outside:8.8.8.8/443 (8.8.8.8/443) to inside:10.102.109.83/54496 (8.8.4.4/54496)

When I click world map for “src_mapped_ip_geolocation” I get the pop up 
error that says:

Could not load map information Map widget is only available for fields 
containing geo data.

Thanks
​

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/fa8292b4-b31a-420b-adaf-536f95dc774f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: re-index after the data mapping is changed with updated Extractor.

[Jochen Schalanda]

Hi Wayne,

On Thursday, 20 October 2016 18:13:21 UTC+2, Wayne wrote:
>
> That probably requires setup of additional Graylog server plus installing 
> logstach as log shipper?
>

No, you can read from the same Elasticsearch cluster and write into the 
same Graylog instance.
 

I can see two types of indexes in /var/lib/elasticsearch/graylog/nodes
>
> (1) graylog_x
>
> (2) logstash-.MM.dd
>
> What is the relationship between between these two types of indexes, and 
> if the configuration is set up to delete old indexes, which indexes will be 
> deleted?
>

The first one, graylog_*, is managed by Graylog, the latter is being 
created and written into by logstash (depending on the configuration).

Graylog doesn't have to do anything with the latter one and can't read from 
it.


Cheers,
Jochen 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/8f9ac4e3-e92c-4b6c-8f0c-d33635a0f51f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: re-index after the data mapping is changed with updated Extractor.

[Wayne]

Hi Jochen,

That probably requires setup of additional Graylog server plus installing 
logstach as log shipper? It seems to be a bit messy.

Another question:

I can see two types of indexes in /var/lib/elasticsearch/graylog/nodes

(1) graylog_x

(2) logstash-.MM.dd

What is the relationship between between these two types of indexes, and if 
the configuration is set up to delete old indexes, which indexes will be 
deleted?


Thanks,

Wayne

On Thursday, October 20, 2016 at 11:50:08 AM UTC-4, Jochen Schalanda wrote:
>
> Hi Wayne,
>
> On Thursday, 20 October 2016 16:49:23 UTC+2, Wayne wrote:
>>
>> I am interested to know if there is a way to re-index all the data once a 
>> mapping is updated?
>>
>
> Graylog doesn't support this out-of-the-box.
>
> If the solution is not available now, is it in the next release?
>>
>
> No. 
>
> On the other hand, is there anyway to do it manually? I understand that 
>> the ELK stack could do a re-index, but I am not sure if there is a way to 
>> do it similarly?
>>
>
> You can re-index messages using logstash (input from Elasticsearch, output 
> to Graylog).
>  
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/27868742-e4f6-47d6-aaa5-72e2b1c0ea79%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Does Graylog server save a copy of the original log messages before indexing the message

[Jochen Schalanda]

Hi Wayne,

On Thursday, 20 October 2016 16:43:46 UTC+2, Wayne wrote: 
>
> Is there a way to convert them back to original text messages?
>

That depends on the type of input. But it's safe to assume that it's not 
possible.


Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/f2fc602a-cca2-4209-b55d-fdf2800fc7d7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Cannot assign requested address

['Adrian O' via Graylog Users]

Thanks for your help!

Let me remind you that you can only bind inputs to IP addresses that have 
> been setup on the machine that's running Graylog. You also don't need to 
> create a new input for each client.


That's where the problem was. I changed the Input IP address to 
10.115.32.69 and it works now. Obviously I didn't really understand how 
graylog works. I thought every client needed its own input. Thanks for 
clearing that up!

Regards,
Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/695bc57b-ddfb-4b5c-ad91-6734c206eb9e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] re-index after the data mapping is changed with updated Extractor.

[Wayne]

Hi All,

I am interested to know if there is a way to re-index all the data once a 
mapping is updated?


I googled it and found out there was no out-of-box solution up to last 
year. I am wondering if Graylog2 has a solution now.


If the solution is not available now, is it in the next release?


On the other hand, is there anyway to do it manually? I understand that the 
ELK stack could do a re-index, but I am not sure if there is a way to do it 
similarly?


Thanks,

Wayne

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2388f6be-dc44-4ee6-9780-23eec26775ab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Does Graylog server save a copy of the original log messages before indexing the message

[Wayne]

Hi Jochen,

Just want to explore a bit further.

These messages are now in binary format, and it seems to be parsed already. 
Is there a way to convert them back to original text messages? or there is 
no way to convert it back to original text form?

I am asking the question on behalf of one of my colleague who was thinking 
about retrieving information from the consolidated data (log messages from 
multiple source).

Thanks,

Wayne

On Thursday, October 20, 2016 at 6:16:14 AM UTC-4, Jochen Schalanda wrote:
>
> Hi Wayne,
>
> On Wednesday, 19 October 2016 21:28:25 UTC+2, Wayne wrote:
>>
>> Let's say we send a query and search a couple of records, now we would 
>> like to retrieve the original text message. Does Graylog keep the original 
>> copy of the log message?
>>
>
> No, it doesn't.
>  
>
> In addition, the disk based journal seems to keep some data, but not 
>> completely visible. Are those the copy of the messages?
>>
>
> Basically yes. The disk journal contains the raw binary message received 
> by an input until a codec decodes the message and indexes it into 
> Elasticsearch.
>
> Cheers,
> Jochen 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e5e308d4-7fab-4952-9d19-859db6f4f1a7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Internal message queue for graylog2?

[Wayne]

I think it is sufficient for us to stick to the default configuration 
without external message queue.

Thanks,

Wayne

On Wednesday, October 19, 2016 at 9:14:11 AM UTC-4, Jochen Schalanda wrote:
>
> Hi Wayne,
>
> On Wednesday, 19 October 2016 15:07:07 UTC+2, Wayne wrote:
>>
>> It is stated in 2.1 document that Kafka and RabbitMQ can be configured as 
>> transport queue.
>>
>> What are the use cases/scenarios which we need to do the above 
>> configuration considering Graylog already has its own way to persist the 
>> messages?
>>
>
> It can be useful for connecting offsite locations with bad network 
> connection or if log messages aren't exclusively consumed by Graylog.
>
> If you can't come up with a use case for using a message broker like 
> RabbitMQ or Apache Kafka, it's probably not necessary for you…
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e8bdc639-f8ba-4d1d-a034-78c7bdd150c8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Cannot assign requested address

[Jochen Schalanda]

Hi Adrian,

are you 100% sure that 10.115.32.40 is the correct IP address?
Is it available on any network interface on the machine running Graylog?
Are you using any security extension for the Linux kernel such as SELinux, 
grsecurity, or AppArmor?
What's the output of `*ip addr*` on that machine?

Let me remind you that you can only bind inputs to IP addresses that have 
been setup on the machine that's running Graylog. You also don't need to 
create a new input for each client.

FWIW, it looks to me like you're mixing up IP addresses.

Cheers,
Jochen

On Thursday, 20 October 2016 15:24:14 UTC+2, Adrian O wrote:
>
> Hi everyone!
>
> We keep getting this error every time we try to start syslog input. This 
> error has been posted many times and we tried every solution but it still 
> won't work. Local inputs work fine but when trying to add an input from 
> another host it fails instantly. Running *sudo graylog-ctl tail *and 
> starting an input it says:
>
>
> ==> /var/log/graylog/server/current <==
>> 2016-10-20_09:32:59.03946 2016-10-20 09:32:59,038 INFO : 
>> org.graylog2.inputs.InputStateListener - Input [Syslog 
>> UDP/57ebad86255aef123d9faea0] is now STOPPING
>> 2016-10-20_09:32:59.04252 2016-10-20 09:32:59,041 ERROR: 
>> org.graylog2.plugin.inputs.transports.NettyTransport - Error in Input 
>> [Syslog UDP/57ebad86255aef123d9faea0] (channel [id: 0x66325e61])
>> 2016-10-20_09:32:59.04254 java.net.BindException: Cannot assign requested 
>> address
>> 2016-10-20_09:32:59.04332   at sun.nio.ch.Net.bind0(Native Method) 
>> ~[?:1.8.0_101]
>> 2016-10-20_09:32:59.04423   at sun.nio.ch.Net.bind(Net.java:433) 
>> ~[?:1.8.0_101]
>> 2016-10-20_09:32:59.04496   at 
>> sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) 
>> ~[?:1.8.0_101]
>> 2016-10-20_09:32:59.04572   at 
>> sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) 
>> ~[?:1.8.0_101]
>> 2016-10-20_09:32:59.04626   at 
>> org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.bind(NioDatagramPipelineSink.java:129)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.04772   at 
>> org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.eventSunk(NioDatagramPipelineSink.java:77)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.04847   at 
>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.05028   at 
>> org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.05091   at 
>> org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.05227   at 
>> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.05512   at 
>> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.05657   at 
>> org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.05724   at 
>> org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.06170   at 
>> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.06240   at 
>> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.06383   at 
>> org.jboss.netty.channel.Channels.bind(Channels.java:561) [graylog.jar:?]
>> 2016-10-20_09:32:59.06441   at 
>> org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:197) 
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.06546   at 
>> org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:198)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.06776   at 
>> org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.07111   at 
>> org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) 
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.07237   at 
>> org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) 
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.07297   at 
>> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
>>  
>> [graylog.jar:?]
>> 2016-10-20_09:32:59.07543   at 
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
>> [?:1.8.0_101]
>> 2016-10-20_09:32:59.07595   at 
>> java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_101]
>> 

[graylog2] Re: "Strange" GrayLog server logs

[Jamie P]

Hi Jochen,

I had not seen them before so I was wondering if something was going on 
that I needed to possibly take care of or if they were just informational. 
 They appeared to be just informational, but I was erring on the side of 
caution and wanted to make sure that they weren't indicative of a system 
issue or something else that needed attention/action on.

Jamie

On Thursday, October 20, 2016 at 9:05:07 AM UTC-4, Jochen Schalanda wrote:
>
> Hi Jamie,
>
> what exactly do you think is strange about these log messages? They're 
> purely informal and you can disable "internal logging" in the Graylog OVA 
> as described at 
> http://docs.graylog.org/en/2.1/pages/configuration/graylog_ctl.html#configuration-commands
> .
>
> Cheers,
> Jochen
>
> On Thursday, 20 October 2016 13:51:38 UTC+2, Jamie P wrote:
>>
>> When looking at my logs I came across some messages that I havent seen 
>> before that were generated by the Graylog server itself.  The numbers after 
>> "factory:" were all different but the messages stayed the same.  I was 
>> curious if this is something to be concerned about?  Here's a copy of one: 
>>   
>>
>> acility
>> runit-service
>> from_gelf
>> true
>> level
>> 6
>> message
>> 2016-10-20 11:45:33,931 INFO : 
>> org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - 
>> Registered pre-bundled extended operation factory: 1.3.6.1.4.1.4203.1.11.3
>> source
>> graylog-server
>> timestamp
>> 2016-10-20T11:45:33.931Z
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/0ea4b79e-dc45-431d-9e0f-238a2177d882%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Cannot assign requested address

['Adrian O' via Graylog Users]

Hi everyone!

We keep getting this error every time we try to start syslog input. This 
error has been posted many times and we tried every solution but it still 
won't work. Local inputs work fine but when trying to add an input from 
another host it fails instantly. Running *sudo graylog-ctl tail *and 
starting an input it says:


==> /var/log/graylog/server/current <==
> 2016-10-20_09:32:59.03946 2016-10-20 09:32:59,038 INFO : 
> org.graylog2.inputs.InputStateListener - Input [Syslog 
> UDP/57ebad86255aef123d9faea0] is now STOPPING
> 2016-10-20_09:32:59.04252 2016-10-20 09:32:59,041 ERROR: 
> org.graylog2.plugin.inputs.transports.NettyTransport - Error in Input 
> [Syslog UDP/57ebad86255aef123d9faea0] (channel [id: 0x66325e61])
> 2016-10-20_09:32:59.04254 java.net.BindException: Cannot assign requested 
> address
> 2016-10-20_09:32:59.04332   at sun.nio.ch.Net.bind0(Native Method) 
> ~[?:1.8.0_101]
> 2016-10-20_09:32:59.04423   at sun.nio.ch.Net.bind(Net.java:433) 
> ~[?:1.8.0_101]
> 2016-10-20_09:32:59.04496   at 
> sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:691) 
> ~[?:1.8.0_101]
> 2016-10-20_09:32:59.04572   at 
> sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91) 
> ~[?:1.8.0_101]
> 2016-10-20_09:32:59.04626   at 
> org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.bind(NioDatagramPipelineSink.java:129)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.04772   at 
> org.jboss.netty.channel.socket.nio.NioDatagramPipelineSink.eventSunk(NioDatagramPipelineSink.java:77)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.04847   at 
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.05028   at 
> org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.05091   at 
> org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.05227   at 
> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.05512   at 
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.05657   at 
> org.jboss.netty.channel.SimpleChannelHandler.bindRequested(SimpleChannelHandler.java:299)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.05724   at 
> org.jboss.netty.channel.SimpleChannelHandler.handleDownstream(SimpleChannelHandler.java:265)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.06170   at 
> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.06240   at 
> org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.06383   at 
> org.jboss.netty.channel.Channels.bind(Channels.java:561) [graylog.jar:?]
> 2016-10-20_09:32:59.06441   at 
> org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:197) 
> [graylog.jar:?]
> 2016-10-20_09:32:59.06546   at 
> org.jboss.netty.bootstrap.ConnectionlessBootstrap.bind(ConnectionlessBootstrap.java:198)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.06776   at 
> org.graylog2.plugin.inputs.transports.NettyTransport.launch(NettyTransport.java:136)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.07111   at 
> org.graylog2.plugin.inputs.MessageInput.launch(MessageInput.java:153) 
> [graylog.jar:?]
> 2016-10-20_09:32:59.07237   at 
> org.graylog2.shared.inputs.InputLauncher$1.run(InputLauncher.java:84) 
> [graylog.jar:?]
> 2016-10-20_09:32:59.07297   at 
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
>  
> [graylog.jar:?]
> 2016-10-20_09:32:59.07543   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [?:1.8.0_101]
> 2016-10-20_09:32:59.07595   at 
> java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_101]
> 2016-10-20_09:32:59.07680   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  
> [?:1.8.0_101]
> 2016-10-20_09:32:59.08331   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  
> [?:1.8.0_101]
> 2016-10-20_09:32:59.08387   at java.lang.Thread.run(Thread.java:745) 
> [?:1.8.0_101]
>
> *2016-10-20_09:32:59.08449 2016-10-20 09:32:59,042 ERROR: 
> org.graylog2.shared.inputs.InputLauncher - The 
> [org.graylog2.inputs.syslog.udp.SyslogUDPInput] input with ID 
> <57ebad86255aef123d9faea0> misfired. Reason: Cannot assign requested 
> address.2016-10-20_09:32:59.08725 
> org.graylog2.plugin.inputs.MisfireException: 
> 

[graylog2] Re: "Strange" GrayLog server logs

[Jochen Schalanda]

Hi Jamie,

what exactly do you think is strange about these log messages? They're 
purely informal and you can disable "internal logging" in the Graylog OVA 
as described at 
http://docs.graylog.org/en/2.1/pages/configuration/graylog_ctl.html#configuration-commands
.

Cheers,
Jochen

On Thursday, 20 October 2016 13:51:38 UTC+2, Jamie P wrote:
>
> When looking at my logs I came across some messages that I havent seen 
> before that were generated by the Graylog server itself.  The numbers after 
> "factory:" were all different but the messages stayed the same.  I was 
> curious if this is something to be concerned about?  Here's a copy of one: 
>   
>
> acility
> runit-service
> from_gelf
> true
> level
> 6
> message
> 2016-10-20 11:45:33,931 INFO : 
> org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - 
> Registered pre-bundled extended operation factory: 1.3.6.1.4.1.4203.1.11.3
> source
> graylog-server
> timestamp
> 2016-10-20T11:45:33.931Z
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/7693342a-a74e-4646-b910-5f4508573682%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Redirect logs from /varlog/messages to a Graylog server

[Benbrahim Anass]

problem Solved, of course it's SElinux
Thank you NSA, FUCK!
Jochen cheers dude

Le jeudi 20 octobre 2016 12:09:04 UTC+2, Jochen Schalanda a écrit :
>
> Hi,
>
> On Thursday, 20 October 2016 11:55:56 UTC+2, Benbrahim Anass wrote:
>>
>> and when i try to configure the input on graylog with the port 514 it 
>> says that i dont have root permission . i dont want to start with root or 
>> chmod graylog, i just want to configure another port but it's not working
>>
>
> While that sounds like a firewall issue in general, you can simply use 
> authbind  (see 
> https://debian-administration.org/article/386/Running_network_services_as_a_non-root_user.
>  
> and 
> https://github.com/Graylog2/fpm-recipes/blob/2.0/recipes/graylog-server/files/environment#L10-L12)
>  
> to allow the Java process running Graylog to bind to a privileged port.
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/dfeb715e-90a4-4aab-9d54-408faca70045%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: error on sudo graylog-ctl reconfigure after upgrade

[Nathaniel Brassington]

HI I did tell you what I did Step by step..

Hi Jochen, 

I did the following on each of the 3 servers.

I have one front end server that runs graylog and then two eleasticsearch 
hosts

sudo apt-get update
sudo apt-get upgrade

then the following from the guide 
http://docs.graylog.org/en/latest/pages/configuration/graylog_ctl.html#upgrade-graylog-omnibus

wget 
https://packages.graylog2.org/releases/graylog-omnibus/ubuntu/graylog_latest.deb
sudo graylog-ctl stop
sudo dpkg -G -i graylog_latest.deb
sudo graylog-ctl reconfigure
sudo reboot




On Wednesday, 19 October 2016 09:59:18 UTC+1, Jochen Schalanda wrote:
>
> Hi Nathaniel,
>
> On Wednesday, 19 October 2016 10:54:37 UTC+2, Nathaniel Brassington wrote:
>>
>> Just an update. I restored the server from backup and reapplied the 
>> update without doing a sudo apt-get update and upgrade. The process failed 
>> at the same point. Service still down.
>>
>
> We cannot help you without knowing what you did exactly (step-by-step) and 
> how you've installed Graylog in the first place.
>
> Cheers,
> Jochen 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e84c31d9-e381-49b8-89c6-c693b766ed63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] "Strange" GrayLog server logs

[Jamie P]

When looking at my logs I came across some messages that I havent seen 
before that were generated by the Graylog server itself.  The numbers after 
"factory:" were all different but the messages stayed the same.  I was 
curious if this is something to be concerned about?  Here's a copy of one: 
  

acility
runit-service
from_gelf
true
level
6
message
2016-10-20 11:45:33,931 INFO : 
org.apache.directory.api.ldap.codec.standalone.CodecFactoryUtil - 
Registered pre-bundled extended operation factory: 1.3.6.1.4.1.4203.1.11.3
source
graylog-server
timestamp
2016-10-20T11:45:33.931Z

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/38421b99-d9c5-4596-a21f-2f040d35310c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Error: There was no master Graylog server node detected in the cluster.

[Ľubo]

Hi all,

We have two nodes in Graylog cluster 2.1.0.

There is Error: There was no master Graylog server node detected in the 
cluster.

This error is every 10 seconds, there is No notification (green) and then 
this error - see system messages.

 

"Certain operations of Graylog server require the presence of a master 
node, but no such master was started. Please ensure that one of your 
Graylog server nodes contains the setting is_master = true in its 
configuration and that it is running. Until this is resolved index cycling 
will not be able to run, which means that the index retention mechanism is 
also not running, leading to increased index sizes. Certain maintenance 
functions as well as a variety of web interface pages (e.g. Dashboards) are 
unavailable.

"

Could you give me advice how to solve this error?

 

discovery.zen.minimum_master_nodes: 2

gateway.recover_after_nodes: 1

Elasticsearch sluster - green

 

NTP enabled: yes

NTP synchronized: yes

cat /etc/graylog/server/server.conf

One node is is_master = true, second node is is_master = false

 

 

System messages:

TimestampNodeMessage

2016-10-20T12:21:27+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:21:13+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:21:10+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:21:00+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:47+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:40+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:36+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:33+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:23+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:19+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:14+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

2016-10-20T12:20:10+02:00 7e58c1ec / 
hostname001Notification condition [NO_MASTER] has been fixed.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/0c4a5b8f-db80-44a0-acce-0fbe83f0bd08%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Does Graylog server save a copy of the original log messages before indexing the message

[Jochen Schalanda]

Hi Wayne,

On Wednesday, 19 October 2016 21:28:25 UTC+2, Wayne wrote:
>
> Let's say we send a query and search a couple of records, now we would 
> like to retrieve the original text message. Does Graylog keep the original 
> copy of the log message?
>

No, it doesn't.
 

In addition, the disk based journal seems to keep some data, but not 
> completely visible. Are those the copy of the messages?
>

Basically yes. The disk journal contains the raw binary message received by 
an input until a codec decodes the message and indexes it into 
Elasticsearch.

Cheers,
Jochen 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/14df9496-d843-4a67-a0f5-9e597799b2cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Some fields generated from Extractor are not searchable

[Jochen Schalanda]

Hi Wayne,

On Wednesday, 19 October 2016 18:34:20 UTC+2, Wayne wrote:
>
> What is strange about it is that the "Stream" rules apparently work with 
> the field "log_message", but a search query does not work. 
>

What does that mean exactly? Do you have some examples? 


The custom mapping is useful if the data type is not the default string 
> type. However, the log_message field is still string type. So it may not 
> make much difference if I set up custom mapping for this field?
>

No, you can also make Elasticsearch analyze fields which wouldn't be 
analyzed otherwise. By default, only the message, full_message, and source 
fields are being analyzed.

See 
https://www.elastic.co/guide/en/elasticsearch/reference/2.4/analysis-analyzers.html
 
for details.


Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ea7ed744-fdfd-4013-8286-b7cae02ce6bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Redirect logs from /varlog/messages to a Graylog server

[Jochen Schalanda]

Hi,

On Thursday, 20 October 2016 11:55:56 UTC+2, Benbrahim Anass wrote:
>
> and when i try to configure the input on graylog with the port 514 it says 
> that i dont have root permission . i dont want to start with root or chmod 
> graylog, i just want to configure another port but it's not working
>

While that sounds like a firewall issue in general, you can simply use 
authbind  (see 
https://debian-administration.org/article/386/Running_network_services_as_a_non-root_user.
 
and 
https://github.com/Graylog2/fpm-recipes/blob/2.0/recipes/graylog-server/files/environment#L10-L12)
 
to allow the Java process running Graylog to bind to a privileged port.

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/c2c48dc8-d673-4b9f-a295-59b2b61b3615%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Redirect logs from /varlog/messages to a Graylog server

[Benbrahim Anass]

i've tested the log forwarding from the syslog server to another server and 
it's only working on port 514, when i trie with any other port it doesnt 
pass. i've already configured the two servers the right way and opened 
ports on the firewall, i've even disabled it but nothing working. and when 
i try to configure the input on graylog with the port 514 it says that i 
dont have root permission . i dont want to start with root or chmod 
graylog, i just want to configure another port but it's not working
the exemple you gave me is so basic i already know how to work with 
syslogs, i may missconfigured something, i'll keep searching but if you 
have any suggestions of where the probleme could be comming from please 
tell me
thank you so much
cheers

Le jeudi 20 octobre 2016 10:29:40 UTC+2, Jochen Schalanda a écrit :
>
> Hi Benbrahim,
>
> On Thursday, 20 October 2016 10:05:32 UTC+2, Benbrahim Anass wrote:
>>
>> Graylog is not recieving anything, i did exactly as the guide . maybe i 
>> did something wrong when adding syslog tcp inpute. can you give me an 
>> exemple please
>>
>
> Examples are already given in the guide I've linked to.
>
> How exactly did you configure your syslog daemon and how did you configure 
> the Syslog UDP or TCP input in Graylog?
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/f3457d8c-9c40-428a-92bb-e14171f485eb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Redirect logs from /varlog/messages to a Graylog server

[Jochen Schalanda]

Hi Benbrahim,

On Thursday, 20 October 2016 10:05:32 UTC+2, Benbrahim Anass wrote:
>
> Graylog is not recieving anything, i did exactly as the guide . maybe i 
> did something wrong when adding syslog tcp inpute. can you give me an 
> exemple please
>

Examples are already given in the guide I've linked to.

How exactly did you configure your syslog daemon and how did you configure 
the Syslog UDP or TCP input in Graylog?

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/b74f7d66-b3b3-431b-a2a4-ab51df227dd4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Redirect logs from /varlog/messages to a Graylog server

[Benbrahim Anass]

Graylog is not recieving anything, i did exactly as the guide . maybe i did 
something wrong when adding syslog tcp inpute. can you give me an exemple 
please
Thank you Dude.


Le mercredi 19 octobre 2016 16:24:52 UTC+2, Jochen Schalanda a écrit :
>
> Hi Benbrahim,
>
> see https://github.com/Graylog2/graylog-guide-syslog-linux#readme for 
> instructions how to configure rsyslog or syslog-ng to forward logs to 
> Graylog.
>
> Cheers,
> Jochen
>
> On Wednesday, 19 October 2016 15:30:06 UTC+2, Benbrahim Anass wrote:
>>
>> Hello
>> i'm wondering if it is possible to redirect all /var/log/message of a 
>> syslog server to a distant graylog server
>> Thanks alot
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/bb63719b-5edd-47fd-8c7e-0a20f8198ba8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.