[graylog2] remote sites / servers

[Damien Hull]

I'm bran new to graylog. I have it installed on Ubuntu 14.04 for testing. 
Here's what I would like to do. 

We have 5 remote offices. 

   1. Is there a secure way to send logs from these remote offices?
   2. Is there a proxy I can use to collect the data onsite and send it to 
   the main server every 10 minutes or so? 

Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/eaaf8676-4eb8-4f04-af7a-ac4d94f7a354%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Load Balancer health check with Big-IP F5

[Marty]

Hi Folks,

Graylog V1.3.4

Just wondering if anyone has integrated the Graylog LB state into the F5 
native http health check.
I can't get this to work when sending:

GET /system/lbstatus HTTP/1.1


>From the command line (using netcat) on the graylog node, this also fails. 
Just get a newline (no output).

$ echo -e "GET /system/lbstatus HTTP/1.1\r\n" | nc 127.0.0.1 12900

Using nc natively is OK, as seen below. Need to send  twice, as shown.

$ nc 127.0.0.1 12900
GET /system/lbstatus HTTP/1.1

HTTP/1.1 200 OK
Content-Type: text/plain
X-Graylog-Node-ID: ----x
X-Runtime-Microseconds: 240 
Transfer-Encoding: chunked 
 
5
ALIVE 
0 


Using curl is fine:

S curl -w '\n' http://127.0.0.1:12900/system/lbstatus
ALIVE

I got around this on the F5, by using curl with an external script.

Just wondering if there is an issue or I'm doing something incorrect.

Cheers,
Martin

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/05364332-c760-472e-9e67-4eb4d2db5205%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: installed marvel - now seemed to have corrupted entire graylog db

[Jason Haar]

Hi Jochen

There wasn't any error that pointed at this explicitly. This is
graylog-1.3.4 with ES-1.7 on CentOS7. There has been a general degradation
as my (single) test box grew to 5TB in size - I think I'm simply tickling
all the edge cases of performance issues and this is the consequence. I do
think ES could do with some major improvements - I continually saw graylog
reporting it couldn't push data into ES and yet ES reported no problems and
showed "green". And yet if I restarted ES (ie stop/start) it immediately
came up as "red" with 10,000+ UNASSIGNED shards - so there's a fairly
serious bug in there IMHO.

In any case I ended up deleting all the indexes that wouldn't move off
"UNASSIGNED" and basically ended up deleting most of my data, so I've
thrown the entire thing away (hey, it is a test box :-) and restarted with
the Beta graylog-2 with ES-2.3.1 and we'll see how that goes. This test box
is almost at an end - new hardware has finally arrived - so I think the
root cause of the issues will soon be resolved by actually having a
cluster/etc.

Thanks

Jason

On Wed, Apr 13, 2016 at 12:55 AM, Jochen Schalanda 
wrote:

> Hi Jason,
>
> what's your Graylog and Elasticsearch configuration? Are there any error
> messages in the logs of either Graylog or Elasticsearch?
>
> Cheers,
> Jochen
>
>
> On Monday, 11 April 2016 07:15:17 UTC+2, Jason Haar wrote:
>>
>> Hi there
>>
>> Over the weekend I installed the ES marvel diagnostics package and the
>> following day noticed that graylog was broken. Restarting ES showed 20,000
>> shards in an UNASSIGNED state. I disabled graylog-server (so there was no
>> new data flowing in) and watched over the next couple of hours as that
>> UNASSIGNED number dropped down to 0 (and GREEN). I then restarted ES and -
>> bam - back to 20,000 UNASSIGNED shards again
>>
>> I've now done three iterations of that - looks like it's completely
>> borked. There's over 5TB of data in there - how can I regain it?
>>
>> PS: I don't know if this has anything to do with marvel - it's just the
>> last change I made. The reason I installed it was because I have had ES
>> continually doing this kind of thing - but previously stopping graylog,
>> restarting ES and waiting would lead to a happy ES - but no longer.
>>
>> --
>> Cheers
>>
>> Jason Haar
>> Information Security Manager, Trimble Navigation Ltd.
>> Phone: +1 408 481 8171
>> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Graylog Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/graylog2/507dbfa3-0239-45ca-9799-2cae7db65418%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Cheers

Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +1 408 481 8171
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAFChrg%2BL5zOhKu%2BHLJVtQjpZyjSQTe8HtsmS_%3DDJyBU%2B44UpTg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Web Interface Certificate differences from v1 and v2

[Bernie Carolan]

I used this format to convert existing keys, seems to work ok.

openssl pkcs8 -nocrypt -topk8 -in /etc/pki/tls/private/graylog-server.key 
-out /etc/pki/tls/private/graylog-server.pk8

On Wednesday, April 13, 2016 at 4:13:15 AM UTC+10, Drew Miranda wrote:
>
> Any quick tips on the command to use with openssl to output the correct 
> format? I found enough documentation to interchange formats but an unclear 
> on the exact switches.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/48785793-d445-46a1-89e7-e0c5f124d30e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Web Interface Certificate differences from v1 and v2

[Drew Miranda]

Any quick tips on the command to use with openssl to output the correct format? 
I found enough documentation to interchange formats but an unclear on the exact 
switches.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/55289179-a870-4ee6-b5dd-cf0fc1851ec0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Graylog email alert frequency

[David Rux]

Hey all,

I have a stream that's set to send an email whenever an alert is triggered 
that matches a channel. The email is received and all is well but graylog 
seems to group a series of events together before sending the email. Is 
there any way to change this? Basically I want an email whenever an event 
matching the criteria hits that stream. One email per event. Does anyone 
know if that's possible? My alert condition is as follows:

Trigger alert when a message arrives that has the field 

 
set to  and 
then wait at least  minutes until triggering a new alert. (grace period) 
When sending an alert, include the last  messages of the stream evaluated 
for this alert condition.

I would have thought that a 0 minute grace period would do this but I 
tested it and graylog lumped 4 backlog messages into the email where I 
wanted 4 emails with one event in each. When I set the number of included 
messages to 1, I only get one email with one alert and it seems to ignore 
the other events that I triggered despite being logged on the dashboard.

Thanks,

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/b08b3efa-886a-4fe9-922f-5938c655265d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Can I convert a field from string to integer?

[Ryan Anstey]

I'm new to this and my scripts were accidentally pouring in data as strings 
instead of integers. I've fixed that, but now those fields are still set to 
be strings only. Is there any way for me to override this?

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e52199f7-234c-4525-8192-b2a9660f608c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Chart treats no sample as 0

[Paweł Lampe]

Hi,

I am using graylog v1.3.4, and I am bit confused about my charts. When I 
have daily resolution and every day there is a sample, chart is ok.
However, once on some day there is no sample, my chart treats lack of 
sample as a 0 value, and chart is looking very strange.
Can I disable this "weird feature" some way ?

P.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/e97fb27f-d379-44a6-98bc-ad22895b10b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Web Interface Certificate differences from v1 and v2

[Jochen Schalanda]

Hi Drew,

you're right, the migration path from Graylog 1.x to 2.x isn't very clearly 
documented yet. We'll eventually fix that once Graylog 2.0.0 has been 
released.

The private key has to be in PKCS#8 format stored as PEM (not DER). The 
X.509 certificate has also be to be stored in PEM format.


Cheers,
Jochen

On Tuesday, 12 April 2016 16:27:10 UTC+2, Drew Miranda wrote:
>
> Hi all, has anyone had any success converting their TLS ceritificates for 
> graylog web from versions 1 (e.g. 1.3.x) to version 2 of graylog?
>
> Maybe I'm just not getting it, but I'm having trouble figuring out EXACTLY 
> what file format the certificate needs to be in.
>
> Previously with v1.x web interface it used a javakeystore. HOWEVER, this 
> is no longer in use and the upgrade path is not clear.
>
> I found some documentation that talks about exporting keys from the 
> keystore but the terminilogy is very inconsistent depending on the 
> webpage/documentation.
>
> I got as far as exporting the "private key" 
> (no clue if this is the correct format)
> keytool -importkeystore -srckeystore graylog2.keystore -destkeystore 
> new-store.p12 -deststoretype PKCS12
> openssl pkcs12 -info -in new-store.p12
> openssl pkcs12 -in new-store.p12 -nocerts -out gl2web_privateKey.pem
>
> to produce supposedly what the documentation for graylog claims it needs,
>
> I do something similar for the public key
> keytool -export -keystore graylog2.keystore -alias graylog2key -file 
> Example.cer
> openssl x509 -in Example.cer -inform der -text -noout
> openssl x509 -inform der -in Example.cer -out gl2web_publickey.pem
>
> I get this error
>
> I end up with this error which is vague, but I think tells me my 
> certificate configuration is useless.
>
> 2016-04-12 10:06:27,503 ERROR: 
> com.google.common.util.concurrent.ServiceManager - Service 
> WebInterfaceService [FAILED] has failed in the STARTING state.
> java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 
> 48)
> at 
> sun.security.util.ObjectIdentifier.(ObjectIdentifier.java:253) 
> ~[?:1.8.0_77]
> at 
> sun.security.util.DerInputStream.getOID(DerInputStream.java:281) 
> ~[?:1.8.0_77]
> at 
> com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267) 
> ~[sunjce_provider.jar:1.8.0_77]
> at 
> java.security.AlgorithmParameters.init(AlgorithmParameters.java:293) 
> ~[?:1.8.0_77]
> at 
> sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132) 
> ~[?:1.8.0_77]
> at sun.security.x509.AlgorithmId.(AlgorithmId.java:114) 
> ~[?:1.8.0_77]
> at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:372) 
> ~[?:1.8.0_77]
> at 
> javax.crypto.EncryptedPrivateKeyInfo.(EncryptedPrivateKeyInfo.java:95) 
> ~[?:1.8.0_77]
> at 
> org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:69)
>  
> ~[graylog.jar:?]
> at 
> org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:96)
>  
> ~[graylog.jar:?]
> at 
> org.graylog2.shared.initializers.AbstractJerseyService.buildSslEngineConfigurator(AbstractJerseyService.java:185)
>  
> ~[graylog.jar:?]
> at 
> org.graylog2.shared.initializers.AbstractJerseyService.setUp(AbstractJerseyService.java:156)
>  
> ~[graylog.jar:?]
> at 
> org.graylog2.initializers.WebInterfaceService.startUp(WebInterfaceService.java:46)
>  
> ~[graylog.jar:?]
> at 
> com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:60)
>  
> [graylog.jar:?]
> at 
> com.google.common.util.concurrent.Callables$3.run(Callables.java:100) 
> [graylog.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/50c4cc51-e01a-43df-b86a-829840d8c5db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog-web time range problem

[Jochen Schalanda]

Hi Hasan,

please provide some example messages, some example search queries 
(including what you expect what the actual result is), and the time zone 
configuration you see in the web interface on the System -> Overview page.

Cheers,
Jochen

On Tuesday, 12 April 2016 15:17:52 UTC+2, hasan akgöz wrote:
>
> Hi Jochen,
>
> I configured web user timezone "Europe/Istanbul".  I get to log gelf 
> protocol. I using graylog-collector product. Yes I'am sure,  log source 
> is come with a proper timestamp and time configuration box on graylog-web 
> is seems ok. But timestamp column and search box time is false. if you 
> wanna any configuration field or screenshot I can show you.  I'm looking 
> for in the following way.  Keyword time frame selector. like this is "4 
> hours ago". But preview is false.
>
> Cheers,
> Hasan
>
>
> On Tuesday, April 12, 2016 at 3:47:49 PM UTC+3, Jochen Schalanda wrote:
>>
>> Hi Hasan,
>>
>> which timezone did you configure for the logged in Graylog user? How 
>> exactly are you ingesting logs (GELF, syslog, or other inputs)? Are you 
>> sure they come with a proper timestamp (e. g. ISO 8601, including a 
>> timezone)?
>>
>> Cheers,
>> Jochen
>>
>> On Tuesday, 12 April 2016 14:34:06 UTC+2, hasan akgöz wrote:
>>>
>>> I use graylog-web 1.3.4. I set timezone configuration "Europe /Istanbul" 
>>> . Time configuration space as everything seems ok. But when I specify a 
>>> date range of the search area it shows the value back to 1 hour and 
>>> time stamp column is 1 hours ago shows. For example, my system clock is 
>>> 15:00 am. I want to previous 1 hours ago logs. But the results are coming 
>>> up empty. When I change my time range for example 2 hours ago records 
>>> after result comes. but results is false. results is 1 hour ago records 
>>> from the last 14.00 hours.. I view elasticsearch record query returns the 
>>> answer for system clock back 1 hour records. How can I fix this 
>>> problem? Where should I look?
>>>
>>> Operation System : Debian 8.03 ( jessie )
>>> JDK version : oracle jdk-8u77
>>> mongo db version : 2.4.10
>>> Elasticsearch version 1.7.5 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ab895cfb-a882-427f-937d-fc9b0bb67aba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Web Interface Certificate differences from v1 and v2

[Drew Miranda]

Hi all, has anyone had any success converting their TLS ceritificates for 
graylog web from versions 1 (e.g. 1.3.x) to version 2 of graylog?

Maybe I'm just not getting it, but I'm having trouble figuring out EXACTLY 
what file format the certificate needs to be in.

Previously with v1.x web interface it used a javakeystore. HOWEVER, this is 
no longer in use and the upgrade path is not clear.

I found some documentation that talks about exporting keys from the 
keystore but the terminilogy is very inconsistent depending on the 
webpage/documentation.

I got as far as exporting the "private key" 
(no clue if this is the correct format)
keytool -importkeystore -srckeystore graylog2.keystore -destkeystore 
new-store.p12 -deststoretype PKCS12
openssl pkcs12 -info -in new-store.p12
openssl pkcs12 -in new-store.p12 -nocerts -out gl2web_privateKey.pem

to produce supposedly what the documentation for graylog claims it needs,

I do something similar for the public key
keytool -export -keystore graylog2.keystore -alias graylog2key -file 
Example.cer
openssl x509 -in Example.cer -inform der -text -noout
openssl x509 -inform der -in Example.cer -out gl2web_publickey.pem

I get this error

I end up with this error which is vague, but I think tells me my 
certificate configuration is useless.

2016-04-12 10:06:27,503 ERROR: 
com.google.common.util.concurrent.ServiceManager - Service 
WebInterfaceService [FAILED] has failed in the STARTING state.
java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 
48)
at 
sun.security.util.ObjectIdentifier.(ObjectIdentifier.java:253) 
~[?:1.8.0_77]
at sun.security.util.DerInputStream.getOID(DerInputStream.java:281) 
~[?:1.8.0_77]
at 
com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267) 
~[sunjce_provider.jar:1.8.0_77]
at 
java.security.AlgorithmParameters.init(AlgorithmParameters.java:293) 
~[?:1.8.0_77]
at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132) 
~[?:1.8.0_77]
at sun.security.x509.AlgorithmId.(AlgorithmId.java:114) 
~[?:1.8.0_77]
at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:372) 
~[?:1.8.0_77]
at 
javax.crypto.EncryptedPrivateKeyInfo.(EncryptedPrivateKeyInfo.java:95) 
~[?:1.8.0_77]
at 
org.graylog2.shared.security.tls.PemKeyStore.generateKeySpec(PemKeyStore.java:69)
 
~[graylog.jar:?]
at 
org.graylog2.shared.security.tls.PemKeyStore.buildKeyStore(PemKeyStore.java:96) 
~[graylog.jar:?]
at 
org.graylog2.shared.initializers.AbstractJerseyService.buildSslEngineConfigurator(AbstractJerseyService.java:185)
 
~[graylog.jar:?]
at 
org.graylog2.shared.initializers.AbstractJerseyService.setUp(AbstractJerseyService.java:156)
 
~[graylog.jar:?]
at 
org.graylog2.initializers.WebInterfaceService.startUp(WebInterfaceService.java:46)
 
~[graylog.jar:?]
at 
com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:60)
 
[graylog.jar:?]
at 
com.google.common.util.concurrent.Callables$3.run(Callables.java:100) 
[graylog.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/13160a96-aeb6-4c5e-82f0-a387d802d983%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog-web time range problem

[hasan akgöz]

Hi Jochen,

I configured web user timezone "Europe/Istanbul".  I get to log gelf 
protocol. I using graylog-collector product. Yes I'am sure,  log source is come 
with a proper timestamp and time configuration box on graylog-web is seems 
ok. But timestamp column and search box time is false. if you wanna any 
configuration field or screenshot I can show you.  I'm looking for in the 
following way.  Keyword time frame selector. like this is "4 hours ago". 
But preview is false.

Cheers,
Hasan


On Tuesday, April 12, 2016 at 3:47:49 PM UTC+3, Jochen Schalanda wrote:
>
> Hi Hasan,
>
> which timezone did you configure for the logged in Graylog user? How 
> exactly are you ingesting logs (GELF, syslog, or other inputs)? Are you 
> sure they come with a proper timestamp (e. g. ISO 8601, including a 
> timezone)?
>
> Cheers,
> Jochen
>
> On Tuesday, 12 April 2016 14:34:06 UTC+2, hasan akgöz wrote:
>>
>> I use graylog-web 1.3.4. I set timezone configuration "Europe /Istanbul" 
>> . Time configuration space as everything seems ok. But when I specify a 
>> date range of the search area it shows the value back to 1 hour and time 
>> stamp column is 1 hours ago shows. For example, my system clock is 15:00 
>> am. I want to previous 1 hours ago logs. But the results are coming up 
>> empty. When I change my time range for example 2 hours ago records after 
>> result comes. but results is false. results is 1 hour ago records from the 
>> last 14.00 hours.. I view elasticsearch record query returns the answer 
>> for system clock back 1 hour records. How can I fix this problem? Where 
>> should I look?
>>
>> Operation System : Debian 8.03 ( jessie )
>> JDK version : oracle jdk-8u77
>> mongo db version : 2.4.10
>> Elasticsearch version 1.7.5 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/cc847864-6b77-4ca0-a086-1bad4d43e1e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: installed marvel - now seemed to have corrupted entire graylog db

[Jochen Schalanda]

Hi Jason,

what's your Graylog and Elasticsearch configuration? Are there any error 
messages in the logs of either Graylog or Elasticsearch?

Cheers,
Jochen

On Monday, 11 April 2016 07:15:17 UTC+2, Jason Haar wrote:
>
> Hi there
>
> Over the weekend I installed the ES marvel diagnostics package and the 
> following day noticed that graylog was broken. Restarting ES showed 20,000 
> shards in an UNASSIGNED state. I disabled graylog-server (so there was no 
> new data flowing in) and watched over the next couple of hours as that 
> UNASSIGNED number dropped down to 0 (and GREEN). I then restarted ES and - 
> bam - back to 20,000 UNASSIGNED shards again
>
> I've now done three iterations of that - looks like it's completely 
> borked. There's over 5TB of data in there - how can I regain it?
>
> PS: I don't know if this has anything to do with marvel - it's just the 
> last change I made. The reason I installed it was because I have had ES 
> continually doing this kind of thing - but previously stopping graylog, 
> restarting ES and waiting would lead to a happy ES - but no longer.
>
> -- 
> Cheers
>
> Jason Haar
> Information Security Manager, Trimble Navigation Ltd.
> Phone: +1 408 481 8171
> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/507dbfa3-0239-45ca-9799-2cae7db65418%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Grok pattern convert to int

[Jochen Schalanda]

Hi Alexey,

FWIW, that's a limitation of the dynamic mapping in Elasticsearch. If you 
want to provide a fixed schema for your data, take a look at 
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/mapping.html 
and 
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-templates.html
.

Cheers,
Jochen

On Monday, 11 April 2016 18:01:46 UTC+2, Alexey Chuenko wrote:
>
> Ok apparently I had to either create new index or rename the field. 
> Because graylog is setting data type once and doesn't update when it 
> changes.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/82d5a3a6-1670-4f39-be45-3b9b7a170a60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Check disk usage

[Jochen Schalanda]

Hi,

ingesting 200 messages per day (which comes down to ~25 messages per 
second) isn't that much and should work with pretty standard systems (4 
CPUs, 8 GB of memory, disk space depending on average message size).

If the disk journal fills up, that usually means that the backend (i. e. 
Elasticsearch) is not fast enough. You should check the hardware specs for 
your Elasticsearch cluster.

Cheers,
Jochen

On Monday, 11 April 2016 18:46:00 UTC+2, Graylog-WAF wrote:
>
> Thank you first for your reply.
>
> We are receiving about 2 millions messages and more per day.
>
> Please do you have any idea how to configure journal size or other 
> parameters in order to treat this big number.
>
> Journal is being full quickly so Graylog seems blocked and I can't find 
> anything in "Search" tab or in the customised dashboard.
>
>
>
> Le lundi 11 avril 2016 11:34:38 UTC+1, Jochen Schalanda a écrit :
>>
>> Hi,
>>
>> messages ingested by Graylog are first persisted to a disk journal. From 
>> there the messages are being read, processed (extractors, sorting into 
>> streams etc.), written to the outputs (by default Elasticsearch, other 
>> outputs depending on the configuration), and finally removed from the disk 
>> journal.
>>
>> On Saturday, 9 April 2016 22:36:38 UTC+2, Graylog-WAF wrote:
>>>
>>> I have used the OVA file in which there is only 4 Gb of RAM.
>>>
>>> Does this have effect on the capacity of storage?
>>>
>>
>> It doesn't affect the storage capacity directly, but the maximum possible 
>> message fields being loaded into memory by Elasticsearch, e. g. for the 
>> quick values functionality or searches in general.
>>
>>
>> Cheers,
>> Jochen
>>
>> On Saturday, 9 April 2016 22:33:48 UTC+2, Graylog-WAF wrote:
>>>
>>> Hello everybody,
>>>
>>> We are implementing Graylog2 and which is integrated with WAF.
>>>
>>> It's receiving about 2 Millions events per day.
>>>
>>> I would like to know where logs are saved at the beginning (I mean are 
>>> they saved directly in disk or in DB and then in disk).
>>>
>>> Also, is it possible to know the exact percentage that's used until now.
>>>
>>> Thanks !
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/463cf6c1-5763-46d7-b4e7-db81a1b0e07e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Graylog-web time range problem

[Jochen Schalanda]

Hi Hasan,

which timezone did you configure for the logged in Graylog user? How 
exactly are you ingesting logs (GELF, syslog, or other inputs)? Are you 
sure they come with a proper timestamp (e. g. ISO 8601, including a 
timezone)?

Cheers,
Jochen

On Tuesday, 12 April 2016 14:34:06 UTC+2, hasan akgöz wrote:
>
> I use graylog-web 1.3.4. I set timezone configuration "Europe /Istanbul" . 
> Time configuration space as everything seems ok. But when I specify a 
> date range of the search area it shows the value back to 1 hour and time 
> stamp column is 1 hours ago shows. For example, my system clock is 15:00 
> am. I want to previous 1 hours ago logs. But the results are coming up 
> empty. When I change my time range for example 2 hours ago records after 
> result comes. but results is false. results is 1 hour ago records from the 
> last 14.00 hours.. I view elasticsearch record query returns the answer 
> for system clock back 1 hour records. How can I fix this problem? Where 
> should I look?
>
> Operation System : Debian 8.03 ( jessie )
> JDK version : oracle jdk-8u77
> mongo db version : 2.4.10
> Elasticsearch version 1.7.5 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/43292254-f0f1-42fd-bca1-866f3c5669ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Graylog-web time range problem

[hasan akgöz]

I use graylog-web 1.3.4. I set timezone configuration "Europe /Istanbul" . 
Time configuration space as everything seems ok. But when I specify a date 
range of the search area it shows the value back to 1 hour and time stamp 
column is 1 hours ago shows. For example, my system clock is 15:00 am. I 
want to previous 1 hours ago logs. But the results are coming up empty. When 
I change my time range for example 2 hours ago records after result comes. 
but results is false. results is 1 hour ago records from the last 14.00 
hours.. I view elasticsearch record query returns the answer for system 
clock back 1 hour records. How can I fix this problem? Where should I look?

Operation System : Debian 8.03 ( jessie )
JDK version : oracle jdk-8u77
mongo db version : 2.4.10
Elasticsearch version 1.7.5 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/5a75f4a5-155e-460f-a313-2bfecbc364d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.