Re: [graylog2] Full list of Permissions?

2016-08-10 Thread Pete GS
c/main/java/ > org/graylog2/shared/security/RestPermissions.java#L37-L128 > > Some plugins may also add additional permissions that you would need to > check in the plugin code if you want to grant them too. > > I hope that helps. > > Regards, > Edmundo > > > On 09 Aug

[graylog2] Re: Full list of Permissions?

2016-08-08 Thread Pete GS
Noone has any ideas on this? If I'm missing some obvious bit of doco or something please point me in the right direction... I don't recall seeing anything on this aside from the standard Roles though. Cheers, Pete On Thursday, 4 August 2016 14:16:02 UTC+10, Pete GS wrote: > > Hi all, &g

[graylog2] Full list of Permissions?

2016-08-03 Thread Pete GS
Hi all, Just wondering where I could find a full list of available permissions I can assign to roles via the API? I've got a few people here I would like to give extra privileges to without granting full Admin rights. Things like creating/deleting dashboards and streams and viewing the status

Re: [graylog2] Graylog Sidecar reports "unable to map property tags"

2016-08-03 Thread Pete GS
Ok, looks like there were some surplus lines in the default collector_sidecar.yml file that I hadn't defined. Now that they've been removed it seems to be working. Cheers, Pete On Thursday, 4 August 2016 07:38:20 UTC+10, Pete GS wrote: > > Thanks for the reply Marius. > > I'm p

Re: [graylog2] Graylog Sidecar reports "unable to map property tags"

2016-08-03 Thread Pete GS
binary. Maybe > you still execute the old binary? Try the one in \Program Files to verify. > > Cheers, > Marius > > > On 3 August 2016 at 00:19, Pete GS <starp...@gmail.com > > wrote: > >> I seem to be encountering this same issue with 0.0.9-be

Re: [graylog2] Re: Active Directory Settings

2016-08-02 Thread Pete GS
Then an > object class of group followed by a name of Graylog*. So putting too much > search criteria can cause an issue because you're looking to definitively, > but broadening the scope allowed it to work. Is that correct? > > Thank you again for your help! This community has been very

Re: [graylog2] Re: Active Directory Settings

2016-08-02 Thread Pete GS
> settings > are correct." > > If I click on that link, it takes me to my LDAP Settings page. Here is my > settings now: > > > ​ > > On Tue, Aug 2, 2016 at 5:24 PM, Pete GS <starp...@gmail.com > > wrote: > >> Glad to hear it! >> >> If

Re: [graylog2] Re: Active Directory Settings

2016-08-02 Thread Pete GS
ht direction there? > > On Tue, Aug 2, 2016 at 4:11 PM, Pete GS <starp...@gmail.com > > wrote: > >> H seems my updates to my fields didn't get saved for some reason. >> >> Simply substitute the distinguished name "dc=company,dc=corp" for >> &qu

Re: [graylog2] Graylog Sidecar reports "unable to map property tags"

2016-08-02 Thread Pete GS
I seem to be encountering this same issue with 0.0.9-beta-1. time="2016-08-03T08:13:26+10:00" level=error msg="[UpdateRegistration] Failed to report collector status to server: PUT http://graylog.lab.melbourneit.com:12900

[graylog2] Re: Active Directory Settings

2016-08-02 Thread Pete GS
H seems my updates to my fields didn't get saved for some reason. Simply substitute the distinguished name "dc=company,dc=corp" for "dc=lab,dc=melbourneit,dc=com". All else should stay the same. Cheers, Pete On Wednesday, 3 August 2016 06:08:11 UTC+10, Joshua Walderbach wrote: > > I need

[graylog2] Re: Active Directory Settings

2016-08-02 Thread Pete GS
You can pretty much copy the examples provided verbatim for it to work. In our test lab environment I just use the provided examples with our domain etc. substituted. Our Production AD is quite different though and very large with many OU's, so I have to be more specific in that scenario. One

Re: [graylog2] Re: Check Graylog Node Status via API

2015-08-09 Thread Pete GS
between the systems. Are you using NTP (or any similar technology) to sync the clocks of the systems running Graylog? Cheers, Jochen On Thursday, 6 August 2015 23:33:33 UTC+2, Pete GS wrote: One of my nodes this morning is reporting the meta info error in the logs so I took the opportunity

[graylog2] Re: Check Graylog Node Status via API

2015-07-30 Thread Pete GS
/system/cluster/stats (or more specifically http://localhost:12900/system/cluster/stats/elasticsearch and http://localhost:12900/system/cluster/stats/mongo). Cheers, Jochen On Thursday, 30 July 2015 02:12:19 UTC+2, Pete GS wrote: This is possibly a little obscure but also possibly useful

[graylog2] Check Graylog Node Status via API

2015-07-29 Thread Pete GS
This is possibly a little obscure but also possibly useful... I've written a Nagios plugin (in Perl) to check the health of all my Graylog nodes but the one thing I can't seem to find how to check is the status of a Graylog node in relation to being able to connect to the MongoDB. I can check

[graylog2] Re: Alert when Graylog stop to receiving data from some server

2015-07-15 Thread Pete GS
The other way to do this would be to output to something like Riemann, particularly if you have (like we do) a very large number of hosts and don't want to configure a stream for each host. The other reason streams may be impractical is if you have hosts being configured to send to Graylog

[graylog2] Re: Load Balancer State Persistence?

2015-07-15 Thread Pete GS
this in order for your setup to work? Cheers, Jochen On Wednesday, 15 July 2015 01:57:05 UTC+2, Pete GS wrote: Hi all, Is there any way for the load balancer state to remain persistent across service restarts at all? I have two nodes that I use as dedicated search nodes but I like

[graylog2] Re: Filter or Drop messages from a specific source

2015-07-13 Thread Pete GS
Sorry for waking up an older thread... however I have an LDAP server out of my controller which is absolutely smashing my Graylog servers due to a misconfigured logging level. Unfortunately the sys admins for this server are pretty much unresponsive so I think my only choice is to drop this via

Re: [graylog2] Riemann Plugin - Slows Message Processing Dramatically

2015-07-12 Thread Pete GS
Hi Marius, Issue logged this morning: https://github.com/Graylog2/graylog2-plugin-output-riemann/issues/3 Let me know if there is further information you require, I've tried to provide as much as I can think of for the moment. Cheers, Pete On Friday, 10 July 2015 15:54:00 UTC+10, Pete GS

[graylog2] Re: Anti-tampering solution for Graylog2?

2015-07-09 Thread Pete GS
I think you'll find the potential tampering is actually on the ElasticSearch side rather than Graylog. Graylog simply sends the data to ElasticSearch and the most it can do once the data is indexed is delete an index, so any tampering as such would need to be done directly on the ElasticSearch

[graylog2] Web Interface Error when Accessing system/nodes

2015-06-29 Thread Pete GS
Hi all, I suspect this is an issue in my environment rather than a bug, but occasionally when trying to access system/nodes the Web Interface generates errors like the below. Sometimes it just happens for no apparent reason, but probably 7 times out of 10 it's due to one or more of my nodes

[graylog2] Confused by message filed truncation

2015-06-23 Thread Pete GS
Hi all, I'm sending my VMware vCenter server logs and Windows event logs into Graylog using nxlog-ce to send to GELF UDP inputs. I'm getting confused as to why the message field is truncated compared with the full_message. At this point I have not tried defining any fields in nxlog for these

Re: [graylog2] Re: Installing from RPM on CentOS6 and java options

2015-06-18 Thread Pete GS
This includes lost+found if you have mounted another file system there. This caught me out when I added an additional larger volume to house the message journal as by default Linux will create the lost+found directory and Kafka absolutely does not like this directory being present. Cheers,

Re: [graylog2] Edit Extractor - Load Message no working?

2015-06-17 Thread Pete GS
. Regards, Edmundo On 15 Jun 2015, at 21:03, Pete GS starp...@gmail.com javascript: wrote: Hi Jochen, At work I'm on Windows 8.1 and have tried both Chrome and IE. I'm not sure what versions of those I'm running (at home for a few days). At home here I see the same

Re: [graylog2] Extractors with Wildcards Cause High CPU/Load Average

2015-06-14 Thread Pete GS
processing the same number of messages/second and extracting the exact same data. So once again thanks Kay! Cheers, Pete On Saturday, 6 June 2015 06:16:54 UTC+10, Pete GS wrote: Ah thanks Kay! I've never looked into Grok patterns, but that sounds like they could help a great deal

[graylog2] Edit Extractor - Load Message no working?

2015-06-14 Thread Pete GS
I'm not sure if this is a bug or not but I have noticed in both my test lab and Production environments that I cannot load a message on the edit extractors screen. When I provide the message ID and index then click the Load Message button, nothing happens. I'm running 1.1.2 in both

Re: [graylog2] Extractors with Wildcards Cause High CPU/Load Average

2015-06-05 Thread Pete GS
Pete GS starp...@gmail.com javascript: wrote: Hi all, I've finally discovered the source of my excess CPU load and high load averages on my Graylog nodes! I've got a bunch of extractors that I use to pull information from my vSphere platform's VMKernel logs. The catch

[graylog2] Extractors with Wildcards Cause High CPU/Load Average

2015-06-04 Thread Pete GS
Hi all, I've finally discovered the source of my excess CPU load and high load averages on my Graylog nodes! I've got a bunch of extractors that I use to pull information from my vSphere platform's VMKernel logs. The catch with these is that a lot of items in the message string vary quite a

Re: [graylog2] 0.20.1 web ui time-outs and reset

2015-05-24 Thread Pete GS
I can confirm it definitely does work, I need to use it for the moment until I can get some better ElasticSearch hardware. Cheers, Pete On Friday, 22 May 2015 21:27:49 UTC+10, Kay Röpke wrote: Yes the parameter should still work. On 22 May 2015, at 13:26, Martin René Mortensen

Re: [graylog2] High CPU and did not find meta info issues since adding new Graylog servers and increased input messages/second

2015-05-24 Thread Pete GS
2015 06:08:44 UTC+2 schreef Pete GS: Ok, here's where I'm at with this... I tried implementing the kernel options on one of the Graylog servers as a test but it made no appreciable difference. In fact shortly after the first reboot the VM froze with a locked CPU error. It hasn't done

[graylog2] Re: Graylog RPM's - Java Version Dependency

2015-05-19 Thread Pete GS
you want to add your thoughts or a use-case description to this issue. Cheers, Jochen On Tuesday, 19 May 2015 00:25:17 UTC+2, Pete GS wrote: Hi all, Not sure if this is the best place for this or not but the Graylog RPM's available via the YUM repository have a dependency on Java 7

[graylog2] Graylog RPM's - Java Version Dependency

2015-05-18 Thread Pete GS
Hi all, Not sure if this is the best place for this or not but the Graylog RPM's available via the YUM repository have a dependency on Java 7. Is it possible to get this dependency either removed or updated to Java 8? I use the ElasticSearch RPM's via their YUM repository also and it has no

Re: [graylog2] High CPU and did not find meta info issues since adding new Graylog servers and increased input messages/second

2015-05-12 Thread Pete GS
+10, Pete GS wrote: I've come back to the office this morning and discovered we had an ElasticSearch issue last night which has resulted in lots of unprocessed messages in the journal. All the Graylog nodes are busy processing these and it seems to be slowly crunching through them. Load

Re: [graylog2] High CPU and did not find meta info issues since adding new Graylog servers and increased input messages/second

2015-05-05 Thread Pete GS
schreef Pete GS: Yesterday I did a yum update on all Graylog and MongoDB nodes and since doing that and rebooting them all (there was a kernel update) it seems that there are no longer issues connecting to the Mongo database. However, I'm still seeing excessively high CPU usage on the Graylog

[graylog2] Re: Newbie to graylog and log management

2015-04-30 Thread Pete GS
It sounds like you need to start reading the Graylog documentation :) Message ID and index can be found when viewing a message. Click on a message in the search window and the right hand pane will show you both of these items. You should have a Sources menu item at the top of the screen which

[graylog2] Re: Securing graylog-server to elasticsearch

2015-04-30 Thread Pete GS
The only way I'm aware of at present is to use iptables or other network ACL's to ensure only your Graylog servers can communicate with Elasticsearch. I don't believe Elasticsearch has any authentication mechanism as yet to facilitate this. Cheers, Pete On Friday, 1 May 2015 05:59:21 UTC+10,

[graylog2] Re: High CPU and did not find meta info issues since adding new Graylog servers and increased input messages/second

2015-04-30 Thread Pete GS
Does anyone have any thoughts on this? Even if someone could identify some scenarios that would cause high CPU on Graylog servers and in what circumstances Graylog would have trouble contacting the MongoDB servers. Cheers, Pete On Wednesday, 29 April 2015 10:34:28 UTC+10, Pete GS wrote: Hi

[graylog2] High CPU and did not find meta info issues since adding new Graylog servers and increased input messages/second

2015-04-28 Thread Pete GS
Hi all, We acquired a company a while ago and last week we added all of their logs to our Graylog environment which all come in from their Syslog server via UDP. After this, I noticed that the Graylog servers were maxing CPU so to alleviate this I increased CPU resources to the existing

[graylog2] Re: High CPU and did not find meta info issues since adding new Graylog servers and increased input messages/second

2015-04-28 Thread Pete GS
Apologies, I should've clarified we're running Graylog 1.0.1. On Wednesday, April 29, 2015 at 10:34:28 AM UTC+10, Pete GS wrote: Hi all, We acquired a company a while ago and last week we added all of their logs to our Graylog environment which all come in from their Syslog server via UDP

Re: [graylog2] Re: Remote Logs to Graylog - Recommended Setup?

2015-03-22 Thread Pete GS
perspective RabbitMQ is probably easier to deploy than Kafka for 4-5 distributed data centers. Best, Kay On 22/03/15 00:11, Pete GS wrote: Thanks Kay, The only thing we need to address is unreliable links, which is why I thought a simple message broker setup at the remote site would

[graylog2] Re: Remote Logs to Graylog - Recommended Setup?

2015-03-21 Thread Pete GS
from the DMZ, we use some RabbitMQ brokers : servers === (syslog over UDP or TCP) === Logstash === (AMQP) === RabbitMQ === (AMQP) === Logstash === (GELF) === Graylog Mathieu Le vendredi 20 mars 2015 05:51:11 UTC+1, Pete GS a écrit : Hi all, We're looking at adding message sources into our

Re: [graylog2] Remote Logs to Graylog - Recommended Setup?

2015-03-21 Thread Pete GS
over TCP. On 20 Mar 2015, at 05:51, Pete GS starp...@gmail.com javascript: wrote: Hi all, We're looking at adding message sources into our Graylog setup from a couple of remote sites. There is the possibility of temporary transit link outages so sending UDP packets would result in lost

Re: [graylog2] Re: Remote Logs to Graylog - Recommended Setup?

2015-03-21 Thread Pete GS
deployments still benefit from message broker setups and as such are a viable solution currently. Feel free to contact me directly if you have more detailed questions. Best regards, Kay On 21/03/15 23:31, Pete GS wrote: I see what you're saying there... however is that a little too

[graylog2] Re: Stream or Search for Excessive Windows Events from the Same Source

2015-03-19 Thread Pete GS
manner. On Monday, March 16, 2015 at 11:38:38 PM UTC+1, Pete GS wrote: NXLog is how we send them also and we get source/system names, the problem is alerting or searching based on the number of events from the same source without having to specify a particular source. I haven't looked

Re: [graylog2] Inputs gone after updating to 1.0.1 from the latest 0.9x

2015-03-19 Thread Pete GS
I had the same issue briefly and the cause of it was actually due to the change in the directory structure from graylog2-server to graylog-server. I run CentOS 6.6 and am using the RPM packages. The upgrade moves to the new directory structure, so you will need to ensure all of your

[graylog2] Re: How to define default streams on a reader role in Graylog v1.0.0

2015-03-19 Thread Pete GS
I can't help with a solution but I'll add a +1 to the request as I have exactly the same issue. Cheers, Pete On Thursday, 19 March 2015 08:17:56 UTC+10, Aydin Doyak wrote: Hey, I have a graylog v1.0.0 running on centos 6.6 like a charm. I have several stream and dashboard definitions in

[graylog2] Remote Logs to Graylog - Recommended Setup?

2015-03-19 Thread Pete GS
Hi all, We're looking at adding message sources into our Graylog setup from a couple of remote sites. There is the possibility of temporary transit link outages so sending UDP packets would result in lost messages. Using TCP will counter this to a certain extent but may result in messages

[graylog2] Re: Stream or Search for Excessive Windows Events from the Same Source

2015-03-16 Thread Pete GS
On Tuesday, 17 March 2015 03:31:32 UTC+10, Arie wrote: We send windows events with nxlog (type: gelf), and the system names are automatically included. We look at ES with kibana and have created a view te see what is going on. Op maandag 16 maart 2015 05:48:12 UTC+1 schreef Pete GS: Hi all, We've

Re: [graylog2] Stream or Search for Excessive Windows Events from the Same Source

2015-03-16 Thread Pete GS
might help with that (scheduled for the 1.1.0 release ATM). The above is not dedicated surge protection but I’ll probably created a separate issue for that soon-ish as well. /HJ On 16 Mar 2015, at 05:48, Pete GS starp...@gmail.com javascript: wrote: Hi all, We've been continuing

[graylog2] Stream or Search for Excessive Windows Events from the Same Source

2015-03-15 Thread Pete GS
Hi all, We've been continuing to discuss various other use cases for Graylog here and there is one scenario that I can't figure out a solution for. Essentially, if an unknown Windows issue occurs, it will generally result in the Windows Event Logs being spammed with hundreds or thousands of

Re: [graylog2] Anybody have a good extractor for ESXi 5.5 logs?

2015-03-11 Thread Pete GS
I meant to weigh in on this the other day... Yes, VMware logs are not the best to work with between multiline formats, facilities that are random at best, and so forth. Before implementing Graylog2, I set up a centralised syslog server with CentOS 7 and the provided rsyslogd to capture all our

Re: [graylog2] VmWare esx 5 log format

2015-01-21 Thread Pete GS
We have vSphere 5.5 logs going to Graylog2 as well but ours go via a centralised Syslog server using rsyslogd and this works very well for us. I would highly recommend this configuration for getting vSphere logs into Graylog2 for two reasons... 1. The aforementioned issues with the log formats

[graylog2] Re: Active Directory Authentication

2015-01-20 Thread Pete GS
Tom, I know this is an old post but did you ever get this solved? I had the same issue and have resolved it so I'll post the solution here in case it is the same... Essentially the issue for us was with the Active Directory user configured in Graylog2 to bind to the Active Directory. If you

Re: [graylog2] Re: How to determine GB/Day for Inputs?

2015-01-14 Thread Pete GS
somewhere. Cheers, Kay On Jan 8, 2015 11:01 PM, Pete GS starp...@gmail.com wrote: Thanks guys, yes messages per second don't help in this case :) My understanding of index size is that it will be larger than the amount of data coming in as it will contain the raw message as well

[graylog2] Re: Graylog2 HA

2015-01-08 Thread Pete GS
We have the same issue here, we actually have 4 Graylog2 servers, two are dedicated search nodes for the web interface (one of which is the master) and the other two are dedicated for inputs. If I ever have the master node offline, the web interface reports it cannot contact a Graylog2 master

[graylog2] Re: How to fix Nodes with too long GC pauses issues in my cluster.

2015-01-08 Thread Pete GS
it prevents problems caused with noisy neighbors and disk-cache thrashing in virtualized environments. Cheers, Jochen On Thursday, 8 January 2015 02:03:39 UTC+1, Pete GS wrote: I've recently been through this in another thread here and some very helpful replies had me looking at ElasticHQ

[graylog2] Re: How to determine GB/Day for Inputs?

2015-01-08 Thread Pete GS
Thanks guys, yes messages per second don't help in this case :) My understanding of index size is that it will be larger than the amount of data coming in as it will contain the raw message as well as extracted fields and header information, so I don't think using index size will give me this

[graylog2] Re: How to fix Nodes with too long GC pauses issues in my cluster.

2015-01-07 Thread Pete GS
I've recently been through this in another thread here and some very helpful replies had me looking at ElasticHQ also and this is an excellent plugin that helped me understand my Elasticsearch nodes were CPU bound. I have gone away from VM's for Elasticsearch and am instead now using physical

[graylog2] How to determine GB/Day for Inputs?

2015-01-07 Thread Pete GS
Hi all, I'm trying to find a way to calculate how much data we're sending to Graylog2 (GB/day). I can see for each input the total GiB it has received, but this is since the input was started. Is there a way to reset this figure without terminating and starting a new input? Pausing doesn't

Re: [graylog2] Buffer Timeouts, GC taking longer than 1 second, how to diagnose?

2014-12-14 Thread Pete GS
to long GC times. Can you check the IO load of your ES machine(s)? Also check your ES logs. Thanks, Lennart On Sun, Dec 14, 2014 at 7:12 PM, Pete GS starp...@gmail.com javascript: wrote: Hi all, we're implementing Graylog2 here at work for general log monitoring/analysis as our

Re: [graylog2] Buffer Timeouts, GC taking longer than 1 second, how to diagnose?

2014-12-14 Thread Pete GS
I'll need to investigate further into Elasticsearch to see what is driving the CPU so hard. On Mon, Dec 15, 2014 at 1:42 PM, Pete GS starpoin...@gmail.com wrote: Excellent thanks again Lennart, I'll take a look into it. The one big issue I have is no SSD's for the Elasticsearch active nodes