[graylog2] Re: Search within an extracted field help

2015-09-09 Thread Jochen Schalanda
Hi, currently only a few message fields are being analyzed by default (source, message, and full_message) which enables wildcard searches (like *vowel*). If you want to analyze the url message field as well, you'll have to create a matching index template in Elasticsearch, see https://www.elas

[graylog2] Re: Graylog extractors/ Grok patterns

2015-09-09 Thread Jochen Schalanda
Hi Ivan, extractors and Grok patterns are stored in MongoDB and at least extractors are linked to the inputs. Please make sure that you also use an external data volume for MongoDB's data files and that the Graylog node ID doesn't change with every start (see GRAYLOG_NODE_ID at http://docs.gra

[graylog2] Re: Search within an extracted field help

2015-09-09 Thread Jochen Schalanda
Hi, the index templates have to be set up on Elasticsearch directly (please refer to https://www.elastic.co/guide/en/elasticsearch/reference/1.7/indices-templates.html). Graylog currently doesn't support setting up custom index templates. Cheers, Jochen On Wednesday, 9 September 2015 16:41:4

[graylog2] Re: Graylog serving up incorrect Common Name in SSL sessions when using Wildcard SSL cert

2015-09-10 Thread Jochen Schalanda
Hi Tim, the CommonName (CN) or subjAltName in X.509 certificates are interpreted very strictly. That means that *.example.com will match foo.example.com and bar.example.com but not foo.bar.example.com because the latter has 4 components in the FQDN while the X.509 certificate only allows 3 com

Re: [graylog2] How to update virtual appliances?

2015-09-10 Thread Jochen Schalanda
Hi, the upgrade procedure for the virtual machine and Docker images is described in our documentation at http://docs.graylog.org/en/1.1/pages/installation/graylog_ctl.html#upgrade-graylog If you want to install an unreleased version of Graylog (e. g. Graylog 1.2.0-rc.4), you have to use the b

[graylog2] Re: "include"-statement in Graylog Collector config

2015-09-14 Thread Jochen Schalanda
Hi Fabian, as a matter of fact, including other configuration files using an "include" statement is supported in the configuration file used by the Graylog Collector, see https://github.com/typesafehub/config/blob/v1.2.1/HOCON.md#includes for details. Centralized management for the Graylog Co

[graylog2] Re: Search Issue ...

2015-09-14 Thread Jochen Schalanda
Hi Claus, certain characters have to be escaped in the Lucene query syntax (which is being used by Graylog and Elasticsearch), see http://docs.graylog.org/en/1.1/pages/queries.html#escaping for details. Cheers, Jochen On Tuesday, 8 September 2015 10:31:14 UTC+2, Claus Koell wrote: > > Hi ! >

[graylog2] Re: Unable to get the graylog web interface login page.

2015-09-14 Thread Jochen Schalanda
Hi Anant, I can't reproduce your problem with the official Graylog 1.1.6 web interface, so I guess it's because of some changes you've made to the code of your custom compiled version. Cheers, Jochen On Monday, 14 September 2015 14:10:29 UTC+2, Anant Sawant wrote: > > Hi, > > I am running Gra

[graylog2] Re: System information unavailable on a three node Graylog cluster

2015-09-14 Thread Jochen Schalanda
Hi Lorenzo, the error message you've seen ("[…] We expected HTTP 200, but got a HTTP -1.") is usually a sign of a request timeout. By default the request timeout for HTTP requests from the Graylog web interface to a Graylog server node is 5 seconds and can be customized in the configuration fil

[graylog2] Re: Streams and Inputs

2015-09-14 Thread Jochen Schalanda
Hi Felipe, by default all messages from all inputs will be checked and tagged for all streams for which they match the stream rules. If you only want to route messages of a specific input into a stream, you can check the "internal" field gl2_source_input in your stream rules which will contain

[graylog2] Re: [ANNOUNCE] Graylog v1.2 has been released

2015-09-15 Thread Jochen Schalanda
Hi Ivan, the entity configurations are 100% compatible with previous versions of Graylog 1.x. If you find any incompatibilities, it's considered a bug and a bug report at https://github.com/Graylog2/graylog2-server/issues would be greatly appreciated. Cheers, Jochen On Tuesday, 15 September

[graylog2] Re: Updated Graylog Appliance 1.1.6

2015-09-16 Thread Jochen Schalanda
Hi James, yes. Cheers, Jochen On Wednesday, 16 September 2015 11:01:35 UTC+2, rogersmanau wrote: > > Hi All, > > Apologies if this has been asked previously. Is this still the correct > guidance to upgrade the Graylog appliance from 1.1.6 to the new 1.2GA? > http://docs.graylog.org/en/1.2/page

[graylog2] Re: create inputs using config file

2015-09-16 Thread Jochen Schalanda
Hi, Graylog currently doesn't support a "bootstrap" configuration file with entity configurations (e. g. inputs, streams, outputs, etc.) but you can use the content pack feature to export a content pack containing your entity configurations and import/apply that to your Graylog setup via the R

[graylog2] Re: Anyone attempted to use AWS cloudsearch service to back graylog rather than clustering elasticsearch?

2015-09-16 Thread Jochen Schalanda
Hi Cory, Graylog is currently pretty much tied in with Elasticsearch as its default output and for searches. Since AWS Cloudsearch isn't compatible with Elasticsearch, it's probably not easily possible to use that as a backend right now. It should be possible, though, to write a relatively simp

[graylog2] Re: receiving netflow

2015-09-17 Thread Jochen Schalanda
d to support IPFIX/AppFlow in the future? > > Cheers, > Rainer > > Am Mittwoch, 26. August 2015 10:37:35 UTC+2 schrieb Jochen Schalanda: >> >> Hi Marsel, >> >> we will publish a Netflow plugin for Graylog 1.2.0 in the near future. >> I'm not

[graylog2] Re: Search Issue ...

2015-09-17 Thread Jochen Schalanda
Hi, I just tested that on Graylog 1.1.6 and Graylog 1.2.0 and I could successfully find the relevant messages with the query source_file:"C\:\\Program Files\\IBM\\WebSphere\\AppServer8\\profiles\\AppSrv01\\logs\\MyServer\\SystemOut.log". It looks like you just forgot to escape the backslash a

[graylog2] Re: One more search question ...

2015-09-17 Thread Jochen Schalanda
Hi Claus, not all message fields are being analyzed during index time, which enables wildcard searches in the first place. By default, only message, full_message, and source are being analyzed. If you want to analyze other message fields as well, you'll need to create an Elasticsearch index tem

[graylog2] Re: Search issues after update to 1.2

2015-09-21 Thread Jochen Schalanda
Hi Arkadiy, are there any related error messages in your Graylog server node's logs? Additionally, please post the output of the following cURL command (replace "localhost" with the hostname or IP address of one of your Elasticsearch nodes, and "graylog2_*" with your actual index prefix): curl

[graylog2] Re: Search issues after update to 1.2

2015-09-21 Thread Jochen Schalanda
t;> >>> "_source":{"gl2_index_range_index_name":"graylog2_34","gl2_index_range_begin":"1970-01-01T00:00:00.000Z","gl2_index_range_end":"2015-08-01T16:14:17.000Z","gl2_index_range_ >>> calculated_at&

[graylog2] Re: Search issues after update to 1.2

2015-09-21 Thread Jochen Schalanda
oblem. > > > On Monday, September 21, 2015 at 2:45:04 PM UTC+3, Jochen Schalanda wrote: >> >> Hi Arkadiy, >> >> thanks for posting these log messages. The underlying issue will be fixed >> in Graylog 1.2.1 which we plan to release soon (see >> https://git

[graylog2] [ANN] Graylog 1.2.1 has been released

2015-09-22 Thread Jochen Schalanda
Hi everyone, we've released Graylog 1.2.1 today, which is a bugfix release for Graylog 1.2.0 and fixes some problems with loading old (Graylog 1.0.x) alarm callbacks, importing and applying content packs, and some incompatibilities with older Elasticsearch versions. You can find the full relea

[graylog2] Re: EL6 rpms missing init.d scripts

2015-09-22 Thread Jochen Schalanda
Hi Jens, please post the output of *rpm -ql graylog-server* on your system. Cheers, Jochen On Tuesday, 22 September 2015 13:07:08 UTC+2, Jens Kuehnel wrote: > > Hi, > > apparently an update from 1.1 to 1.2 on RHEL6 is removing the init > scripts, because they are missing from the RPMS. > > C

[graylog2] Re: Filter messages before saving them ?

2015-09-23 Thread Jochen Schalanda
Hi Florent, as a matter of fact, there's a blacklisting feature in Graylog but currently there's no UI for it. That means that you'll have to use the Graylog REST API to create those filters. Take a look at the API browser at http://127.0.0.1:12900/api-browser#!/Filters/create_post_0 (replace

[graylog2] Re: Graylog collector not send apache logs

2015-09-23 Thread Jochen Schalanda
Hi Emerson, please verify that the Graylog Collector has sufficient permissions to read files in the /var/log/httpd/ directory. Usually there's a dedicated group (e. g. log or adm, or the Apache httpd user apache or httpd) which is allowed to read those files. You can add the Graylog Collector

[graylog2] Re: Send apache log to Graylog Syslog Input

2015-09-24 Thread Jochen Schalanda
Hi, Apache httpd supports sending error logs into the local syslog (see https://httpd.apache.org/docs/2.4/mod/core.html#errorlog) but not the access logs ( https://httpd.apache.org/docs/2.4/mod/mod_log_config.html#customlog). As for now, you'll need to read the access logs with a log shipper li

[graylog2] Re: any chance we'll have more pluggable datastores for graylog?

2015-09-24 Thread Jochen Schalanda
Hi Dave, simple answer: not in the near future, but it's on our todo list. Cheers, Jochen On Thursday, 24 September 2015 20:57:59 UTC+2, David Dunstan wrote: > > Given graylog's interaction with mongodb is pretty minimal, I'm wondering > if there are any plans to implement more pluggable persis

Re: [graylog2] Syslog Logs from Linksys Accesspoint with DD-WRT not shown

2015-09-24 Thread Jochen Schalanda
Hi, could you please provide some of those messages that DD-WRT is sending? Cheers, Jochen On Thursday, 24 September 2015 22:27:59 UTC+2, js.l...@gmail.com wrote: > > I'm having the exact same problem. > > Timestamps of the log messages in DD-WRT and Graylog are all correct. > -- You received

Re: [graylog2] Re: hyper-v virtual appliance

2015-09-27 Thread Jochen Schalanda
Hi Gareth, I don't have a Windows machine here to test this, but there's an article about converting OVF/OVA images for the use with Hyper-V at Microsoft Technet: https://technet.microsoft.com/en-us/library/jj158932.aspx Additionally, this blog article seems to describe the process very well:

[graylog2] Re: Graylog2 setup, how to send data now?

2015-09-27 Thread Jochen Schalanda
Hi Anthony, you can create a Syslog UDP or Syslog TCP input for Graylog in the web interface at System -> Inputs. Also see http://docs.graylog.org/en/1.2/pages/sending_data.html#syslog for a description how to configure different syslog daemons to work smoothly with Graylog. As for Cisco devi

[graylog2] Re: Graylog.-server service doesn't start after remove /var/lib/graylog-server/journal/* files

2015-09-28 Thread Jochen Schalanda
Hi Roberto, your Elasticsearch cluster health status is RED, which means that ES can't index documents anymore. Check the logs of your Elasticsearch nodes for the reason. Additionally I'd recommend upgrading to Graylog 1.2.1, which also starts when Elasticsearch is unhealthy or not accessible.

[graylog2] Re: Elasticsearch cluster unavailable with graylog 1.2.1 and elasticsearch 1.7.2

2015-09-29 Thread Jochen Schalanda
Hi Alejandro, it seems like Elasticsearch is announcing a different hostname in the cluster than the one it's actually using. I would recommend disabling zen multicast discovery and use unicast discovery instead, see http://docs.graylog.org/en/1.2/pages/configuring_es.html#discovery-mode for

[graylog2] Re: Mongodb not replicated?

2015-09-30 Thread Jochen Schalanda
Hi Jesse, all Graylog nodes must access and use the same MongoDB database. MongoDB itself can of course be replicated to achieve a HA setup, see http://docs.mongodb.org/manual/core/replication-introduction/ for details. Cheers, Jochen On Tuesday, 29 September 2015 23:22:50 UTC+2, Jesse Skrivs

[graylog2] Re: How to load archived syslog data into Graylog.

2015-09-30 Thread Jochen Schalanda
Hi Steve, the easiest way to ingest old log files is sending them via netcat, nxlog, or logstash to Graylog. In case of netcat, you'll probably need a Raw/Plaintext input and a set of extractors in Graylog. In case of nxlog or logstash you could pre-process the logs (e. g. parse them and create

Re: [graylog2] How to update virtual appliances?

2015-09-30 Thread Jochen Schalanda
anks, >> the upgrade to 1.2 went smooth and flawless. >> I can not think of an application in our portfolio that has similar >> complexity merged with similar robustness! bravo! >> >> On Thursday, September 10, 2015 at 3:15:19 PM UTC+2, Jochen Schalanda >>

[graylog2] Re: Grok extractor + break on match

2015-09-30 Thread Jochen Schalanda
Hi Alex, unfortunately that's not possible with Graylog at the moment (skipping the following grok patterns if the current one already matched). This being said, just creating multiple extractors per input will at least give you the extracted message fields, by running all extractors of that i

Re: [graylog2] How to update virtual appliances?

2015-09-30 Thread Jochen Schalanda
Hi Jérôme, currently Elasticsearch 1.7.2 and MongoDB 3.0.6 are included: - https://github.com/Graylog2/omnibus-graylog2/blob/1.2.1-1/config/software/elasticsearch.rb - https://github.com/Graylog2/omnibus-graylog2/blob/1.2.1-1/config/software/mongodb.rb Cheers, Jochen On Wedn

[graylog2] Re: Run collector as a service

2015-10-01 Thread Jochen Schalanda
es? > > On Tuesday, June 16, 2015 at 11:14:27 AM UTC-5, Jochen Schalanda wrote: >> >> Hi Jeremy, >> >> the Graylog Collector comes with a service script for Windows (see >> http://docs.graylog.org/en/1.1/pages/collector.html#windows). Init >> scripts (or s

[graylog2] Re: Elasticsearch - evenly rebalancing shards

2015-10-01 Thread Jochen Schalanda
Thanks for sharing! On Thursday, 1 October 2015 19:20:06 UTC+2, Jesse Skrivseth wrote: > > I tried this in a lab environment and ended up with a split-brain cluster. > You've been warned. ;) > > On Wednesday, September 30, 2015 at 10:36:37 AM UTC-6, Jesse Skrivseth > wrote: >> >> This may be off

Re: [graylog2] Re: Remove old source

2015-10-01 Thread Jochen Schalanda
Hi Juan, you pretty much only have to replace your last GET request with a DELETE request, see https://www.elastic.co/guide/en/elasticsearch/reference/1.7/docs-delete-by-query.html for reference and Jean-Luc's last post for an example. Cheers, Jochen On Thursday, 1 October 2015 16:56:31 UT

[graylog2] Re: System->Indices page times out

2015-10-02 Thread Jochen Schalanda
Hi Jesse, do you see any corresponding errors in the logs of the Graylog server node? Cheers, Jochen On Friday, 2 October 2015 01:49:40 UTC+2, Jesse Skrivseth wrote: > > Since upgrading from 1.1.6 to 1.2.0, the System->Indices page - which used > to load in 2-5 seconds - now takes several minu

[graylog2] Re: amqp/rabbitmq into graylog 1.2.0 ?

2015-10-02 Thread Jochen Schalanda
Hi, Graylog doesn't emulate an AMQP or Kafka broker but it contains a performant and battle-tested message journal which can be used to buffer messages if the backend (i. e. Elasticsearch) is overwhelmed or down. You could use EasyGelf (https://github.com/Pliner/EasyGelf) to send your log mess

Re: [graylog2] SNMP Plugin - BIND Error

2015-10-02 Thread Jochen Schalanda
Hi Arie, alternatively use something like authbind ( https://github.com/tootedom/authbind-centos-rpm) to allow the Java process to bind to privileged ports. Cheers, Jochen On Friday, 2 October 2015 10:38:20 UTC+2, Marius Sturm wrote: > > Hi Arie, > only the root user can bind ports below 1024.

[graylog2] Re: graylog collector: client certificates

2015-10-05 Thread Jochen Schalanda
Hi Marco, the Graylog Collector currently doesn't support TLS client authentication. Feel free to file a feature request for this at https://github.com/Graylog2/collector/issues/new. Cheers, Jochen On Monday, 5 October 2015 11:52:40 UTC+2, Marco Dickert wrote: > > Hi guys, > > with graylog we

[graylog2] Re: System->Indices page times out

2015-10-05 Thread Jochen Schalanda
Hi Jesse, did you manually trigger the calculation of all the index ranges? During normal operations of Graylog 1.2.x, only the index range of the latest index should be calculated and stored (in contrast to Graylog 1.1.x and earlier, which most of the time calculated all index ranges). Cheers

[graylog2] Re: Graylog Sources Tab always showing Error "Could not load histogram Data"

2015-10-05 Thread Jochen Schalanda
Hi Michel, first of all I'd recommend upgrading to the latest stable version of Elasticsearch (which is Elasticsearch 1.7.2 at the time of writing). Are there any error messages in the logs of your Elasticsearch nodes? Chee

[graylog2] Re: Graylog Sources Tab always showing Error "Could not load histogram Data"

2015-10-05 Thread Jochen Schalanda
ates Transport Disconnected? > > > You reckon that would be an issue with ES 1.4.2? Im withholding upgrading > yet just incase it messes up my config. > > Thanks > > > On Monday, 5 October 2015 16:47:49 UTC+1, Jochen Schalanda wrote: >> >> Hi Michel, &

[graylog2] Re: System->Indices page times out

2015-10-06 Thread Jochen Schalanda
5_14:37:34.66598 at > org.graylog2.rest.resources.sources.SourcesResource$1.call(SourcesResource.java:93) > 2015-10-05_14:37:34.66598 at > org.graylog2.rest.resources.sources.SourcesResource$1.call(SourcesResource.java:89) > 2015-10-05_14:37:34.66599 at > com.go

[graylog2] Re: Hostname instead of ip address for the source when SNMP plugin is used

2015-10-06 Thread Jochen Schalanda
Hi Ubay, while the SNMP plugin itself doesn't provide this, you could use the (kind of unsupported) DNS Resolver plugin: https://github.com/Graylog2/graylog-plugin-dnsresolver. This would resolve all IP addresses in the "source" message field and not only the messages from the SNMP plugin, tho

[graylog2] Re: Internal Graylog logging

2015-10-07 Thread Jochen Schalanda
Hi David, Graylog is using log4j 1.2 for its own logging needs. You can download and configure one of the existing log4j GELF appenders (see https://marketplace.graylog.org/addons?search=log4j) to write Graylog's log messages into Graylog itself. This being said, there is the possibility of fe

[graylog2] Re: AWS Elasticsearch

2015-10-09 Thread Jochen Schalanda
Hi William, Graylog is currently joining the Elasticsearch cluster as a regular client (i. e. no master, no data node) which is not possible with the AWS Elasticsearch service. We might come up with a solution to this in the future, but for now you'd have to setup and manage your own Elasticsea

[graylog2] Re: migrating inputs between hosts

2015-10-09 Thread Jochen Schalanda
Hi David, that sounds as if the node ID of your new node changed, see the node_id_file setting ( https://github.com/Graylog2/graylog2-server/blob/1.2.1/misc/graylog2.conf#L5-L7) in your Graylog configuration. Make sure that your new Graylog node is using the same node ID and the inputs should

[graylog2] Re: Stream ID

2015-10-12 Thread Jochen Schalanda
Hi Yoram, the user experience is a little bit lacking in that regard at the moment. You can find the stream ID in the URL of the stream page, e. g. if http://graylog.example.com/streams/556c2208e4b0f1234567890/messages?q=*&rangetype=relative&relative=300 was the URL of the stream page, then *5

[graylog2] Re: Where does Graylog keep audit logs for users logging into Graylog?

2015-10-13 Thread Jochen Schalanda
Hi Josh, Graylog currently doesn't have a dedicated audit log. You can use the normal log output of Graylog (see https://github.com/Graylog2/graylog2-server/blob/1.2.1/graylog2-bootstrap/src/main/resources/log4j.xml) for something similar. Cheers, Jochen On Tuesday, 13 October 2015 10:21:57

[graylog2] Re: Lost all previous data up on upgrade to 1.2.1-1

2015-10-13 Thread Jochen Schalanda
Hi, the commands you've listed look good and shouldn't remove any data from the virtual machine image. Graylog is storing log messages inside Elasticsearch. Please check first, if your Elasticsearch instance still contains some data. Cheers, Jochen On Monday, 12 October 2015 09:03:53 UTC+2,

[graylog2] Re: Graylog GROK and INPUTS

2015-10-13 Thread Jochen Schalanda
Hi, the configuration of inputs and grok patterns are stored in MongoDB in the inputs and grok_patterns collections. Cheers, Jochen On Tuesday, 13 October 2015 21:24:40 UTC+2, kaiser wrote: > > Hello, > > could you please tell me in which file GROK patterns and INPUT > configuration are store

[graylog2] Re: Graylog: set default value with GROK

2015-10-14 Thread Jochen Schalanda
Hi, you could extract that string into a dedicated message field and then use quick values to come up with a pie chart (and data table) for that field. Cheers, Jochen On Wednesday, 14 October 2015 09:49:19 UTC+2, kaiser wrote: > > Hello, > > I would like to generate charts from string value: "

[graylog2] Re: setup ElasticSearch and Graylog

2015-10-14 Thread Jochen Schalanda
Hi Zsolt, please post your Graylog server and your Elasticsearch configuration so we can take a look at them. Make sure to remove sensitive information like password_secret or MongoDB credentials before posting. Cheers, Jochen On Wednesday, 14 October 2015 16:27:25 UTC+2, Zsolt Osztrovszky wro

[graylog2] Re: Elasticsearch 2.0 and Graylog compatibility?

2015-10-15 Thread Jochen Schalanda
Hi David, that depends entirely if Elasticsearch 2.x will be compatible with Elasticsearch 1.x on a transport protocol level. So the answer is most likely: not. This being said, there's an experimental ES 2.x branch for Graylog at https://github.com/Graylog2/graylog2-server/tree/elasticsearch-

[graylog2] Re: AWS Elasticsearch

2015-10-17 Thread Jochen Schalanda
search service, can you be more > explicit? Sorry by that. > > thank you. > > On Friday, October 9, 2015 at 9:11:58 AM UTC-3, Jochen Schalanda wrote: >> >> Hi William, >> >> Graylog is currently joining the Elasticsearch cluster as a regular >> client (

[graylog2] Re: setup ElasticSearch and Graylog

2015-10-19 Thread Jochen Schalanda
Hi Zsolt, On Monday, 19 October 2015 12:13:32 UTC+2, Zsolt Osztrovszky wrote: > elasticsearch_config_file = /etc/elasticsearch/elasticsearch.yml This setting is probably the culprit. The elasticsearch_config_file setting is being used to point to an Elasticsearch configuration file to customiz

Re: [graylog2] Re: setup ElasticSearch and Graylog

2015-10-20 Thread Jochen Schalanda
Hi Zsolt, depending on the operating system you've installed Graylog on you can either use the init script (Debian Wheezy, `service graylog-server restart`), the Upstart service (Ubuntu, `restart graylog-server`), or the systemd service (Debian Jessie, `systemctl restart graylog-server`) to res

[graylog2] Re: grok pattern not working

2015-10-20 Thread Jochen Schalanda
Hi Zsolt, did you add the required Grok patterns to your Graylog system? Cheers, Jochen On Tuesday, 20 October 2015 12:56:17 UTC+2, Zsolt Osztrovszky wrote: > > Hello Guys! > I'd like to setup an extractor with Grok pattern. > This is my sample message and pattern: > 10.10.1.1 - - [13/Oct/2015:

Re: [graylog2] grok pattern not working

2015-10-21 Thread Jochen Schalanda
Hi Zsolt, that's no valid grok pattern on your screenshot. You can for example import the standard grok patterns from Logstash ( https://raw.githubusercontent.com/logstash-plugins/logstash-patterns-core/master/patterns/grok-patterns) into Graylog. Cheers, Jochen On Wednesday, 21 October 2015

[graylog2] Re: Is syslog RFC 5424 output possible with sysklogd (contains syslogd 1.4.1)

2015-10-21 Thread Jochen Schalanda
Hi Richard, the $template directive is a feature of rsyslog ( http://www.rsyslog.com/doc/v8-stable/configuration/templates.html). It seems like you're using a relatively old version of the original BSD syslogd, which doesn't support changing it's output format. Is there any chance for you to up

[graylog2] Re: Multiline logs (Java, tomcat)

2015-10-25 Thread Jochen Schalanda
Hi, for now you'd need some preprocessing of your logs (e. g. with logstash's multiline filter) to accomplish this. Alternatively you can use one of the many GELF appenders (see https://marketplace.graylog.org/addons

[graylog2] Re: Writing to mongo from a plugin

2015-10-25 Thread Jochen Schalanda
Hi Jesse, there are several possibilities to write plugin-specific data into MongoDB (none of which are documented, sorry for that). - If you can live with the overhead, you can simply inject ClusterConfigService

[graylog2] Re: How to forward syslog-ng server messages to graylog2 server ?

2015-10-26 Thread Jochen Schalanda
Hi, syslog-ng should support reloading (instead of restarting) by sending a simple SIGHUP to the syslog-ng process, e. g. by running kill -s HUP $(cat /var/run/syslog-ng.pid) Cheers, Jochen On Monday, 26 October 2015 13:17:48 UTC+1, T.J. Yang wrote: > > > Hi > > This is a beginner question. >

[graylog2] Re: How to forward syslog-ng server messages to graylog2 server ?

2015-10-26 Thread Jochen Schalanda
Hi, On Monday, 26 October 2015 19:24:47 UTC+1, T.J. Yang wrote: > > My goal is to not having graylog2-sever bind to udp/tcp 514 since my > syslog-ng server is using them already. > I am look for a way to have syslog-ng send logs graylog2 server for > processing, using port above 1024. > Is this

[graylog2] Re: Help sending logs from server journald to graylog

2015-10-26 Thread Jochen Schalanda
Hi Stephen, journal2gelf seems to be a bit out of date (last commit in 2013) but journal-gateway-gelf (https://github.com/travelping/journal-gateway-gelf) seems to be well maintained. Maybe give that project a try. Cheers, Jochen On Monday, 26 October 2015 19:50:10 UTC+1, Stephen Fox wrote: >

[graylog2] Re: Hash and split conversion examples

2015-10-26 Thread Jochen Schalanda
Hi, what exactly do you need to know? Converters can be applied to the result of an extractor and can be used to transform the input (the result of an extractor) in an arbitrary way. The Hash converter simply calculates the MD5 hash of the input and replaces the respective field with this hash

Re: [graylog2] Re: Need sample of plugin with PluginRestResource

2015-10-27 Thread Jochen Schalanda
Hello Kai, you can take a look at the Anonymous Statistics plugin which comes along with a custom JAX-RS resource. - https://github.com/Graylog2/graylog-plugin-anonymous-usage-statistics/blob/1.1.1/src/main/java/org/graylog/plugins/usagestatistics/UsageStatsResource.java - https:

[graylog2] Re: Parse Kafka syslog input

2015-10-27 Thread Jochen Schalanda
Hi Daniel, the message format cries for either a Regular Expression or a Grok Extractor. Given that the message already contains a description of the fields in the "#Fields: " line, it should be relatively straight forward to come up with a matching Grok pattern. In general, I'd recommend usin

[graylog2] Re: Create Converter

2015-10-27 Thread Jochen Schalanda
Hi Felipe, currently you would have to create a fork of Graylog (graylog2-server and graylog2-web-interface) to implement a custom converter but we plan to change this in Graylog 2.0.x (not to be confused with Graylog2…). What kind of converter would you implement? Maybe it's useful for others

[graylog2] Re: Elasticsearch 2.0 and Graylog compatibility?

2015-10-29 Thread Jochen Schalanda
Hi Mike, Graylog 1.x will *not* support Elasticsearch 2.x. Graylog 2.x will *maybe* support Elasticsearch 2.x, but it's not set in stone. Cheers, Jochen On Thursday, 29 October 2015 19:11:45 UTC+1, Mike Daoust wrote: > > I wondered if there is more information about this now that 2.0 is out? >

[graylog2] Re: Regex

2015-10-30 Thread Jochen Schalanda
Hi, this will be possible in Graylog 1.3.0. Cheers, Jochen On Friday, 30 October 2015 10:49:56 UTC+1, kaiser wrote: > > Hi, > > When creating a regex extractor, is it possible to get all occurences of a > pattern? > > For instance given a message "A B C A" would extract "A A"? > > Regards. >

[graylog2] Re: Key=value pairs of field

2015-10-30 Thread Jochen Schalanda
Hi, simply add an extractor for the field you like to parse (e. g. a "Copy Input" extractor) and assign the "Key=Value pairs to fields" converter. This will expand the key-value pairs (e. g. "a=1,b=2,c=3") into separate message fields. Cheers, Jochen On Friday, 30 October 2015 10:48:18 UTC+1,

[graylog2] Re: JAVA warning with graylog 1.2.2

2015-10-30 Thread Jochen Schalanda
Hello Yves, just add *usage_statistics_enabled = false* to your Graylog configuration file (see https://github.com/Graylog2/graylog-plugin-anonymous-usage-statistics/tree/1.1.1#configuration-options ). Cheers, Jochen On Friday, 30 October 2015 16:17:21 UTC+1, yvesloui...@gmail.com wrote: > >

[graylog2] Re: Keyword search : "Last weekend"

2015-10-31 Thread Jochen Schalanda
Hi Mehmet, Graylog is using natty (http://natty.joestelmach.com/) under the hood to parse natural language dates. As long as natty can "translate" the term, Graylog should be able to cope with it. Cheers, Jochen On Thursday, 15 October 2015 21:15:46 UTC+2, Mehmet Ali Büyükkarakaş wrote: > > >

[graylog2] Re: Get some error at Search page on Graylog Web Interface

2015-11-02 Thread Jochen Schalanda
Hi, please check the logs of your Graylog server node(s) for the reason of the internal server error (HTTP response status 500). Cheers, Jochen On Monday, 2 November 2015 08:05:55 UTC+1, Exzitep wrote: > > Hi All, > > After install graylog2 on Ubuntu 14.04. I got an error when went to > 'Sear

[graylog2] Re: Required disk space for a new graylog2 installation unter Linux...

2015-11-02 Thread Jochen Schalanda
Hi Klaus, unfortunately it's not that easy to calculate the exact disk space requirement for the given numbers. For example your log messages could be as small as a few bytes and as big as several kilobytes or even megabytes. Additionally it's important how heterogenous the log messages are. If

[graylog2] Re: JAVA warning with graylog 1.2.2

2015-11-03 Thread Jochen Schalanda
till have java warnings. > Do you know how to activate graylog debug ? > > Thanks in advance. > > Regards > Yves Louis > > Le vendredi 30 octobre 2015 17:32:21 UTC+1, Jochen Schalanda a écrit : >> >> Hello Yves, >> >> just add *usage_statistics

[graylog2] Re: Elasticsearch desactivation

2015-11-03 Thread Jochen Schalanda
Hi Yves, what exactly do you mean with "deactivate Elasticsearch in the Graylog configuration"? Since Graylog is indexing the processed log messages in Elasticsearch, it would be quite useless to deactivate the default Elasticsearch output. Cheers, Jochen On Tuesday, 3 November 2015 16:07:38

[graylog2] Re: Elasticsearch desactivation

2015-11-04 Thread Jochen Schalanda
e better if I could desactive it > > Could I do so ? > > Regards, > Yves Louis > > > > Le mardi 3 novembre 2015 18:15:29 UTC+1, Jochen Schalanda a écrit : >> >> Hi Yves, >> >> what exactly do you mean with "deactivate Elasticsearch in the Graylo

[graylog2] Re: JAVA warning with graylog 1.2.2

2015-11-04 Thread Jochen Schalanda
em. > > I still have java warning (each seconde) and micro freeze of graylog > servers. > > Regards > Yves Louis > > > Le mardi 3 novembre 2015 18:14:23 UTC+1, Jochen Schalanda a écrit : >> >> Hi Yves, >> >> you can activate DEBUG logging in Graylog

[graylog2] Re: help !!!!! for a newbie

2015-11-04 Thread Jochen Schalanda
Hi, On Wednesday, 4 November 2015 16:19:34 UTC+1, Charles Francis wrote: > > I ended up using this link for the install. > > > https://www.digitalocean.com/community/tutorials/how-to-install-graylog2-and-centralize-logs-on-ubuntu-14-04 > Holy cow, that article is quite out of date and I wouldn't

[graylog2] Re: Required disk space for a new graylog2 installation unter Linux...

2015-11-05 Thread Jochen Schalanda
How is the *relationship between* the *used > space from MongoDB (/var/lib/mongodb)* and the *space, used by > Elasticsearch (/var/lib/elasticsearch*)? > > Is it possible to sa 1/3 to 2/3 or something similar? > > > Thank you! > > Klaus. > > Am Montag, 2. November 2015

[graylog2] Re: Extractor Help

2015-11-05 Thread Jochen Schalanda
Hi Charles, I'm not aware that any such "converter" exists. Cheers, Jochen On Wednesday, 4 November 2015 21:02:20 UTC+1, Charles Francis wrote: > > Hello all, > I couldn't find it listed anywhere so I was wonder if anyone had a magic > way to take some of the information from the logstash conf

[graylog2] Re: Getting "handshake_failure" using ''graylog2-plugin-input-httpmonitor"

2015-11-05 Thread Jochen Schalanda
Hi, the best way to tackle this is probably to file a bug report in the plugin repository on GitHub: https://github.com/sivasamyk/graylog2-plugin-input-httpmonitor/issues If you're still using Java 7, you should probably upgrade to Java 8, which supports a wider range of TLS protocol versions

[graylog2] Re: Java-Exception, while protecting access to graylog-web using Apache HTTPD and .htaccess with Basic Authentication!

2015-11-05 Thread Jochen Schalanda
Hi Klaus, simple answer is: This currently can't be disabled in the Graylog web interface. If the web interface discovers that there's a Basic Auth happening "in front" of Graylog, it tries to use those credentials to log in and the error messages you've posted simply show those login attempts

[graylog2] Re: Add an hyperlink in a field

2015-11-05 Thread Jochen Schalanda
Hi Jerome, Graylog currently doesn't support this. Cheers, Jochen On Thursday, 5 November 2015 17:50:09 UTC+1, jerome wrote: > > Hi , I use graylog since 1 month > > I use JSON to send my logs to graylog > > I want to add a column call "detail" to redirect (in a new window) the > graylog user

[graylog2] Re: Required disk space for a new graylog2 installation unter Linux...

2015-11-06 Thread Jochen Schalanda
Hi Klaus, when you take a look into the "index_failures" collection, you should be able to find the reason why those messages couldn't be properly indexed. Cheers, Jochen On Thursday, 5 November 2015 14:01:26 UTC+1, kl...@tachtler.net wrote: > > Hi Jochen, > > thank you for the explanation, BU

[graylog2] Re: Troubleshooting Output messages

2015-11-06 Thread Jochen Schalanda
Hi Zach, I'm not really sure what your question is. Could you please elaborate? tcpdump can also be used to dump (well, duh!) the contents of TCP packets and not only their metadata (header fields etc.) by adding the -X parameter, see https://danielmiessler.com/study/tcpdump/ for an example.

[graylog2] Re: Error after updating to latest version.

2015-11-09 Thread Jochen Schalanda
Hi, which actions in the web interface produce those timeout messages? Can you reproduce what you did, when the timeout occurred? Cheers, Jochen On Monday, 9 November 2015 13:01:28 UTC+1, eleftherios Banos wrote: > > Hi, > > After we updated to the latest 1.2.2 version an error occurs. In det

Re: [graylog2] Re: Required disk space for a new graylog2 installation unter Linux...

2015-11-10 Thread Jochen Schalanda
Hi Klaus, I've heard from some users that their MongoDB server is using lots of space for their journal. You might want to try to set smallfiles=true in the MongoDB configuration file (see https://docs.mongodb.org/v2.4/reference/configuration-options/#smallfiles) to reduce the amount of disk s

[graylog2] Re: In charts natural number converted to float.

2015-11-11 Thread Jochen Schalanda
Hi Lefteris, "1,766" is not a decimal number but simply 1766 with the default English digit grouping (see https://en.wikipedia.org/wiki/Decimal_mark#Digit_grouping). Cheers, Jochen On Wednesday, 11 November 2015 12:50:29 UTC+1, eleftherios Banos wrote: > > Hi all, > > Although at the search r

[graylog2] Re: unable to post GLEF messages in UDP port

2015-11-11 Thread Jochen Schalanda
Hi Stefan, nmap can't really tell if a UDP socket is open or not if the listener doesn't send a reply (which the GELF UDP input doesn't). FWIW, I think this problem has been solved on IRC (GELF UDP input was bound to 127.0.0.1 instead of 0.0.0.0 or a public network interface): https://botbot.m

[graylog2] Re: How to change login screen and UI

2015-11-16 Thread Jochen Schalanda
Hi, the Graylog web interface is currently not customizable. You'll have to build your own version of the web interface (https://github.com/Graylog2/graylog2-web-interface) to customize parts of it. Cheers, Jochen On Sunday, 15 November 2015 09:06:51 UTC+1, Dương Quang Thọ wrote: > > Dear al

[graylog2] Re: graylog-web in different machine with graylog-server

2015-11-16 Thread Jochen Schalanda
Hi, make sure that the Graylog web interface on machine B can access port 12900/tcp on machine A, e. g. with curl (`curl -i http://x.x.x.x:12900/` on machine B), and that the Graylog server node on machine A is listening on the correct interface (e. g. with `netstat -tplen|grep :12900` on machi

[graylog2] Re: Graylog Collector Connection Refused

2015-11-16 Thread Jochen Schalanda
Hi Sean, which version of Graylog are you running? Also make sure that the machine running the Graylog Collector can access the Graylog REST API, e. g. by running `curl -i http://xxx.xxx.xxx.xxx:12900/` on that machine. Cheers, Jochen On Monday, 16 November 2015 10:44:59 UTC+1, Sean McGurk wr

[graylog2] Re: Graylog Collector Connection Refused

2015-11-16 Thread Jochen Schalanda
Hi Sean, those responses with HTTP status code 404 are fine, there's simply no handler defined for the root resource and it was merely meant to check the connectivity to the Graylog REST API in general. Now that we have established that communications between the machine you're running the Gra

<    1   2   3   4   5   6   7   8   9   10   >