[graylog2] How to set additional fields for sidecar/filebeat

[Evgueni Gordienko]

While configuring 'Configure Beats Input' there is a possibility to add 
additional field.
I tried to set bulk_max_size equal to 8192 but it looks like it does not 
make any effect - I couldn't
find it in any config file.
In general - how to set additional fields for sidecar/filebeat?

Thanks,
Evgueni

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/9ff906ca-d93a-46eb-8e3a-4bee8f7751b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Error Initialising publisher: No outputs are defined. Please define one under the output section in graylog collector sidecar and filebeat

[GiangCoi Mr]

Yes. in file config .yml. I configured tag "apache" but it doesn't working.

On Monday, September 26, 2016 at 2:43:23 PM UTC+7, Jochen Schalanda wrote:
>
> Hi,
>
> did you assign the correct tags ("apache") in your collector_sidecar.yml 
> file? See 
> http://docs.graylog.org/en/2.1/pages/collector_sidecar.html#configuration for 
> details.
>
> Cheers,
> Jochen
>
> On Monday, 26 September 2016 05:47:41 UTC+2, GiangCoi Mr wrote:
>>
>> Hi all
>>
>> I started with this instruction 
>> http://docs.graylog.org/en/2.1/pages/collector_sidecar.html 
>> 
>>  
>> for configure Collector Sidecar and Filebeat. I install Collector sidecar 
>> in Client server in ubuntu, after I configure Graylog as in this 
>> instruction, I configured to collected Apache logs and Graylog show status 
>> "running", but when I click show message: Nothing found
>>
>>
>>
>> I showed log in collector sidecar, it show
>>
>> Error Initialising publisher: No outputs are defined. Please define one 
>> under the output section.
>> Error Initialising publisher: No outputs are defined. Please define one 
>> under the output section.
>> Error Initialising publisher: No outputs are defined. Please define one 
>> under the output section.
>> Error Initialising publisher: No outputs are defined. Please define one 
>> under the output section.
>> Error Initialising publisher: No outputs are defined. Please define one 
>> under the output section.
>>
>> I configured this follow picture
>>
>> I don't know why Graylog server don't show log in Client server, please 
>> help me to fix it. Thanks.
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/267aeeef-0a30-4e13-a6e4-7b61a1a36a92%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] ELastic search for logs in graylog server

[sam]

Hi All,

As said like elastic search is responsible for storing the logs. Other than 
graylog web interface, is there anyway I could look for those stored logs 
in elastic search (where the logs are stored) URL (If there such kind of 
search exists) 

I am trying to look for the logs that are being stored in elastic search. 

And mongo db stores configuration data ,  why do we need this for graylog 
server, as we don't configure anything during gray log installation ? 




Thank you 
Sam

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/9b2815f5-14ce-49b5-9ba4-1ee52a01b329%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Reference Configuration with Graylog

[Jochen Schalanda]

Hi Evgueni,

see 
http://docs.graylog.org/en/2.1/pages/architecture.html#bigger-production-setup 
for an example HA production setup.

Cheers,
Jochen

On Monday, 26 September 2016 21:14:33 UTC+2, Evgueni Gordienko wrote:
>
> Hi All,
>
> Is there any suggested/reference configuration with Graylog which does not 
> have single point of failure?
>
> Thanks,
> Evgueni
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/719d2d43-167b-4463-b0eb-3bc9f6098bf5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Reference Configuration with Graylog

[Evgueni Gordienko]

Hi All,

Is there any suggested/reference configuration with Graylog which does not 
have single point of failure?

Thanks,
Evgueni

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/06da90b0-c78e-47d8-9cf2-08a7bc92987f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: graylog search head

[max xu]

Thanks Jochen!

On Mon, Sep 26, 2016 at 12:35 AM, Jochen Schalanda 
wrote:

> Hi Max,
>
> unless your Graylog Cluster spans all data centers (not recommended,
> though), that's currently not possible.
>
> Cheers,
> Jochen
>
> On Friday, 23 September 2016 23:54:09 UTC+2, max xu wrote:
>>
>> Hi,
>>
>> Our environment is distributed (multiple datacenters). Our search will
>> need to cover all of them. How does Graylog support distributed search
>> (like spunk search head)?
>>
>> Thanks,
>> -max
>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Graylog Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/graylog2/B92DJo0uGiQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/graylog2/25d35f06-476d-44b5-864b-6ec892800291%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAKi0KzsC-ktVFn2DY_LPp3YCLaCWyerz3AKF-507fxOLOSOuqA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Elasticsearch bulk api with gelf http input

[eugen.biegler via Graylog Users]

Hello Jochen,

thank you for the quick response. I was looking for a way to send bulk data 
in and since an gelf input comes next to sending it directly into 
elasticsearch i thought there could be a sort of a mechanism which can use 
the es bulk api

Am Montag, 26. September 2016 16:50:11 UTC+2 schrieb Jochen Schalanda:
>
> Hi,
>
>
> On Monday, 26 September 2016 15:20:24 UTC+2, eugen@googlemail.com 
> wrote:
>>
>> could it be that the gelf http input type is not able to handle bulk 
>> requests?
>>
>
> Yes, that's correct. Why did you think it would support this?
>
> Cheers,
> Jochen
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/6987dd13-a86a-4705-9971-bcb034d75d8e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Web interface not starting-Graylog-v2.1.0

[Jochen Schalanda]

Hi Shrawan,

that's still not the required information. Please adapt the command to your 
MongoDB configuration (localhost/127.0.0.1 is obviously not the correct 
MongoDB server).

Cheers,
Jochen

On Monday, 26 September 2016 16:19:06 UTC+2, Shrawan Bhagwat wrote:
>
> Hi Jochen,
>
> I am getting following after running below mentioned command
> command:
> echo 'db.serverStatus()' | ./mongo > server-status.txt
>
> Log:
>
> MongoDB shell version: 3.2.9
> connecting to: test
> 2016-09-26T19:47:17.598+0530 W NETWORK  [thread1] Failed to connect to 
> 127.0.0.1:27017, reason: errno:111 Connection refused
> 2016-09-26T19:47:17.602+0530 E QUERY[thread1] Error: couldn't connect 
> to server 127.0.0.1:27017, connection attempt failed :
> connect@src/mongo/shell/mongo.js:229:14
> @(connect):1:6
>
> Regards,
> Shrawan
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/a6dc79ec-b161-412c-a554-00b1f5190255%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Elasticsearch bulk api with gelf http input

[Jochen Schalanda]

Hi,


On Monday, 26 September 2016 15:20:24 UTC+2, eugen@googlemail.com wrote:
>
> could it be that the gelf http input type is not able to handle bulk 
> requests?
>

Yes, that's correct. Why did you think it would support this?

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/20b528e1-4b00-476c-85d1-7bd28c8fccab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] "Did not find meta info of this node. Re-registering." on single server setup

[julioqc47]

Hello,

My logs are filled with those since upgrading from 2.0.3 to 2.1.1 OVA setup.


2016-09-26 10:08:52,636 WARN : org.graylog2.periodical.NodePingThread - Did 
not find meta info of this node. Re-registering.
2016-09-26 10:10:45,387 WARN : org.graylog2.periodical.NodePingThread - Did 
not find meta info of this node. Re-registering.
2016-09-26 10:12:06,493 WARN : org.graylog2.periodical.NodePingThread - Did 
not find meta info of this node. Re-registering.
2016-09-26 10:12:10,058 WARN : org.graylog2.periodical.NodePingThread - Did 
not find meta info of this node. Re-registering.
2016-09-26 10:12:47,145 WARN : org.graylog2.periodical.NodePingThread - Did 
not find meta info of this node. Re-registering.
2016-09-26 10:13:22,589 WARN : org.graylog2.periodical.NodePingThread - Did 
not find meta info of this node. Re-registering.
2016-09-26 10:20:09,306 WARN : org.graylog2.periodical.NodePingThread - Did 
not find meta info of this node. Re-registering.


I only have a single server so I'm assuming NTP is not to blame (it is 
configured anyways)

MongoDB seems ok to me as well:

{
"_id" : ObjectId("57e92e9a0ae2f10632ba3ba1"),
"is_master" : true,
"hostname" : "graylog",
"last_seen" : 1474901038,
"transport_address" : "http://192.168.xx.xxx:9000/api/";,
"type" : "SERVER",
"node_id" : "b0b6b61e-aaab-42a2-af6e-2aabfdb57370"
}


Any other leads on to what can cause this behaviour? 

Thank you for your time. 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/9b32cf21-1090-4934-8835-c204c37f3d74%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Processing of stream failed to return within 2000ms.

[julioqc47]

Hello,

So recently I've been getting those errors followed obviously by a 
disabling of the stream in question. 
I wouldn't however expect this to happen since the rules are rather simple 
(for WinEvents collected by sidecar):


   - *EventID* must match exactly *4656*
  - *SubjectUserName* must match regular expression *.*\$$*
  - *EventType* must match exactly *AUDIT_FAILURE*
   

Regex debug returned either an immediate fail or a 6 step match depending 
on the possible nature of '*SubjectUserName* ' within the timeframe the 
issue occurred. (it, user1, admin, PC01$, Server02$)

What else could cause this to fail? 

As a 'coincidence', the the journal filled up to maximum capacity (and 
failed) really quickly during the same period due to spikes in events at 
that time (expected) so I adjusted the journal 
size, processbuffer_processors and outputbuffer_processors in hopes it will 
solve that part.

However, can both events be related? If so, how? I'm not sure how the 
journal issue can lead to the stream processing issue.

Thank you for your time. 

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/36a7039f-5cd4-46d9-8688-f81c1c259a83%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Web interface not starting-Graylog-v2.1.0

[Shrawan Bhagwat]

Hi Jochen,

I am getting following after running below mentioned command
command:
echo 'db.serverStatus()' | ./mongo > server-status.txt

Log:

MongoDB shell version: 3.2.9
connecting to: test
2016-09-26T19:47:17.598+0530 W NETWORK  [thread1] Failed to connect to 
127.0.0.1:27017, reason: errno:111 Connection refused
2016-09-26T19:47:17.602+0530 E QUERY[thread1] Error: couldn't connect 
to server 127.0.0.1:27017, connection attempt failed :
connect@src/mongo/shell/mongo.js:229:14
@(connect):1:6

Regards,
Shrawan

On Friday, 23 September 2016 19:17:20 UTC+5:30, Jochen Schalanda wrote:
>
> Hi Shrawan,
>
> that's not the command I mentioned.
>
> Cheers,
> Jochen
>
> On Friday, 23 September 2016 15:41:22 UTC+2, Shrawan Bhagwat wrote:
>>
>> Hi Jochen,
>>
>> i have got following output after executing that command:
>>
>>  ./mongo 192.168.178.228/Graylog
>> MongoDB shell version: 3.2.9
>> connecting to: 192.168.178.228/Graylog
>> Server has startup warnings:
>> 2016-09-23T18:52:38.126+0530 I CONTROL  [initandlisten] ** WARNING: You 
>> are running this process as the root user, which is not recommended.
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten]
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten]
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten] ** WARNING: 
>> /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten] **We 
>> suggest setting it to 'never'
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten]
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten] ** WARNING: 
>> /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten] **We 
>> suggest setting it to 'never'
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten]
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten] ** WARNING: soft 
>> rlimits too low. rlimits set to 1024 processes, 65536 files. Number of 
>> processes should be at least 32768 : 0.5 times number of files.
>> 2016-09-23T18:52:38.127+0530 I CONTROL  [initandlisten]
>> MongoDB Enterprise > ^C
>> bye
>>
>> Regards,
>> Shrawan
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/fb432b31-c3cd-46e3-8620-6cea60b4b54e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Elasticsearch bulk api with gelf http input

[eugen.biegler via Graylog Users]

Hi,

could it be that the gelf http input type is not able to handle bulk 
requests? For example:

T 127.0.0.1:38721 -> 127.0.0.1:12201 [AP] 
POST /gelf HTTP/1.1. 
Host: 127.0.0.1:12201. 
Accept: */*. 
Content-Type: application/json; charset=UTF-8. 
Content-Length: 279. 
. 
{"host":"debian-4gb-fra1","timestamp":"1474895667","name":"stats","value":5,"short_message":"Stats","full_message":"More-Stats"}
 

{"host":"debian-4gb-fra1","timestamp":"1474895667","name":"metric","value":10,"short_message":"Metrics","full_message":"More-
Metrics"} 


Thank you!

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/5e276c5d-f1d6-48d6-b162-d55a775f7b77%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: nginx reverse proxy messing up gl2_remote_ip

[Daniele]

Hi Jochen,

thanks for your reply, the host is set:

{
  "version": "1.1",
  "host": "demo.server-name.de",
  "short_message": "A short message that helps you identify what is going 
on...",
  "full_message": "trace",
  "timestamp": "2016-09-19T15:58:25.237Z",
  "level": 1,
  "_application": "demo-application",
  "_environment": "integration",
  "_userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, 
like Gecko) Ubuntu Chromium/51.0.2704.79 Chrome/51.0.2704.79 Safari/537.36"
}

I used a sample host-name here as example. I am not able to set the 
client-ip to the host field, because the javascript app has no access to 
the client ip.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/af26857a-c5bb-4b4a-b2d7-f22fa33936fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: nginx reverse proxy messing up gl2_remote_ip

[Jochen Schalanda]

Hi Daniele,

your client has to set the "host" field of the GELF message 
correctly: 
http://docs.graylog.org/en/2.1/pages/gelf.html#gelf-format-specification

Cheers,
Jochen

On Monday, 26 September 2016 10:50:17 UTC+2, Daniele wrote:
>
> Hi,
>
> we're using nginx as reverse proxy in front of our graylog server. This 
> works fine beside logging the clients IP address.
> Following rule adds the IP address to our GELF http input:
>
> rule "add ip"
> when
> true
> then
> set_field("ip", to_ip($message.gl2_remote_ip));
> end
>
> If we post data to the input-ip / port combination 
> (http://<*ip>*:*12202*/gelf) 
> it works as expected.
>
> We configured our nginx as reverse proxy as follows:
>
> server {
>listen 443 ssl;
>server_name log-input.server-name.de;
>
>ssl_certificate_key /etc/ssl/server.key;
>ssl_certificate /etc/ssl/server.crt;
>
>error_log /var/log/nginx/error.log debug;
>
>location / {
>proxy_pass http://127.0.0.1:12202;
>proxy_redirect off;
>
>proxy_set_header Host $host;
>proxy_set_header X-Real-IP $remote_addr;
>proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>proxy_set_header X-Forwarded-Host   $host:443;
>
>proxy_set_header X-Forwarded-Server $host;
>proxy_set_header X-Forwarded-Port   443;
>proxy_set_header X-Forwarded-Proto  https;
>proxy_set_header X-Scheme $scheme;
>
># testing remote addr combinations
>proxy_set_header REMOTE_ADDR $remote_addr;
>proxy_set_header Remote_Addr $remote_addr;
># /testing remote addr combinations
>
>proxy_pass_request_headers on;
>proxy_connect_timeout 150;
>proxy_send_timeout 100;
>proxy_read_timeout 100;
>proxy_buffers 4 32k;
>client_max_body_size 8m;
>client_body_buffer_size 128k;
>}
> }
>
>
>
> Unfortunately the logging result with post-requests to *https://log-input 
> ..de/gelf* shows 127.0.0.1 as IP address, 
> instead of the real client *$remote_addr*.
> The activated debug log (*/var/log/nginx/error.log*) shows the IP address 
> correctly. It seems that *gl2_remote_ip* does not handle HTTP headers as 
> expected. 
>
> Has someone a solution for our problem?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/93ed7b59-2c8e-4566-928e-34341d6f6d9b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: nginx reverse proxy messing up gl2_remote_ip

[Daniele]

I also added the *trusted_proxy* configuration in *server.conf* as 
described here: http://docs.graylog.org/en/2.1/pages/securing.html

# Comma separated list of trusted proxies that are allowed to set the 
client address with X-Forwarded-For
# header. May be subnets, or hosts.
trusted_proxies = 127.0.0.1/32, 0:0:0:0:0:0:0:1/128


-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/c9a25000-ab7a-4467-ab71-8c2ba5c4adb8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Grok filter issue

[Kunal Patil]

​Hello

Graylog 2.1 is successfully installed and working flawlessly thanks to you
guys
im done with installation part and im able to received mysql slow log
successfully
im using grok pattern for parsing the data

i want to create conditional grok filter for table name in mysql data
i search on different sites for it but all solution which available is for
LOGSTASH and it is not running on graylog server
can u guide me with my requirement or suggest me different approach ?

Follwing is the sample log
# Time: 2016-09-25T21:36:23.964093+05:30 # User@Host: analytics[analytics]
@
​IP​
Id: 13394907 # Schema: justbuylive Last_errno: 0 Killed: 0 # Query_time:
1.042817 Lock_time: 0.000129 Rows_sent: 393069 Rows_examined: 393069
Rows_affected: 0 # Bytes_sent: 77760957 SET timestamp=1474819583;
select * from justdat.`*daily_order_sales*`; ​ similar with this example

​insert into justdat.brands_order_sales

it shall match pattern and filter table name

It would be great if you guys help me with it ​





*REGARDS:KUNAL VIKAS PATIL9860265594*

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAJa2o%3D_ENTxDExsfFgywP_Th69w8jeyqNz%2BMUh7t9wkLp5ZG%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] nginx reverse proxy messing up gl2_remote_ip

[Daniele]

Hi,

we're using nginx as reverse proxy in front of our graylog server. This 
works fine beside logging the clients IP address.
Following rule adds the IP address to our GELF http input:

rule "add ip"
when
true
then
set_field("ip", to_ip($message.gl2_remote_ip));
end

If we post data to the input-ip / port combination (http://<*ip>*:*12202*/gelf) 
it works as expected.

We configured our nginx as reverse proxy as follows:

server {
   listen 443 ssl;
   server_name log-input.server-name.de;

   ssl_certificate_key /etc/ssl/server.key;
   ssl_certificate /etc/ssl/server.crt;

   error_log /var/log/nginx/error.log debug;

   location / {
   proxy_pass http://127.0.0.1:12202;
   proxy_redirect off;

   proxy_set_header Host $host;
   proxy_set_header X-Real-IP $remote_addr;
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   proxy_set_header X-Forwarded-Host   $host:443;

   proxy_set_header X-Forwarded-Server $host;
   proxy_set_header X-Forwarded-Port   443;
   proxy_set_header X-Forwarded-Proto  https;
   proxy_set_header X-Scheme $scheme;

   # testing remote addr combinations
   proxy_set_header REMOTE_ADDR $remote_addr;
   proxy_set_header Remote_Addr $remote_addr;
   # /testing remote addr combinations

   proxy_pass_request_headers on;
   proxy_connect_timeout 150;
   proxy_send_timeout 100;
   proxy_read_timeout 100;
   proxy_buffers 4 32k;
   client_max_body_size 8m;
   client_body_buffer_size 128k;
   }
}



Unfortunately the logging result with post-requests to 
*https://log-input..de/gelf* shows 127.0.0.1 as IP address, 
instead of the real client *$remote_addr*.
The activated debug log (*/var/log/nginx/error.log*) shows the IP address 
correctly. It seems that *gl2_remote_ip* does not handle HTTP headers as 
expected. 

Has someone a solution for our problem?

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/ec4d001e-fac6-4b23-9d09-b15ac9583aae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Reset index number

[Aykisn]

Thanks, it is indeed only an aesthetic choice, still good to know.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/41c7f099-ed5c-4f17-a7bb-cbafab5ed003%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Using the Beats inputs - forwarder configuration

[Jochen Schalanda]

Hi Chris,

On Saturday, 24 September 2016 21:11:22 UTC+2, Chris wrote:
>
> I am curious as to what the beats forwarders configuration should be. I am 
> used to using Logstash to parse logs before sending to Elasticsearch so I 
> am wondering how the Graylog Beats input works? Is the data sent directly 
> to Elasticsearch so the forwarder output should be Elasticsearch. I ask 
> this as I noticed a comment on the plugin from Joschi saying that Logstash 
> is the correct output.
>

Through an unfortunate series of events, the output for the Beats protocol 
(ex-Lumberjack protocol) is called "logstash" in *beats. So if you want to 
use Graylog to ingest these messages, you have to use a "logstash" output 
in your beats configuration.

Cheers,
Jochen

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/5aaadfd9-2523-480d-8400-a0236a696771%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Error Initialising publisher: No outputs are defined. Please define one under the output section in graylog collector sidecar and filebeat

[Jochen Schalanda]

Hi,

did you assign the correct tags ("apache") in your collector_sidecar.yml 
file? See 
http://docs.graylog.org/en/2.1/pages/collector_sidecar.html#configuration for 
details.

Cheers,
Jochen

On Monday, 26 September 2016 05:47:41 UTC+2, GiangCoi Mr wrote:
>
> Hi all
>
> I started with this instruction 
> http://docs.graylog.org/en/2.1/pages/collector_sidecar.html 
> 
>  
> for configure Collector Sidecar and Filebeat. I install Collector sidecar 
> in Client server in ubuntu, after I configure Graylog as in this 
> instruction, I configured to collected Apache logs and Graylog show status 
> "running", but when I click show message: Nothing found
>
>
>
> I showed log in collector sidecar, it show
>
> Error Initialising publisher: No outputs are defined. Please define one 
> under the output section.
> Error Initialising publisher: No outputs are defined. Please define one 
> under the output section.
> Error Initialising publisher: No outputs are defined. Please define one 
> under the output section.
> Error Initialising publisher: No outputs are defined. Please define one 
> under the output section.
> Error Initialising publisher: No outputs are defined. Please define one 
> under the output section.
>
> I configured this follow picture
>
> I don't know why Graylog server don't show log in Client server, please 
> help me to fix it. Thanks.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/c3456acd-2b33-4c7e-a232-c47604188be0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: MongoDB issues

[Jochen Schalanda]

Hi Werner,

make sure that the node ID (
https://github.com/Graylog2/graylog2-server/blob/master/misc/graylog.conf#L49-L51)
 
didn't change during the upgrade of that Graylog installation.

Inputs are bound to the node ID, so changing the node ID means, that inputs 
for that node won't be found.

Cheers,
Jochen

On Saturday, 24 September 2016 09:43:35 UTC+2, Werner van der Merwe wrote:
>
> I am fairly sure I am missing something obvious here..
> I've upgraded another site's Graylog instance, but having some issues with 
> mongodb:
>
> grep mongo /etc/graylog/server/server.conf
> mongo_uri = mongodb://127.0.0.1/graylog2
>
> Yet none of my inputs / saved searches and dashboards appear:
>
>  elasticsearch]# mongo 127.0.0.1/graylog2
> MongoDB shell version: 2.6.12
> connecting to: 127.0.0.1/graylog2
> > db.saved_searches.count()
> 1
> > db.dashboards.count()
> 3
>
>
> Not sure what I am missing?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/887192ee-2f43-4307-8f2d-17362d7f2343%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Re: unable to figure out permissions using REST API

[Jochen Schalanda]

Hi Jason,

roles have a set of permissions, so it's just another layer of indirection.

The Graylog web interface currently doesn't support assigning fine-granular 
permissions to roles, so you have to use the Graylog REST API 
directly: 
http://docs.graylog.org/en/2.1/pages/users_and_roles/system_users.html#creating-the-role

Cheers,
Jochen

On Saturday, 24 September 2016 01:20:26 UTC+2, Jason Haar wrote:
>
>
> On Fri, Sep 23, 2016 at 6:48 PM, Jochen Schalanda wrote:
>
>> the required permissions are:
>>
>>- 
>>
>> ?? What are these "permissions" you talk about :-)
>
> ie looking at the Authentication GUI, it says setting permissions is 
> deprecated and I should use Roles instead. Is that a mistake? Also I can't 
> see how to add permissions against the account - is this action not 
> supported through the GUI?
>
> Thanks
>
>
>
> -- 
> Cheers
>
> Jason Haar
> Information Security Manager, Trimble Navigation Ltd.
> Phone: +1 408 481 8171
> PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/861dc599-fcc5-49a1-a5a5-52d367fd1c7e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: graylog search head

[Jochen Schalanda]

Hi Max,

unless your Graylog Cluster spans all data centers (not recommended, 
though), that's currently not possible.

Cheers,
Jochen

On Friday, 23 September 2016 23:54:09 UTC+2, max xu wrote:
>
> Hi,
>
> Our environment is distributed (multiple datacenters). Our search will 
> need to cover all of them. How does Graylog support distributed search 
> (like spunk search head)?
>
> Thanks,
> -max
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/25d35f06-476d-44b5-864b-6ec892800291%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: What is this message ID actually called?

[Jochen Schalanda]

Hi,

the UUID you've mentioned is the internal document ID of the message. You 
can search for it with the following search query:

_id:49ff64f1-81ca-11e6-bb22-bc764e119bb9


Cheers,
Jochen

On Friday, 23 September 2016 22:22:34 UTC+2, 8bits...@gmail.com wrote:
>
>
> 
>
> I cannot seem to search for "49ff64f1-81ca-11e6-bb22-bc764e119bb9" to find 
> this message but as the screen shot shows, it's there.  Does anyone how do 
> I store for this particular string?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/2da00872-1f0b-479c-bce5-ccd0b50003ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Sending a link to the log entry that triggered an alert.

[Jochen Schalanda]

Hi,

alerts are always generated for a time frame which can contain multiple 
matching messages.

In other words, there is no single message that you could link to.

Cheers,
Jochen

On Friday, 23 September 2016 20:41:59 UTC+2, 8bits...@gmail.com wrote:
>
> I'm using the HipChat plugin to send alerts if a field value is a certain 
> number.  I would the message to contain a URL to the message just like 
> clicking Permalink would generate when looking at the message.  Anyone know 
> how to accomplish this?
>
> Here is my message template as it stands now:
>
> ${if stream_url}${end}
> Alert for ${stream.title}
> ${if stream_url}
> 
> ${end}
> (${check_result.triggeredCondition})
> 
> ${check_result.resultDescription}, triggered at 
> ${check_result.triggeredAt}
> 
> ${if backlog}Last messages accounting for this alert:
> 
> My FieldDetails
> ${foreach backlog message}
> 
> ${message.fields.myField}
> ${message.source}, ${message.id}
> 
> 
> ${end}
> ${else}
> (No messages to display.)
> ${end}
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/6c87047e-78d7-4f85-97ed-643fb7965fa5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Re: Reset index number

[Jochen Schalanda]

Hi Aykisn,

the index number is really just an internal counter and doesn't have to 
concern you.

If you really have to reset that counter for *aesthetic* reasons, you can 
do so by doing the following:

   - Stop Graylog
   - Delete all Elasticsearch indices with the Graylog index prefix: 
   
https://github.com/Graylog2/graylog2-server/blob/2.1.1/misc/graylog.conf#L245-L246
   - Delete the Elasticsearch index alias "graylog_deflector" (given the 
   elasticsearch_index_prefix is "graylog_")
   - Start Graylog

Cheers,
Jochen

On Monday, 26 September 2016 08:59:48 UTC+2, Aykisn wrote:
>
> Hello,
>
> I did a lost of indices manual cycles to test some things, and as a 
> result, my actual index number is quite high.
> I was wondering if there was any way to reset that number please ? (all 
> previous indexes have been deleted)
>
> Thank you.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/cf6add01-5d4c-4f71-ab20-0817e904cdac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Reset index number

[Aykisn]

Hello,

I did a lost of indices manual cycles to test some things, and as a result, 
my actual index number is quite high.
I was wondering if there was any way to reset that number please ? (all 
previous indexes have been deleted)

Thank you.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/58b40b77-7f7c-4578-af59-fccd7c003beb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.