Hi,
Please allow me to rephrase my question. It may not have been clear the first
time.
I have many folders as below with files as shown.
/var/log/xxx/jack/program.log.1
/var/log/xxx/jack/program.log.2
/var/log/xxx/john/program.log.1
/var/log/xxx/john/program.log.3
/var/log/xxx/sam/program.log.1
...
The LogStreamerInput is shown below and I have tested it - it works just fine.
[SomethingLogInput]
type = "LogstreamerInput"
log_directory = "/var/log/xxx"
file_match = '(?P<something>\s+)/program\.log\.?(?P<Seq>\d*)'
differentiator = ["something", "program","Seq"]
decoder = "SomethingLogDecoder"
The SomethingLogDecoder is below and I have verified that it works for the
fields which are in the match_regex i.e comp and host
[SomethingLogDecoder]
type = "PayloadRegexDecoder"
match_regex = '(?P<comp>[^(]+\w+)\@(?P<host>\D+[^)]).*node[(]+.*\.'
[SomethingLogDecoder.message_fields]
Type = "erlang"
comp = '%comp%'
host = '%host%'
seq = '%Seq%'
logger = '%Component%'
I would like the ElasticSearchJSONEncoder to have access to the "something"
field which was used as a differentiator in the Input phase. I need it since it
gives me information if it was the "jack" or "john" or "sam" folder the
messages came from. But I have not been able to figure out a way to do this
i.e. have the encoder include fields from the input phase. Would anyone be able
to help me out here?
Thanks!
From: Heka [mailto:[email protected]] On Behalf Of Satish Anupindi Rao
Sent: Thursday, August 04, 2016 1:28 PM
To: [email protected]
Subject: [heka] Heka input differentiator fields to decoder
Hi!
I have many folders with files as follows :
/var/log/xxx/jack/program.log.1
/var/log/xxx/jack/program.log.2
/var/log/xxx/john/program.log.1
/var/log/xxx/john/program.log.3
/var/log/xxx/sam/program.log.1
...
I would like to pass the "something" field to the decoder , so that the message
fields of the decoder also have access to the "something" fields. Is there a
way for me to do this ?
I post these fields via a REST api (using an elasticsearchoutput) and store
them in a mongo DB. I would like the "something" field also stored.
[SomethingLogInput]
type = "LogstreamerInput"
log_directory = "/var/log/xxx"
file_match = '(?P<something>\s+)/program\.log\.?(?P<Seq>\d*)'
differentiator = ["something", "erlang","Seq"]
decoder = "SomethingLogDecoder"
Thanks!
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
