Heka community
I am using the following config file (copy from heka doc) to decode the Apache
access log.
[TestWebserver]
type = "LogstreamerInput"
log_directory = "/opt/heka/tmp"
file_match = 'access\.log'
decoder = "CombinedLogDecoder"
[CombinedLogDecoder]
type = "SandboxDecoder"
filename = "lua_decoders/apache_access.lua"
[CombinedLogDecoder.config]
type = "combined"
user_agent_transform = true
# common log format
log_format = '%h %l %u %t \"%r\" %>s %O'
# combined log format
# log_format = '%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"'
# vhost_combined log format
# log_format = '%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\"
\"%{User-Agent}i\"'
# referer log format
# log_format = '%{Referer}i -> %U'
[RstEncoder]
[FileOutput]
message_matcher = "TRUE"
encoder = "RstEncoder"
path = "/tmp/zhekaOut"
The decoder encounters parsing error for some messages. For example the decoder
parse the following message correctly:
10.56.22.105 - - [25/Aug/2015:00:09:55 +0000] "GET /secure/common/begin2.do
HTTP/1.1" 200 1466 6803 +
I can examine the RstEncoder output as follow:
:Fields:
| name:"remote_addr" type:string value:"10.56.22.105" representation:"ipv4"
| name:"response_length" type:double value:1466 representation:"B"
| name:"request" type:string value:"GET /secure/common/begin2.do HTTP/1.1"
| name:"remote_user" type:string value:"-"
| name:"status" type:double value:200
But I got parsing failure for the following message:
2015/08/28 20:29:11 Decoder 'TestWebserver-CombinedLogDecoder-1' error: Failed
parsing: payload: 10.56.22.105 - - [25/Aug/2015:00:09:55 +0000] "GET
/secure/static/css/main.css HTTP/1.1" 304 - 1390 +
My suspicious is the "response_length" is now not numeric but a character "-",
instead of "1466" in previous message. I tried the other log_format above but
got the same errors. How can I modify the log_format so that it can handle both
cases?
Thanks.
Ming
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
