Hello Radu,
None of the above worked.
elasticsearch.yml is blank for me. (all lines are commented).
logging.yml had the following lines
rootLogger: INFO, file
logger:
# log action execution errors for easier debugging
action: DEBUG
# reduce the logging for aws, too much is logged under the default INFO
com.amazonaws: WARN
# gateway
#gateway: DEBUG
#index.gateway: DEBUG
# peer shard recovery
#index.shard.recovery: DEBUG
# discovery
#discovery: TRACE
appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ABSOLUTE}][%-5p][%-25c] %m%n"
syslog:
type: syslog
syslogrHost: localhost:514
facility: LOCAL6
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
rootLogger: INFO, file
logger:
# log action execution errors for easier debugging
action: DEBUG
# reduce the logging for aws, too much is logged under the default INFO
com.amazonaws: WARN
# gateway
#gateway: DEBUG
#index.gateway: DEBUG
# peer shard recovery
#index.shard.recovery: DEBUG
# discovery
#discovery: TRACE
appender:
console:
type: console
layout:
type: consolePattern
conversionPattern: "[%d{ABSOLUTE}][%-5p][%-25c] %m%n"
syslog:
type: syslog
syslogrHost: localhost:514
facility: LOCAL6
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
The entries are in rsyslog.conf are
template (name="apsimTemplate" type="list" option.json="on") {
constant(value="{")
constant(value="\"@message\":\"")
property(name="msg")
constant(value="\"}")
}
*.* action(type="omelasticsearch" template="apsimTemplate"
server="localhost" serverport="9200")
Thanks
Mahesh
On Tue, Jun 18, 2013 at 4:10 PM, Radu Gheorghe <[email protected]>wrote:
> Interesting. The log is there, so it's just about getting it. The correct
> command to search for everything would be:
>
> curl 'http://localhost:9200/_search?pretty=1&q=*:*'
>
> Note the quotes around the URI - they're needed because the & sign will
> fork the thing to background and ignore the rest. But it should still work
> only with the first part:
>
> curl http://localhost:9200/_search?pretty
>
> You can also try Elasticsearch
> Head<https://github.com/mobz/elasticsearch-head>to see what's
> happening. If logs don't show with curl, try refreshing first:
> curl localhost:9200/_refresh
>
> If you still got issues, please post your elasticsearch.yml and tell what's
> your ES version.
>
> 2013/6/18 Mahesh V <[email protected]>
>
> > Hi
> > I see the following when running rsyslogd -nd (debug mode)
> >
> > 359.892976482:7f9f631cf700: omelasticsearch: result doAction: 0 (bulkmode
> > 0)
> > 1359.892983879:7f9f631cf700: Action 0x7f9f6b748300 transitioned to state:
> > rdy
> > 1359.892990396:7f9f631cf700: action 0x7f9f6b748300 call returned 0
> > 1359.892995969:7f9f631cf700: omelasticsearch: beginTransaction
> > 1359.893001589:7f9f631cf700: Action 0x7f9f6b748300 transitioned to state:
> > itx
> > 1359.893007132:7f9f631cf700: entering actionCalldoAction(), state: itx
> > 1359.894200154:7f9f631cf700: omelasticsearch: es reply:
> >
> >
> '{"ok":true,"_index":"system","_type":"events","_id":"r2M4dw28QYGr-FuacMQ-SA","_version":1}'
> >
> > which means the insert has happened.
> >
> > but when I run
> > curl http://localhost:9200/_search?pretty=1&q=*.*
> >
> > I dont see the message.
> >
> > /var/log/elasticsearch/elasticsearch.log has nothing as well.
> >
> > Where does the message go?
> >
> > thanks
> > Mahesh
> >
> >
> >
> >
> >
> > On Tue, Jun 18, 2013 at 2:21 PM, Radu Gheorghe <[email protected]
> > >wrote:
> >
> > > Hi Mahesh,
> > >
> > > Your template is suppose to output a valid JSON. Your message can be
> > > whatever you want (JSON or not). Here's a reference:
> > >
> > >
> >
> http://blog.sematext.com/2013/05/28/structured-logging-with-rsyslog-and-elasticsearch/
> > >
> > > A valid template would be this one:
> > > template (name="apsimTemplate" type="list" option.json="on") {
> > > constant(value="{")
> > > constant(value="\"@message\":\"")
> > > property(name="msg")
> > > constant(value="\"}")
> > > }
> > >
> > >
> > > Which differs only a bit from the one you pasted, which had an extra
> > quote
> > > and comma before @message:
> > > constant(value="\",\"@message\":\"") instead of
> > > constant(value="\"@message\":\"")
> > >
> > > Best regards,
> > > Radu
> > >
> > > 2013/6/18 Mahesh V <[email protected]>
> > >
> > > > Hello,
> > > >
> > > > i would like to know how to format the rsyslog message in the client
> > > > programs to get it parsed by json to be entered
> > > > in elasticsearch.
> > > >
> > > > My rsyslog.conf has the following entries
> > > >
> > > > template (name="apsimTemplate" type="list" option.json="on") {
> > > > constant(value="{") constant(value="\",\"@message\":\"")
> > > > property(name="msg") constant(value="\"}") }
> > > >
> > > > *.* action(type="omelasticsearch" template="apsimTemplate" )
> > > >
> > > >
> > > > and I use syslog with the formatted message as shown here : syslog
> > > > (LOG_NOTICE, "{ \"@message\":\"A tree falls in a forest %d\" }", i);
> > > >
> > > > But the entries do not show up in elasticsearch.log
> > > >
> > > > what am I missing?
> > > >
> > > > thanks
> > > > Mahesh
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Jun 17, 2013 at 7:16 PM, Mahesh V <
> > > [email protected]
> > > > >wrote:
> > > >
> > > > > Hello Folks,
> > > > >
> > > > > if I put the below two lines in rsyslog.conf and run a simple
> > > executable
> > > > > as shown
> > > > > below I do not get any entries in elasticsearch logs.
> > > > >
> > > > > void main()
> > > > > {
> > > > > int i = 0;
> > > > > setlogmask (LOG_UPTO (LOG_NOTICE));
> > > > > openlog ("exampleprog", LOG_PID | LOG_NDELAY, LOG_DAEMON);
> > > > > perror("openlog");
> > > > > for(i = 0; i < 500000; i++) {
> > > > > //syslog (LOG_NOTICE, "A tree falls in a forest
> %d",
> > > i);
> > > > > syslog (LOG_NOTICE, "{ \"@message\":\"A tree falls
> > in a
> > > > > forest %d\" }", i);
> > > > > //usleep(200);
> > > > > }
> > > > > closelog ();
> > > > > }
> > > > > template (name="apsimTemplate" type="list" option.json="on") {
> > > > > constant(value="{") constant(value="\",\"@message\":\"")
> > > > > property(name="msg") constant(value="\"}") }
> > > > >
> > > > > *.* action(type="omelasticsearch" template="apsimTemplate" ) #
> > > > > searchIndex="srchidx")
> > > > >
> > > > >
> > > > > whereas if I put the below line in /etc/rsyslog.conf, the lines
> > appear.
> > > > > jasonlint validates the line in syslog function.
> > > > >
> > > > > *.* /var/log/elasticsearch/elasticsearch.log
> > > > > What could be the issue?
> > > > > thanks
> > > > > Mahesh
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com/professional-services/
> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > myriad
> > > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
> you
> > > > DON'T LIKE THAT.
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> myriad
> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > > DON'T LIKE THAT.
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> > DON'T LIKE THAT.
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
