What do you think about such concept:
Syslog clients are in any UTC zone & have a sending template with
unixtimestamp:
template(name="cee" type="list") {
...
constant(value="@cee: {")
...
##ES TIMESTAMP
constant(value="\"@timestamp\":\"")
property(name="timegenerated" dateFormat="unixtimestamp")
property(name="timegenerated" dateFormat="subseconds"
position.to="3")
constant(value="\", ")
}
Server is in UTC+00 and use own $year, $month and $day as ES index
mapping:
## DEFAULT ES INDEX ##
template(name="es_index-default" type="list") {
property(name="$!msg_class" )
constant(value="-")
property(name="$!msg_view" )
constant(value="-")
property(name="$year")
constant(value=".")
property(name="$month")
constant(value=".")
property(name="$day")
}
---
Best regards,
Eugene Istomin
> Also i read about another approach:
>
> "...I think you are far better off running your servers (and the timestamps
> on the logs) on UTC. This not only avoids the problems of "where is this
> server located, so what time zone is it in", but it also doesn't have
> daylight savings time changes (with all the related problems of jobs
> running multiple times or not at all)"
>
> Well, sounds good. But needs additional configuration in every
> time-dependent application (DB/schedulers/etc).
> ---
> Best regards,
> Eugene Istomin
>
> On Thursday, July 17, 2014 10:49:02 PM Eugene Istomin wrote:
> > Can you describe a little bit two 8.3.4 changelog lines?
> >
> > + new parser config object — permits to define custom parser
definitions
> > + new tzinfo config object — permits to define time zone offsets
> > /---/
> > */Best regards,/*
> > /Eugene Istomin/
> >
> > > Hello Rainer,
> > >
> > > during implementing of the flexible ES templates we are stopped by
time
> >
> > zone
> >
> > > problem, let's me describe:
> > >
> > > 1) We have a message template like:
> > >
> > > ##CEE TEMPLATE
> > > template(name="cee" type="list") {
> > > ...
> > > constant(value="@cee: {")
> > > ...
> > >
> > > property(name="timegenerated" dateFormat="rfc3339"
> >
> > format="jsonf"
> >
> > > outname="@timestamp") constant(value=", ") ...
> > >
> > >
> > > 2) Sending this message using RELP:
> > >
> > > ruleset(name="relp_cee") {
> > >
> > > action(type="omrelp" Template="cee" Target="core"
Port="20514")
> > >
> > > }
> > >
> > >
> > > 3) Log server is receiving this message:
> > >
> > > input(type="imrelp" Port="20514")
> > > action(type="mmjsonparse")
> > >
> > > if $parsesuccess == "OK" then {
> > >
> > > action(type="omelasticsearch" server="localhost"
> >
> > template="ES-All"
> >
> > > searchIndex="es_index-default" searchType="events"
> >
> > dynSearchIndex="on"
> >
> > > bulkmode="on" queue.dequeuebatchsize="5000"
queue.size="100000"
> > > queue.workerthreads="5" ); }
> > >
> > >
> > > and puts to ES using index
> > >
> > > ## ES Index template
> > > template(name="es_index-default" type="list") {
> > >
> > > property(name="$!msg_class" ) constant(value="-")
> > > property(name="$!msg_view" ) constant(value="-")
> > > property(name="@timestamp" dateFormat="rfc3339"
> >
> > position.from="1"
> >
> > > position.to="4") constant(value=".") property(name="@timestamp"
> > > dateFormat="rfc3339" position.from="6" position.to="7")
> >
> > constant(value=".")
> >
> > > property(name="@timestamp" dateFormat="rfc3339"
position.from="9"
> > > position.to="10") }
> > >
> > >
> > > Let's date will be 2014-07-01T01:30:00.000000+03:00
> > >
> > > if we extract template index using "es_index-default" - we'll get
> > > "X-Y-2014-07-01". But in fact, the correct date relatively to ES
> > > timestamp
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
