Simon Elliston Ball created METRON-691:
------------------------------------------
Summary: Elastic Writer index partitions on system time, not event
time
Key: METRON-691
URL: https://issues.apache.org/jira/browse/METRON-691
Project: Metron
Issue Type: Bug
Affects Versions: 0.3.0
Reporter: Simon Elliston Ball
Currently the elastic writer determines the index destination for messages
based on system time, rather than message time. As a consequence, around time
boundaries, where there is more than a small lag in the topologies, an event
can end up in the wrong index.
This means the event is ignored in Kibana dashboard, which quite sensible limit
the indices consulted, but filter on the exact timestamp.
To reproduce this, index an older event, and note that a current time index is
created. Searching within the actual event time period will not find the event,
because it consults the wrong index. Searching within the index period will
also not return the event due to the filtering on the actual event timestamp
field.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
