[
https://issues.apache.org/jira/browse/METRON-815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15953437#comment-15953437
]
ASF GitHub Bot commented on METRON-815:
---------------------------------------
Github user JonZeolla commented on a diff in the pull request:
https://github.com/apache/incubator-metron/pull/503#discussion_r109411418
--- Diff: metron-deployment/roles/sensor-stubs/templates/start-bro-stub ---
@@ -47,8 +47,8 @@ TOPIC="bro"
while true; do
# transform the bro timestamp and push to kafka
- SEARCH="\"ts\"\:[0-9]\+.[0-9]\{6\}"
- REPLACE="\"ts\"\:`date +%s`.000000"
+ SEARCH="\"ts\"\:[0-9]\+\."
+ REPLACE="\"ts\"\:`date +%s`\."
--- End diff --
Bro timestamps are often out of order depending on the log because some
lines are written when the connection ends and others are written when an event
within a connection occurs. As such, timestamps can be confusing to look at
initially, but it is very normal for them not to be in order. Also, we are
already breaking any sort ordering by randomly selecting logs from bro.out and
replacing the timestamps with the current timestamp, so I'm not concerned with
my changes causing any more of a headache than flattening the decimal places
with 0s.
> sensor-stubs sometimes send malformed bro timestamps
> ----------------------------------------------------
>
> Key: METRON-815
> URL: https://issues.apache.org/jira/browse/METRON-815
> Project: Metron
> Issue Type: Bug
> Reporter: Jon Zeolla
> Assignee: Jon Zeolla
>
> The bro sensor-stub sends malformed timestamps when transforming an input
> timestamp that has less than 6 digits. For instance:
> [vagrant@node1 bin]$ SEARCH="\"ts\"\:[0-9]\+.[0-9]\{6\}"
> [vagrant@node1 bin]$ REPLACE="\"ts\"\:`date +%s`.000000"
> [vagrant@node1 bin]$ cat /opt/sensor-stubs/data/bro.out | sed -e
> "s/$SEARCH/$REPLACE/g"
> ...
> {"dns":
> {"ts":1491064638.000000.38621,"uid":"CQ5vBa2GcEToa4NKt5","id.orig_h":"192.168.66.1","id.orig_p":5353,"id.resp_h":"224.0.0.251","id.resp_p":5353,"proto":"udp","trans_id":0,"query":"_googlecast._tcp.local","qclass":1,"qclass_name":"C_INTERNET","qtype":12,"qtype_name":"PTR","AA":false,"TC":false,"RD":false,"RA":false,"Z":0,"rejected":false}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
