[
https://issues.apache.org/jira/browse/METRON-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634018#comment-15634018
]
ASF GitHub Bot commented on METRON-402:
---------------------------------------
GitHub user mmiklavc reopened a pull request:
https://github.com/apache/incubator-metron/pull/320
Metron-402: Snort timestamp field shows up wrong value
Need to manually test in full-dev, but wanted to get this up for review
asap.
Resolves https://issues.apache.org/jira/browse/METRON-402
UPDATE 10/25/16 - Tested in full-dev. Tested with incorrect/unparsable
timezone and format. Verified configuration logging statements also.
```
{
"parserClassName":"org.apache.metron.parsers.snort.BasicSnortParser",
"sensorTopic":"snort",
"parserConfig": {
"dateFormat" : "MM/dd/yy-HH:mm:ss.SSSSSS",
"timeZone" : "America/New_York"
}
}
```
**Changes:**
- Change Snort configuration to include date in the timestamp output
- Fix BasicSnortParser to handle microseconds properly - switched to Java
8's java.time API.
- Added the ability to specify timezone and dateformat configuration to the
snort parser. Defaults to system default zone for the ZoneId,
MM/dd/yy-HH:mm:ss.SSSSSS for the dateformat. Note the addition of "yy" to the
dateformat.
**Testing:**
Can pass in different dateformat and timezone configuration and note the
different behavior.
Options are "timeZone" and "dateFormat". Valid timezones are per
ZoneId.getAvailableZoneIds(). DateFormats should be valid per options here -
https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mmiklavc/incubator-metron METRON-402
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-metron/pull/320.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #320
----
commit b5dce190d098e1c4e1d67c5b7dd96639ec2ff937
Author: Michael Miklavcic <[email protected]>
Date: 2016-10-24T14:15:46Z
partial commit
commit 6798c6dd9eba45833f110f4f02ace72a9b8ffcfa
Author: Michael Miklavcic <[email protected]>
Date: 2016-10-24T21:00:34Z
METRON-402: Fix Snort parser to handle microseconds properly.
commit c42050ed5aa14d3e139c8624b4f281126fc0a5c3
Author: Michael Miklavcic <[email protected]>
Date: 2016-10-24T21:08:50Z
METRON-402: Fix Snort parser to handle microseconds properly.
commit 81b8e21c875e9c5acaeccd2963fe73029d29be54
Author: Michael Miklavcic <[email protected]>
Date: 2016-10-25T16:49:06Z
METRON-402: Add logging to configure method
commit 007849d7a91fd51aa67e95365040c5253b21c758
Author: Michael Miklavcic <[email protected]>
Date: 2016-11-03T13:39:53Z
METRON-402: Reset the junit version
Separate PR upgraded global junit from 4.4 to 4.12, so this change refers
back again to that new global version.
----
> Snort timestamp field shows up wrong value
> ------------------------------------------
>
> Key: METRON-402
> URL: https://issues.apache.org/jira/browse/METRON-402
> Project: Metron
> Issue Type: Bug
> Affects Versions: 0.2.2BETA
> Reporter: Neha Sinha
> Assignee: Michael Miklavcic
> Fix For: 0.2.2BETA
>
>
> Hi,
> i injected the following snort log:-
> 07/28-06:37:58.922676 ,1,999158,0,"'snort test
> alert'",TCP,192.168.138.158,49188,62.75.195.236,80,00:00:00:00:00:00,00:00:00:00:00:00,0x3C,***A****,0xF017C4DA,0xABDB8426,,0xF6C9,128,0,2319,40,40960,,,,
> I expected timestamp field in indexed json to be epoch equivalent of what is
> given in the log.However the indexed snort json represents the current date
> and time.
> ========================================================
> {
> * "_index": "snort_index_2016.09.01.09",
> * "_type": "snort_doc",
> * "_id": "AVblCLtfZ5WQUn7o8i6U",
> * "_version": 1,
> * "_score": 1,
> * "_timestamp": 1469688800676,
> * "_source": {
> * "msg": ""'snort test alert'"",
> * "enrichments:geo:ip_dst_addr:locID": "794448",
> * "enrichments:geo:ip_dst_addr:location_point":
> "48.5839,7.7455",
> * "sig_rev": "0",
> * "ip_dst_port": "80",
> * "threatinteljoinbolt:joiner:ts": "1472721369718",
> * "ethsrc": "00:00:00:00:00:00",
> * "tcpseq": "0xF017C4DA",
> * "dgmlen": "40",
> * "enrichmentsplitterbolt:splitter:begin:ts":
> "1472721369701",
> * "enrichmentjoinbolt:joiner:ts": "1472721369707",
> * "adapter:geoadapter:begin:ts": "1472721369702",
> * "tcpwindow": "0xF6C9",
> * "enrichments:geo:ip_dst_addr:latitude": "48.5839",
> * "tcpack": "0xABDB8426",
> * "protocol": "TCP",
> * "source:type": "snort",
> * "adapter:threatinteladapter:end:ts": "1472721369718",
> * "ip_dst_addr": "62.75.195.236",
> * "original_string": "07/28-06:37:58.922676
> ,1,999158,0,"'snort test
> alert'",TCP,192.168.138.158,49188,62.75.195.236,80,00:00:00:00:00:00,00:00:00:00:00:00,0x3C,***A****,0xF017C4DA,0xABDB8426,,0xF6C9,128,0,2319,40,40960,,,,",
> * "adapter:hostfromjsonlistadapter:end:ts":
> "1472721369702",
> * "tos": "0",
> * "adapter:geoadapter:end:ts": "1472721369707",
> * "id": "2319",
> * "ip_src_addr": "192.168.138.158",
> * "threatintelsplitterbolt:splitter:end:ts":
> "1472721369707",
> * "enrichments:geo:ip_dst_addr:longitude": "7.7455",
> * "timestamp": 1469688800676,
> * "ethdst": "00:00:00:00:00:00",
> * "enrichmentsplitterbolt:splitter:end:ts":
> "1472721369701",
> * "enrichments:geo:ip_dst_addr:city": "Strassbourg",
> * "enrichments:geo:ip_dst_addr:postalCode": "67100",
> * "is_alert": "true",
> * "adapter:hostfromjsonlistadapter:begin:ts":
> "1472721369702",
> * "ttl": "128",
> * "ethlen": "0x3C",
> * "iplen": "40960",
> * "ip_src_port": "49188",
> * "threat:triage:level": 10,
> * "threatintelsplitterbolt:splitter:begin:ts":
> "1472721369707",
> * "adapter:threatinteladapter:begin:ts": "1472721369708",
> * "tcpflags": "***A****",
> * "enrichments:geo:ip_dst_addr:country": "FR",
> * "sig_id": "999158",
> * "sig_generator": "1"
> * }
> }
> ========================================================
> Inorder to investigate this case I went through the following
> https://github.com/hortonworks/metron/blob/apache-ref/master/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/snort/BasicSnortParser.java
> and found the following "TODO" note in the "toEpoch" function :-
> ========================================================
> private long toEpoch(String snortDatetime) throws ParseException {
>
> /*
> * TODO how does Snort not embed the year in their default
> timestamp?! need to change this in
> * Snort configuration. for now, just assume current year.
> */
> int year = Calendar.getInstance().get(Calendar.YEAR);
> String withYear = Integer.toString(year) + " " + snortDatetime;
> // convert to epoch time
> SimpleDateFormat df = new SimpleDateFormat("yyyy MM/dd-HH:mm:ss.S");
> Date date = df.parse(withYear);
> return date.getTime();
> }
> ========================================================
> As per the above "TODO" note the year would match to the current year but
> rest of the time fields should match to what is in the original snort log.
> However this is not the case.
> Also Do we have any jira to track the "todo" part?We should be having one as
> an enhancement atleast.
> Regards,
> neha
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
