You're right, with ES 5 we can use periods directly instead of transforming
them in indexing to colons (actually, this feature was reintroduced sin 2.4
<https://github.com/elastic/elasticsearch/pull/19937/files>). I outlined
this as a benefit in the original JIRA
<https://issues.apache.org/jira/browse/METRON-939?filter=-1>, along with a
ton of other benefits including native IPv6 support </shameless plug>
Jon
On Wed, Oct 4, 2017 at 5:03 PM Casey Stella <ceste...@gmail.com> wrote:
> Ok, so this is subtle. Your rules are wrong and I totally understand why
> you thought they were right.
>
> When we index into ES, we take . and convert them to :, however PRIOR to
> indexing (when threat triage is running) those fields have .'s not :'s
> Therefore, your rules should be:
>
> userIdentity.sessionContext.attributes.mfaAuthenticated == 'False'
> and
> additionalEventData.MFAUsed == 'No'
>
> The same general argument goes for your threat triage stellar expressions.
>
>
> Sorry about the confusion, we do that mapping because ES doesn't handle
> those .'s well. Hey, maybe ES 5 is more sane about that sort of thing and
> we can avoid doing that transformation.
>
> Casey
>
> On Wed, Oct 4, 2017 at 4:38 PM, Laurens Vets <laur...@daemon.be> wrote:
>
> > No idea whether it's a bug yet, I just need a 2nd set of eyes :)
> >
> > This is my event as indexed in ES (Obviously some parts have been
> > obfuscated):
> >
> > {
> > "_index": "cloudtrail_index_2017.10.04.19",
> > "_type": "cloudtrail_doc",
> > "_id": "95617686-bd39-46ff-b5c0-db3aeb5b6bab",
> > "_score": null,
> > "_timestamp": 1507143907108,
> > "_source": {
> > "eventID": "9e3d5468-2d97-4b9a-9821-5c61fec8c158",
> > "additionalEventData:MFAUsed": "No",
> > "adapter:stellaradapter:end:ts": "1507143907145",
> > "threatinteljoinbolt:joiner:ts": "1507143907153",
> > "eventVersion": "1.05",
> > "threat:triage:rules:0:comment": "Checks whether the field is_work is
> > true or false.",
> > "sourceIPAddress": "208.110.73.106",
> > "eventSource": "signin.amazonaws.com",
> > "enrichmentsplitterbolt:splitter:begin:ts": "1507143907143",
> > "enrichmentjoinbolt:joiner:ts": "1507143907147",
> > "additionalEventData:MobileVersion": "No",
> > "threat:triage:rules:0:name": "Not WORK",
> > "source:type": "cloudtrail",
> > "original_string": "{\"eventVersion\":\"1.05\",\"
> > userIdentity\":{\"type\":\"IAMUser\",\"principalId\":\"AIDAI
> > 5ITCMVR3BQV5DUFW\",\"arn\":\"arn:aws:iam::<ACCOUNTID>:user/
> > <EMAIL>\",\"accountId\":\"<ACCOUNTID>\",\"userName\":\"<
> > EMAIL>\"},\"eventTime\":\"2017-10-04T18:57:31Z\",\"eventSource\":\"
> > signin.amazonaws.com\",\"eventName\":\"ConsoleLogin\"
> > ,\"awsRegion\":\"us-east-1\",\"sourceIPAddress\":\"208.110.7
> > 3.106\",\"userAgent\":\"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0)
> > Gecko/20100101 Firefox/56.0\",\"requestParame
> > ters\":null,\"responseElements\":{\"ConsoleLogin\":\"
> > Success\"},\"additionalEventData\":{\"LoginTo\":\"https://
> > console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true\
> <http://console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true%5C>
> > <
> https://console.aws.amazon.com/console/home?state=hashArgs%23&isauthcode=true%5C
> >
> > ",\"MobileVersion\":\"No\",\"MFAUsed\":\"No\"},\"eventID\":
> > \"9e3d5468-2d97-4b9a-9821-5c61fec8c158\",\"eventType\":\
> > "AwsConsoleSignIn\",\"recipientAccountId\":\"<ACCOUNTID>\"}",
> > "eventTime": "2017-10-04T18:57:31Z",
> > "eventName": "ConsoleLogin",
> > "recipientAccountId": "<ACCOUNTID>",
> > "userIdentity:principalId": "AIDAI5ITCMVR3BQV5DUFW",
> > "threatintelsplitterbolt:splitter:end:ts": "1507143907148",
> > "threat:triage:rules:0:score": 20,
> > "timestamp": 1507143907108,
> > "threat:triage:rules:0:reason": "208.110.73.106 is not an WORK
> > network!",
> > "awsRegion": "us-east-1",
> > "is_work": false,
> > "userIdentity:userName": "<EMAIL>",
> > "enrichmentsplitterbolt:splitter:end:ts": "1507143907143",
> > "threat:triage:score": 20,
> > "is_alert": "true",
> > "userAgent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:56.0)
> > Gecko/20100101 Firefox/56.0",
> > "adapter:stellaradapter:begin:ts": "1507143907145",
> > "eventType": "AwsConsoleSignIn",
> > "userIdentity:arn": "arn:aws:iam::<ACCOUNTID>:user/<EMAIL>",
> > "userIdentity:accountId": "<ACCOUNTID>",
> > "userIdentity:type": "IAMUser",
> > "threatintelsplitterbolt:splitter:begin:ts": "1507143907148",
> > "guid": "95617686-bd39-46ff-b5c0-db3aeb5b6bab",
> > "additionalEventData:LoginTo": "https://console.aws.amazon.co
> > m/console/home?state=hashArgs%23&isauthcode=true",
> > "responseElements:ConsoleLogin": "Success"
> > },
> > "fields": {
> > "adapter:stellaradapter:end:ts": [
> > 1507143907145
> > ],
> > "threatinteljoinbolt:joiner:ts": [
> > 1507143907153
> > ],
> > "enrichmentsplitterbolt:splitter:end:ts": [
> > 1507143907143
> > ],
> > "enrichmentsplitterbolt:splitter:begin:ts": [
> > 1507143907143
> > ],
> > "enrichmentjoinbolt:joiner:ts": [
> > 1507143907147
> > ],
> > "adapter:stellaradapter:begin:ts": [
> > 1507143907145
> > ],
> > "eventTime": [
> > 1507143451000
> > ],
> > "threatintelsplitterbolt:splitter:begin:ts": [
> > 1507143907148
> > ],
> > "threatintelsplitterbolt:splitter:end:ts": [
> > 1507143907148
> > ],
> > "timestamp": [
> > 1507143907108
> > ]
> > },
> > "sort": [
> > 1507143451000
> > ]
> > }
> >
> > This is my sensor configuration:
> >
> >
> > {
> > "enrichment": {
> > "fieldMap": {
> > "stellar": {
> > "config": {
> > "is_work": "IN_SUBNET(if
> > IS_IP(sourceIPAddress) then sourceIPAddress else NULL, '1.2.3.4/16', '
> > 5.6.7.8/23')"
> > }
> > }
> > },
> > "fieldToTypeMap": {},
> > "config": {}
> > },
> > "threatIntel": {
> > "fieldMap": {
> > "stellar": {
> > "config": [
> > "is_alert := exists(is_work) &&
> > is_work != true && eventName == \"ConsoleLogin\"",
> > "is_alert := is_alert ||
> > (eventName == \"ConsoleLogin\" &&
> userIdentity:sessionContext:attributes:mfaAuthenticated
> > == \"False\")",
> > "is_alert := is_alert ||
> > (eventName == \"ConsoleLogin\" && additionalEventData:MFAUsed == \"No\")"
> > ]
> > }
> > },
> > "fieldToTypeMap": {},
> > "config": {},
> > "triageConfig": {
> > "riskLevelRules": [
> > {
> > "name": "Not WORK",
> > "comment": "Checks whether the
> > field is_work is true or false.",
> > "rule": "is_work == false",
> > "score": 20,
> > "reason": "FORMAT('%s is not an
> > WORK network!', sourceIPAddress)"
> > },
> > {
> > "name": "MFA",
> > "comment": "Checks whether MFA
> > used or not.",
> > "rule":
> > "userIdentity:sessionContext:attributes:mfaAuthenticated == 'False'",
> > "score": 20,
> > "reason": null
> > },
> > {
> > "name": "MFA2",
> > "comment": "Checks whether MFA
> > used or not.",
> > "rule":
> > "additionalEventData:MFAUsed == 'No'",
> > "score": 20,
> > "reason": null
> > }
> > ],
> > "aggregator": "SUM",
> > "aggregationConfig": {}
> > }
> > },
> > "configuration": {}
> > }
> >
> > Any idea why the score isn't 40? I would expect riskLevelRule 1 & 2 to be
> > SUMmed?
> >
>
--
Jon
