Justin Leet created METRON-1632:
-----------------------------------
Summary: Cannot filter by metaalert parents fields in Solr
Key: METRON-1632
URL: https://issues.apache.org/jira/browse/METRON-1632
Project: Metron
Issue Type: Sub-task
Reporter: Justin Leet
>From the alerts UI, attempt to filter by source.type:metaalert. No results
>will be returned. Similar results happen on other parent fields in metaalerts
>(but these are primarily metadata fields that wouldn't usually be filtered on,
>e.g. the 'groups' field that maintains the UI groupings that led to the
>creation of that metaalert in the first place).
After creating metaalert(s), this query, for example returns no results:
{code}
curl -X POST --header 'Content-Type: application/json' --header 'Accept:
application/json' -d '
{ "from": 0, "indices": [ "metaalert" ], "query": "source.type:metaalert",
"size": 5 }
' 'http://node1:8082/api/v1/search/search'
{code}
There should be results returned if metalaerts exist.
h5. Root Cause Analysis
This is because in Solr, we used nested objects to handle metaalerts.
For example, for the following query:
{code}
“ip.src.addr:192.168.1.1 AND source.type:metaalert”, we turn it into
{!parent which=source.type:metaalert v='ip_src_addr:192.168.1.1 AND
source.type:metaalert'}{code}
The “v=” syntax is just the syntax used to handle complicated clauses in the
<someChildren> element described in the link.
The catch here is that this filters on children alone, i.e. this query also
won’t get applied to the parent. For many queries this isn’t a big deal because
metaalerts don’t contain most fields, e.g. querying on ip.src.addr alone is a
child only field and functions as expected. However, anything at the parent
level, e.g. source.type:metaalert, won’t be matched by the query. This causes
queries acting attempting with a parent field to fail (In particular, being
AND’ed with a parent field).
The fields at the parent level I reasonably expect to be queried are
source.type – It’s the one that gets used a lot and shows up visibly. This is
the most important field to handle, particularly for UI reasons.
threat.triage.field – It’s substantially less likely, but someone could want to
search it.
The correct way to handle this, per the link above, is to do something like
{code:java}
+source.type:metaalert +
{!parent which=source.type:metaalert v='ip_src_addr:192.168.1.1’ }{code}
What this does is pull the parent specific filter out in front of the query and
treats it as an AND while the child specific portion is kept contained in the
“v=” clause.
Unfortunately, this is a bit more involved than what we currently have because
it involves a deeper understanding of the query than just plugging in what the
UI grabs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
