[
https://issues.apache.org/jira/browse/METRON-1380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16317169#comment-16317169
]
ASF GitHub Bot commented on METRON-1380:
----------------------------------------
Github user justinleet commented on the issue:
https://github.com/apache/metron/pull/882
I ran through the instructions. The new data flowing automatically into
the default ES mapping causes the problem that fielddata isn't true, so
grouping queries don't match on the squid index and it doesn't show up in the
UI.
```
{
"responseCode": 500,
"message": "Failed to execute search; error='IllegalArgumentException:
Fielddata is disabled on text fields by default. Set fielddata=true on
[source:type] in order to load fielddata in memory by uninverting the inverted
index. Note that this can however use significant memory. Alternatively use a
keyword field instead.',
search='{\"query\":{\"bool\":{\"must\":[{\"query_string\":{\"query\":\"*\",\"fields\":[],\"use_dis_max\":true,\"tie_breaker\":0.0,\"default_operator\":\"or\",\"auto_generate_phrase_queries\":false,\"max_determinized_states\":10000,\"enable_position_increments\":true,\"fuzziness\":\"AUTO\",\"fuzzy_prefix_length\":0,\"fuzzy_max_expansions\":50,\"phrase_slop\":0,\"escape\":false,\"split_on_whitespace\":true,\"boost\":1.0}}],\"must_not\":[{\"exists\":{\"field\":\"metaalerts\",\"boost\":1.0}}],\"disable_coord\":false,\"adjust_pure_negative\":true,\"boost\":1.0}},\"aggregations\":{\"source:type_group\":{\"terms\":{\"field\":\"source:type\",\"size\":1000,\"min_doc_count\":1,\"shard_min_doc_count\":0,\"show_term_doc_count_error\":false,\"order\":{\"_term\":\"desc\"}},\"aggregations\":{\"threat:triage:score_score\":{\"sum\":{\"field\":\"threat:triage:score\",\"missing\":0}}}}}}'",
"fullMessage": "IllegalArgumentException: Fielddata is disabled on text
fields by default. Set fielddata=true on [source:type] in order to load
fielddata in memory by uninverting the inverted index. Note that this can
however use significant memory. Alternatively use a keyword field instead."
}
```
The data does show up as expected in ES via direct query,
```
curl -XGET 'localhost:9200/squid*/_search?pretty' -H 'Content-Type:
application/json' -d'
{
"query": {
"match_all": {}
}
}
'
```
while on full dev.
This seems primarily like a matter of updating the instructions
appropriately.
> Create a typosquatting use-case
> -------------------------------
>
> Key: METRON-1380
> URL: https://issues.apache.org/jira/browse/METRON-1380
> Project: Metron
> Issue Type: New Feature
> Reporter: Casey Stella
>
> We should have a use-case to support detecting typosquatted domains in stream.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
