[
https://issues.apache.org/jira/browse/METRON-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700530#comment-16700530
]
ASF GitHub Bot commented on METRON-1869:
----------------------------------------
GitHub user nickwallen reopened a pull request:
https://github.com/apache/metron/pull/1280
METRON-1869 Unable to Sort an Escalated Meta Alert
This fixes a bug that causes meta-alerts to not be visible in the Alerts UI
when the UI is sorted by 'alert_status' and a meta-alert has been escalated.
This is only a problem when indexing into Elasticsearch.
The root cause is that the 'alert_status' field needs to be defined as a
keyword in the metaalert index template. This field only exists in the
meta-alert index after a meta-alert has changed status, like when an alert is
escalated.
## Changes
* Fixed the Elasticsearch meta-alerts template that is deployed with the
MPack.
* Added an integration test for this bug.
## Testing
Follow the "steps to reproduce" outlined in the JIRA to ensure that the bug
has been squashed.
1. Create a meta-alert.
2. Escalate the meta-alert.
3. Submit another search that filters the results to only show meta-alerts.
4. Sort the results by “alert_status” field.
5. The meta-alert should still be visible.
## Pull Request Checklist
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1869
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1280.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1280
----
commit d2f56b4f41bead7b3490cb81bdce553359ce09fc
Author: Nick Allen <nick@...>
Date: 2018-11-14T13:34:17Z
Attempting to create test that highlights the problem
commit 70d1dc4bea3992f8e94eb855c36c99fde7f6cad9
Author: Nick Allen <nick@...>
Date: 2018-11-26T19:41:21Z
METRON-1869 Fix missing alert_status field in meta-alert template
commit 09558fb5fa5eb24cebfc974170fd14abf047e2e5
Author: Nick Allen <nick@...>
Date: 2018-11-26T20:51:43Z
Completed integration test that highlights the problem
commit afb87ba0b4efbc74913698e123c5661a42ac7f45
Author: Nick Allen <nick@...>
Date: 2018-11-26T20:58:47Z
Testing sorted search results under 3 scenarios
commit 60c3af90061ce82a89696c96b9f07b3e4fb35324
Author: Nick Allen <nick@...>
Date: 2018-11-26T20:59:49Z
Reset ordering
----
> Unable to Sort an Escalated Meta Alert
> --------------------------------------
>
> Key: METRON-1869
> URL: https://issues.apache.org/jira/browse/METRON-1869
> Project: Metron
> Issue Type: Bug
> Reporter: Nick Allen
> Assignee: Nick Allen
> Priority: Major
> Attachments: Can Sort on Metaalert Before Escalation.png, Cannot View
> Metaalert Sorted by Status After Escalation.png
>
>
> Steps to Replicate:
> 1. Create a meta-alert.
> 2. Escalate the meta-alert.
> 3. Submit another search that filters the results to only show meta-alerts.
> 4. Sort the results by “alert_status” field.
> 5. The search will not return any meta-alerts in the results.
> 6. The REST log will contain the exception shown above.
> {code:java}
> 18/11/12 22:25:50 ERROR dao.ElasticsearchRequestSubmitter: Shard search
> failure [1/1]; reason=ElasticsearchException: Elasticsearch exception
> [type=illegal_argument_exception, reason=Fielddata is disabled on text fields
> by default. Set fielddata=true on [alert_status] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.],
> index=metaalert_index, shard=0, status=INTERNAL_SERVER_ERROR,
> nodeId=uuG7-lZoSiaowJK_cUD4IQ
> ElasticsearchException[Elasticsearch exception
> [type=illegal_argument_exception, reason=Fielddata is disabled on text fields
> by default. Set fielddata=true on [alert_status] in order to load fielddata
> in memory by uninverting the inverted index. Note that this can however use
> significant memory. Alternatively use a keyword field instead.]]
> at
> org.elasticsearch.ElasticsearchException.innerFromXContent(ElasticsearchException.java:490)
> at
> org.elasticsearch.ElasticsearchException.fromXContent(ElasticsearchException.java:406)
> at
> org.elasticsearch.action.search.ShardSearchFailure.fromXContent(ShardSearchFailure.java:205)
> at
> org.elasticsearch.action.search.SearchResponse.fromXContent(SearchResponse.java:297)
> at
> org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:526)
> at
> org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$2(RestHighLevelClient.java:382)
> at
> org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:413)
> at
> org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:382)
> at
> org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:323)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchRequestSubmitter.submitSearch(ElasticsearchRequestSubmitter.java:62)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchSearchDao.search(ElasticsearchSearchDao.java:128)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchDao.search(ElasticsearchDao.java:191)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertSearchDao.search(ElasticsearchMetaAlertSearchDao.java:81)
> at
> org.apache.metron.elasticsearch.dao.ElasticsearchMetaAlertDao.search(ElasticsearchMetaAlertDao.java:209)
> at
> org.apache.metron.rest.service.impl.SearchServiceImpl.search(SearchServiceImpl.java:92)
> at
> org.apache.metron.rest.controller.SearchController.search(SearchController.java:54){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
