Ryan Merriman created METRON-1535:
-------------------------------------
Summary: IndexDao patch function is not atomic
Key: METRON-1535
URL: https://issues.apache.org/jira/browse/METRON-1535
Project: Metron
Issue Type: Bug
Reporter: Ryan Merriman
Our Index and Elasticsearch DAO patch operations are not atomic. Currently a
[patch|https://github.com/apache/metron/blob/master/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/IndexDao.java#L125]
involves looking up the current version of a document, patching it, then
reindexing the whole document in separate steps. This could lead to a race
condition that causes incorrect or corrupt data.
This pattern also exists in a couple different places in the
ElasticsearchMetaAlertDao including
[here|https://github.com/apache/metron/blob/master/metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/IndexDao.java#L125].
Elasticsearch offers an [Update
API|https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-update.html#docs-update]
that may be more appropriate in these cases. I don't know if it works on
nested documents so that would need to be tested.
Looking up the whole doc as part of an update is also causing this
[bug|https://github.com/apache/metron/pull/995] in the SolrMetaAlertDao feature
because Solr returns extra fields that can't be directly reindexed. There is
also an [Atomic
Update|https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-AtomicUpdates]
in Solr that is similar to the Elasticsearch Update API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
