Michel Sumbul created METRON-1640:
-------------------------------------
Summary: Allow the user to store or not the original_string in ES
Key: METRON-1640
URL: https://issues.apache.org/jira/browse/METRON-1640
Project: Metron
Issue Type: Improvement
Reporter: Michel Sumbul
Hi,
Currently, all the messages stored in ES have the original_string. Keeping the
original string and all the parse data might conduct to store up to 2 times
more data. This implied that the needed infrastructure to handle this will have
to be much more important, so a bigger cost.
The user might be interested to keep it for legal reason or to be able to do
text search on it but he might also decide to remove it because it has no added
value and cost space.
Moreover the original_string will remain in HDFS in case they need it, so no
lost of data.
The goal of this request is:
* to have a check box (or another UI interface) in the configuration of the
sensor specifying if the original string has to be ingested in ES.
* Modifying the indexing storm topology to remove that field before saving the
data to ES
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
