[
https://issues.apache.org/jira/browse/METRON-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Zeolla updated METRON-544:
------------------------------
Description:
If a value is > 32766 long, add an integrity checking field using a hash such
as SHA3-256. This provides a method to both validate and compare the data
against another store without a 32k limitation. For instance, if the field is
"uri", add a key of "uri_hash" and a value of the hash of the original field's
value (pre-truncation).
Where this should be added is up for debate - I would initially suggest putting
it upstream of the indexingBolt in the indexing topology (as a splitter), but I
could see some arguments for putting it in the indexingBolt, or somewhere in
the enrichments topology.
was:
If a value is > 32766 long, add an integrity checking field using a hash such
as SHA3-256. This provides a method to both validate and compare the data
against another store without a 32k limitation. For instance, if the field is
"uri", add a key of "uri_hash" and a value of the hash of the original field's
value.
Where this should be added is up for debate - I would initially suggest putting
it upstream of the indexingBolt in the indexing topology (as a splitter), but I
could see some arguments for putting it in the indexingBolt, or somewhere in
the enrichments topology.
> Add an integrity checking field for large values
> ------------------------------------------------
>
> Key: METRON-544
> URL: https://issues.apache.org/jira/browse/METRON-544
> Project: Metron
> Issue Type: Sub-task
> Reporter: Jon Zeolla
> Priority: Minor
>
> If a value is > 32766 long, add an integrity checking field using a hash such
> as SHA3-256. This provides a method to both validate and compare the data
> against another store without a 32k limitation. For instance, if the field
> is "uri", add a key of "uri_hash" and a value of the hash of the original
> field's value (pre-truncation).
> Where this should be added is up for debate - I would initially suggest
> putting it upstream of the indexingBolt in the indexing topology (as a
> splitter), but I could see some arguments for putting it in the indexingBolt,
> or somewhere in the enrichments topology.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
