[
https://issues.apache.org/jira/browse/METRON-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Zeolla updated METRON-545:
------------------------------
Summary: Truncate fields larger than 32766 bytes (was: Truncate fields
larger than 32766)
> Truncate fields larger than 32766 bytes
> ---------------------------------------
>
> Key: METRON-545
> URL: https://issues.apache.org/jira/browse/METRON-545
> Project: Metron
> Issue Type: Sub-task
> Reporter: Jon Zeolla
> Priority: Minor
>
> Due to a limitation with using lucene where an individual term cannot be
> larger than 32766 bytes (assuming UTF-8 encoding, this is 8,191 characters),
> and assuming that we cannot easily identify the field datatype per the intent
> of the user (string vs integer vs ...), we should truncate fields if they are
> larger than 32766. This should be somewhat rare, but even in cases where it
> occurs we can leverage the dual storage (HDFS and Lucene), integrity checking
> fields (METRON-544), and customizability of the UI (METRON-195) in order to
> retrieve the full original field value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
