[
https://issues.apache.org/jira/browse/NIFI-2787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511031#comment-15511031
]
ASF GitHub Bot commented on NIFI-2787:
--------------------------------------
GitHub user mosermw opened a pull request:
https://github.com/apache/nifi/pull/1043
NIFI-2787 truncate flowfile attributes that get indexed to fit within
Lucene limits
NIFI-2787
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mosermw/nifi NIFI-2787
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/1043.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1043
----
commit 0765a243e8d1e3af83b112bf116618bcac521aa9
Author: Mike Moser <[email protected]>
Date: 2016-09-21T20:10:49Z
NIFI-2787 truncate flowfile attributes that get indexed to fit within
Lucene limits
----
> PersistentProvenanceRepository rollover can fail on immense indexed attributes
> ------------------------------------------------------------------------------
>
> Key: NIFI-2787
> URL: https://issues.apache.org/jira/browse/NIFI-2787
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.0.0, 0.7.0
> Reporter: Michael Moser
>
> Accidentally created an immense attribute (36,000 bytes), which I indexed
> with nifi.provenance.repository.indexed.attributes. Received this error.
> ERROR [Provenance Repository Rollover Thread-1]
> o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance
> repository due to java.lang.IllegalArgumentException: Document contains at
> least one immense term in field="FOO" (whose UTF8 encoding is longer than the
> max length 32766), all of which were skipped. Please correct the analyzer to
> not produce such terms.
> Perhaps this is as simple as changing
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/RepositoryConfiguration.java#L37
> to 32766 to match Lucene. Investigation & testing needed.
> For background, this Lucene ticket made exceeding the term size limit an
> IllegalArgumentException https://issues.apache.org/jira/browse/LUCENE-5472
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
