[
https://issues.apache.org/jira/browse/NIFI-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16952174#comment-16952174
]
Mark Payne commented on NIFI-6777:
----------------------------------
Are you using G1GC? In `boostrap.conf`, you will see the following line:
`java.arg.13=-XX:+UseG1GC`
It may or may not be commented out. If it is not commented out, this is likely
the concern. There are some problems with G1GC that were fixed in Java 9 but
not backported to Java 8, is my understanding. That can cause Lucene to have
some problems. Recommend you comment out that line so that you're not using
G1GC and instead use the JDK default, and restart. If you still are running
into the issue, then please let us know.
> Data Provenance SIGSEGV : Search Lucene Index-1
> -----------------------------------------------
>
> Key: NIFI-6777
> URL: https://issues.apache.org/jira/browse/NIFI-6777
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.8.0
> Environment: Linux CentOS Linux release 7.5.1804 (Core)
> Linux 3.10.0-862.11.6.el7.x86_64
> jre1.8.0_181-amd64 et jre1.8.0_221-amd64
> Reporter: Firenz
> Priority: Major
>
> NIFI crashes with a core dump when using data provenance (5 times in 1
> months, 4 differents VMs).
>
> It happens radomly when one of my data analyst is using dataprovenance, but
> the thread is always "Search Lucene Index-1". My monitoring does not show any
> memory or thread issues (overload...)
>
> Any advice ? (I cannot attach the hs_err_pid54305.log, token issue with Jira
> ?)
>
> Beginning of core dump :
> {code:java}
> # JRE version: Java(TM) SE Runtime Environment (8.0_181-b13) (build
> 1.8.0_181-b13)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.181-b13 mixed mode
> linux-amd64 compressed oops)
> # Problematic frame:
> # J 33233 C2
> org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock()V (448
> bytes) @ 0x00007f9c52baabf9 [0x00007f9
> c52baaa40+0x1b9]
> #
> # Core dump written. Default location: /appli/nifi-1.8.0/core or core.54305
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> #--------------- T H R E A D ---------------Current thread
> (0x00007f9c3c02b000): JavaThread "Search Lucene Index-1" daemon
> [_thread_in_Java, id=64766, stack(0x00007f9b9a
> c4d000,0x00007f9b9ad4e000)]siginfo: si_signo: 11 (SIGSEGV), si_code: 1
> (SEGV_MAPERR), si_addr: 0x00007f9b82019d0c
> {code}
> My Dataprovenance config :
> {code:java}
> # Persistent Provenance Repository Properties
> nifi.provenance.repository.directory.default=/data_nifi_provenance_repo/1.8.0
> nifi.provenance.repository.max.storage.time=480 hours
> nifi.provenance.repository.max.storage.size=9 GB
> nifi.provenance.repository.rollover.time=30 secs
> nifi.provenance.repository.rollover.size=100 MB
> nifi.provenance.repository.query.threads=2
> nifi.provenance.repository.index.threads=2
> nifi.provenance.repository.compress.on.rollover=true
> nifi.provenance.repository.always.sync=false
> nifi.provenance.repository.journal.count=16
> # Comma-separated list of fields. Fields that are not indexed will not be
> searchable. Valid fields are:
> # EventType, FlowFileUUID, Filename, TransitURI, ProcessorID,
> AlternateIdentifierURI, Relationship, Details
> nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename,
> ProcessorID, Relationship
> # FlowFile Attributes that should be indexed and made searchable. Some
> examples to consider are filename, uuid, mime.type
> nifi.provenance.repository.indexed.attributes=external_id, batch_id,
> property_id, property_code, http.headers.x-request-id
> # Large values for the shard size will result in more Java heap usage when
> searching the Provenance Repository
> # but should provide better performance
> nifi.provenance.repository.index.shard.size=500 MB
> # Indicates the maximum length that a FlowFile attribute can be when
> retrieving a Provenance Event from
> # the repository. If the length of any attribute exceeds this value, it will
> be truncated when the event is retrieved.
> nifi.provenance.repository.max.attribute.length=65536
> nifi.provenance.repository.concurrent.merge.threads=2
> nifi.provenance.repository.warm.cache.frequency=1 hour
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
