[
https://issues.apache.org/jira/browse/NIFI-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729762#comment-17729762
]
Giovanni commented on NIFI-11530:
---------------------------------
Hi Joe,
thanks for your reply.
Here are my settings:
{code:java}
# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=./flowfile_repository
nifi.flowfile.repository.checkpoint.interval=20 secs
nifi.flowfile.repository.always.sync=false
nifi.flowfile.repository.retain.orphaned.flowfiles=true
nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000{code}
{code:java}
# Content Repository
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=50 KB
nifi.content.repository.directory.default=./content_repository
nifi.content.repository.archive.max.retention.period=7 days
nifi.content.repository.archive.max.usage.percentage=50%
nifi.content.repository.archive.enabled=true
nifi.content.repository.always.sync=false
nifi.content.viewer.url=../nifi-content-viewer/{code}
{code:java}
# Provenance Repository Properties
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
# Persistent Provenance Repository Properties
nifi.provenance.repository.directory.default=./provenance_repository
nifi.provenance.repository.max.storage.time=30 days
nifi.provenance.repository.max.storage.size=10 GB
nifi.provenance.repository.rollover.time=10 mins
nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.query.threads=2
nifi.provenance.repository.index.threads=2
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
# Comma-separated list of fields. Fields that are not indexed will not be
searchable. Valid fields are:
# EventType, FlowFileUUID, Filename, TransitURI, ProcessorID,
AlternateIdentifierURI, Relationship, Details
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename,
ProcessorID, Relationship
# FlowFile Attributes that should be indexed and made searchable. Some
examples to consider are filename, uuid, mime.type
nifi.provenance.repository.indexed.attributes=
# Large values for the shard size will result in more Java heap usage when
searching the Provenance Repository
# but should provide better performance
nifi.provenance.repository.index.shard.size=500 MB
# Indicates the maximum length that a FlowFile attribute can be when retrieving
a Provenance Event from
# the repository. If the length of any attribute exceeds this value, it will be
truncated when the event is retrieved.
nifi.provenance.repository.max.attribute.length=65536
nifi.provenance.repository.concurrent.merge.threads=2
# Volatile Provenance Respository Properties
nifi.provenance.repository.buffer.size=100000{code}
In response to your point, I confirm each node has only a single disk.
> Disk full even with nifi.content.repository.archive.max.usage.percentage set
> to 50%
> -----------------------------------------------------------------------------------
>
> Key: NIFI-11530
> URL: https://issues.apache.org/jira/browse/NIFI-11530
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.20.0
> Environment: Ubuntu 20.04.5 LTS
> Reporter: Giovanni
> Priority: Major
> Attachments: 20230605_disk_usage.jpg, content_archive.jpg,
> flowfile_archive.jpg, jvm.jpg, nifi1-app.log, nifi2-app.log, nifi3-app.log,
> nifi_bug.jpg, provenance_archive.jpg
>
>
> Nifi primary node reports disk full causing all nodes to stop working.
> Restarting nifi service does not resolve.
> Restarting the VM does not resolve.
> The only way to fix is to clean te content_repository dir:
> rm -rf ./nifi/content_repository/*
>
> Unfortunately I have no logs of the issue ongoing.
>
> UPDATE:
> I'm having the problem again.
> Every archive size is more than 50% on each node, with 70%+ peak on
> coordinator node (see attachments).
> I'm also attaching nifi-app.log this time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
