# Content Repository nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository nifi.content.claim.max.appendable.size=10 MB nifi.content.claim.max.flow.files=100 nifi.content.repository.directory.default=/srv/nifi-content/data/content-repository nifi.content.repository.archive.max.retention.period=12 hours nifi.content.repository.archive.max.usage.percentage=50% nifi.content.repository.archive.enabled=true nifi.content.repository.always.sync=false nifi.content.viewer.url=../nifi-content-viewer/
So we even use 10MB… will check if lowering the value changes anything On 22-09-13 20:04, Patrick Timmins wrote:
No, I agree. Lars, please give up the rest of your evening and drive back to work and report back with your findings ASAP. It may be past normal working hours in Germany, but you have NiFi brothers and sisters around the world that are counting on you ... please don't let us down.:) <- international smiley/joking symbol On 9/13/2022 10:15 AM, Joe Witt wrote:read that again and hopefully it was obvious I was joking. But I am looking forward to hearing what you learn. Thanks On Tue, Sep 13, 2022 at 10:10 AM Joe Witt <[email protected]> wrote:LarsI need you to drive back to work because now I am very vested in the outcome :)But yeah this was an annoying problem we saw hit some folks. Changing that value after fixing the behavior was the answer. I owe the community a blog on this.... Thanks On Tue, Sep 13, 2022 at 9:57 AM Lars Winderling <[email protected]> wrote:Sorry, misread the jira. We're still on the old default value. Thank you for being persistant about it. I will try it tomorrow with the lower value and get back to you. Not at work atm, so I can't paste the config values in detail.On 13 September 2022 16:45:30 CEST, Joe Witt <[email protected]> wrote:Lars You should not have to update to 1.17. While I'm always fond of peoople being on the latest the issue i mentioned is fixed in 1.16.3.HOWEVER, please do confirm your values. The one I'd really focus you on isnifi.content.claim.max.appendable.size=50 KB Our default before was like 1MB and what we'd see is we'd hang on to large content way longer than we intended because some queue had one tiny object in it. So that value became really important. If you're on 1MB change to 50KB and see what happens. Thanks On Tue, Sep 13, 2022 at 9:40 AM Lars Winderling <[email protected]> wrote:I guess the issue you linked, is related. I have seen similar messages in the log occasionally, but didn't directly connect it. Our config is pretty similar to the defaults, none of it should directly cause the issue. Will give 1.17.0 a try and come back if the issue persists. Your help is really appreciated, thanks!On 13 September 2022 16:33:53 CEST, Joe Witt <[email protected]> wrote:Lars The issue that came to mind ishttps://issues.apache.org/jira/browse/NIFI-10023 but that is fixed in1.16.2 and 1.17.0 so that is why I asked. What is in your nifi.properties for # Content Repository nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository nifi.content.claim.max.appendable.size=50 KB nifi.content.repository.directory.default=./content_repository nifi.content.repository.archive.max.retention.period=7 days nifi.content.repository.archive.max.usage.percentage=50% nifi.content.repository.archive.enabled=true nifi.content.repository.always.sync=false Thanks On Tue, Sep 13, 2022 at 7:04 AM Lars Winderling <[email protected]> wrote:I'm using 1.16.3 from upstream (no custom build) on java 11 temurin, debian 10, virtualized, no docker setup.On 13 September 2022 13:37:15 CEST, Joe Witt <[email protected]> wrote:Lars What version are you using? ThanksOn Tue, Sep 13, 2022 at 3:11 AM Lars Winderling <[email protected]> wrote:Dear community,sometimes our content repository grows out of bounds. Since it has been separated on disk from the rest of NiFi, we can still use the NiFi UI and empty the respective queues. However, the disk remains jammed. Sometimes, it gets cleaned up after a few mintes, but most of the time we need to restart NiFi manually, for the cleanup to happen. So. is there any way of triggering the content eviction manually without restarting NiFi? Btw. the respective files on disk are not archived in the content repository (thus not below */archive/*).Thanks in advance for your support! Best, Lars
OpenPGP_signature
Description: OpenPGP digital signature
