No, I agree. Lars, please give up the rest of your evening and drive
back to work and report back with your findings ASAP. It may be past
normal working hours in Germany, but you have NiFi brothers and sisters
around the world that are counting on you ... please don't let us down.
:) <- international smiley/joking symbol
On 9/13/2022 10:15 AM, Joe Witt wrote:
read that again and hopefully it was obvious I was joking. But I am
looking forward to hearing what you learn.
Thanks
On Tue, Sep 13, 2022 at 10:10 AM Joe Witt <[email protected]> wrote:
Lars
I need you to drive back to work because now I am very vested in the outcome :)
But yeah this was an annoying problem we saw hit some folks. Changing
that value after fixing the behavior was the answer. I owe the
community a blog on this....
Thanks
On Tue, Sep 13, 2022 at 9:57 AM Lars Winderling
<[email protected]> wrote:
Sorry, misread the jira. We're still on the old default value. Thank you for
being persistant about it. I will try it tomorrow with the lower value and get
back to you. Not at work atm, so I can't paste the config values in detail.
On 13 September 2022 16:45:30 CEST, Joe Witt <[email protected]> wrote:
Lars
You should not have to update to 1.17. While I'm always fond of
peoople being on the latest the issue i mentioned is fixed in 1.16.3.
HOWEVER, please do confirm your values. The one I'd really focus you on is
nifi.content.claim.max.appendable.size=50 KB
Our default before was like 1MB and what we'd see is we'd hang on to
large content way longer than we intended because some queue had one
tiny object in it. So that value became really important.
If you're on 1MB change to 50KB and see what happens.
Thanks
On Tue, Sep 13, 2022 at 9:40 AM Lars Winderling
<[email protected]> wrote:
I guess the issue you linked, is related. I have seen similar messages in the
log occasionally, but didn't directly connect it. Our config is pretty similar
to the defaults, none of it should directly cause the issue. Will give 1.17.0 a
try and come back if the issue persists. Your help is really appreciated,
thanks!
On 13 September 2022 16:33:53 CEST, Joe Witt <[email protected]> wrote:
Lars
The issue that came to mind is
https://issues.apache.org/jira/browse/NIFI-10023 but that is fixed in
1.16.2 and 1.17.0 so that is why I asked.
What is in your nifi.properties for
# Content Repository
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=50 KB
nifi.content.repository.directory.default=./content_repository
nifi.content.repository.archive.max.retention.period=7 days
nifi.content.repository.archive.max.usage.percentage=50%
nifi.content.repository.archive.enabled=true
nifi.content.repository.always.sync=false
Thanks
On Tue, Sep 13, 2022 at 7:04 AM Lars Winderling
<[email protected]> wrote:
I'm using 1.16.3 from upstream (no custom build) on java 11 temurin, debian
10, virtualized, no docker setup.
On 13 September 2022 13:37:15 CEST, Joe Witt <[email protected]> wrote:
Lars
What version are you using?
Thanks
On Tue, Sep 13, 2022 at 3:11 AM Lars Winderling <[email protected]>
wrote:
Dear community,
sometimes our content repository grows out of bounds. Since it has been
separated on disk from the rest of NiFi, we can still use the NiFi UI and empty
the respective queues. However, the disk remains jammed. Sometimes, it gets
cleaned up after a few mintes, but most of the time we need to restart NiFi
manually, for the cleanup to happen.
So. is there any way of triggering the content eviction manually without
restarting NiFi?
Btw. the respective files on disk are not archived in the content repository
(thus not below */archive/*).
Thanks in advance for your support!
Best,
Lars
