[
https://issues.apache.org/jira/browse/NIFI-14707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Secules updated NIFI-14707:
--------------------------------
Description:
Affected versions: All, but I didn't know if its worth it to go and tag every
release since the initial commit where NiFi went open source.
Impacts:
* Increased heap use
* Increased time taken to upload process groups with many connected components
I was investigating why it takes longer and longer to upload process groups
containing many connected components to nifi and after taking a flamegraph I
noticed only this call to getLogger inside `DebugEnabledTimedLock` was showing
up. All the other loggers I see in my nifi heap dump look like they only go as
specific as the Class name and not the specific instance. I believe this is
leading to excess heap use and increased time uploading process groups
containing many connected components.
I've attached a flamegraph that I sampled when I was uploading many process
groups with many connected components ad well as a screenshot of a heap dump
showing all the Loggers I have allocated over 400,000 and from the spot checks
I did, it was very hard to find one that wasn't a TimedLock logger.
I'm looking for feedback on the correct course of action. The simplest would be
removing the specificity and just using the logger for `TimedLock`. Or if we
still want this specific debugging feature we could use a system property to
gate whether we create `TimedLock` loggers with the instance ID or not.
was:
Affected versions: All, but I didn't know if its worth it to go and tag every
release since the initial commit where NiFi went open source.
Impacts:
* Increased heap use
* Increased time taken to upload process groups with many connected components
I was investigating why it takes longer and longer to upload process groups
containing many connected components to nifi and after taking a flamegraph I
noticed only this call to getLogger inside `DebugEnabledTimedLock` was showing
up. All the other loggers I see in my nifi heap dump look like they only go as
specific as the Class name and not the specific instance. I believe this is
leading to excess heap use and increased time uploading process groups
containing many connected components.
I'm looking for feedback on the correct course of action. The simplest would be
removing the specificity and just using the logger for `TimedLock`. Or if we
still want this specific debugging feature we could use a system property to
gate whether we create `TimedLock` loggers with the instance ID or not.
> TImedLock creates too many loggers
> ----------------------------------
>
> Key: NIFI-14707
> URL: https://issues.apache.org/jira/browse/NIFI-14707
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 2.4.0
> Reporter: Eric Secules
> Assignee: Eric Secules
> Priority: Major
> Attachments: Screenshot 2025-07-02 at 2.14.32 PM.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Affected versions: All, but I didn't know if its worth it to go and tag every
> release since the initial commit where NiFi went open source.
> Impacts:
> * Increased heap use
> * Increased time taken to upload process groups with many connected
> components
> I was investigating why it takes longer and longer to upload process groups
> containing many connected components to nifi and after taking a flamegraph I
> noticed only this call to getLogger inside `DebugEnabledTimedLock` was
> showing up. All the other loggers I see in my nifi heap dump look like they
> only go as specific as the Class name and not the specific instance. I
> believe this is leading to excess heap use and increased time uploading
> process groups containing many connected components.
> I've attached a flamegraph that I sampled when I was uploading many process
> groups with many connected components ad well as a screenshot of a heap dump
> showing all the Loggers I have allocated over 400,000 and from the spot
> checks I did, it was very hard to find one that wasn't a TimedLock logger.
> I'm looking for feedback on the correct course of action. The simplest would
> be removing the specificity and just using the logger for `TimedLock`. Or if
> we still want this specific debugging feature we could use a system property
> to gate whether we create `TimedLock` loggers with the instance ID or not.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
