[
https://issues.apache.org/jira/browse/MINIFICPP-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16845793#comment-16845793
]
Mr TheSegfault commented on MINIFICPP-839:
------------------------------------------
[~bakaid] In taking a look at MINIFICPP-836 I'm thinking these issues are
interrelated. I'm not sure we fully unload allowing these cycles to be. Some
things arose as I expected:
We don't fully empty these connections ( allowing FlowFiles to persist ).
They should totally go away and all references severed.
We also have cases where our python interpreters don't "go away" property (PR
for this coming )
Stop is a notification mechanism in MiNiFi, not guaranteed, so some
processors could persist data ( this is not something we guarantee ).
Those cycles aren't broken. Once they are these connections and processors
are reclaimed. Some of this originated prior to me arriving on the project, but
I think the intent was that we should always stop in the following order: stop
processors, drain connections, sever connections and run data to auto
terminated processors, and ultimately we break the cycle. I think C2 breaks
this paradigm a little and hence I noticed what you did. Tests don't fully do
all of this, so I imagine that's a separate case. In my opinion, if we can
better ensure determinism through this reload procedure we will ultimately have
a better result.
Did you try ensuring these connections were severed and all processors were
stopped appropriately in your testing?
> Multiple memory leaks because of shared_ptr cycles
> --------------------------------------------------
>
> Key: MINIFICPP-839
> URL: https://issues.apache.org/jira/browse/MINIFICPP-839
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
> Reporter: Daniel Bakai
> Assignee: Daniel Bakai
> Priority: Blocker
> Labels: ReloadValidation
> Fix For: 0.7.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> There seems to be a Connection<->Processor shared_ptr cycle which means that
> the Processors never get destructed.
> This causes leaks in reloads and in tests.
> There are also other cycles causing leaks:
> * Connection <-> FlowFile (this seems to be partially mitigated by
> Connection::drain(), but not entirely)
> * ResourceClaim <-> VolatileContentRepository
> * VolatileContentRepository <-> VolatileContentRepository (capturing a
> shared_ptr into a thread func and detaching the thread)
> * FlowController <- > SchedulingAgent (TimerDrivenSchedulingAgent,
> EventDrivenSchedulingAgent, CronDrivenSchedulingAgent) and even
> FlowController <- > util::ThreadPool because of this
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
