FrederikP commented on issue #3518: NIFI-6322: Introduced EvaluationContext to store state while making evaluator tree reusable URL: https://github.com/apache/nifi/pull/3518#issuecomment-501603608 Hi @markap14 , thanks for your in-depth answer. Actually the approach you are taking in #3531 is pretty similar to what I've done in PR #3500 that I decided to close in the end. I have commented in detail why I did so on that PR, but to summarize: - profiling showed that iterating over the `allEvaluators` `HashSet` to reset is pretty expensive, I fixed that later on it that PR - most importantly: this approach works quite well single threaded but it is not thread safe at all! Thread safety is not (and probably cannot be caught deterministically) in test cases, but we found out by trying out in our test cluster with concurrency enabled. One thread will call reset and removing the state, while another is in the midst of using the state and NPEs will be thrown or (even worse) logical errors will happen without any exception. I also think that, while the approach of this PR (#3518 ) is a little bit more complex, state should be separated from the evaluators. I also think that garbage collecting the state is not much worse that garbage collecting after the reset. Performance testing on our side supports that. And for either approach: The developer of a new stateful evaluator needs to be aware that he/she needs to (1) implement clean up or (2) use the `EvaluationContext`, so there is not much difference there. My conclusion: I think the approach you take in #3531 is pretty much the same (without some optimizations) that I took in the final version of #3500. And that approach is not thread safe and will lead to NPEs and other issues when used concurrently. I still think this PR (#3518) is the best option of all those PRs.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
