[
https://issues.apache.org/jira/browse/NIFI-12236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793614#comment-17793614
]
Simon Bence commented on NIFI-12236:
------------------------------------
Hi [~joewitt]!
Thanks for your attention on the topic!
My understanding at the time was that your main concerns were about the
implementation's possible implications on the core functionality of NiFi such
as handlink different kinds of errors like corrupted database or insufficient
disk space. So the main focus of the change was to rule out possible situations
where an unexpected situation with QuestDb could prevent core functionality
from working properly. I think this part is aimed reasonably and are covered
with tests.
As for visualisation, I did not aim for replacing that part however it sound a
good idea to follow the changes on the backend. From my perspective the main
benefit is to have some persistent storage behind is to be able to access
statistics after NiFi has stopped (or shutdown abruptly).
I think in the long run the opportunity to "plug in" different kind of
statistics repositories just like with processors could (and maybe should) be
opened, but I did not aim for that this time. I consider this as a step forward
which might be used as a stepping stone for future developments, including
non-embedded solutions for persisting and other things. In many environments
having Datatod or even MySQL backend might be more satifsfying (again: with
this story I did not aim for that which does not closes out the possibility of
furute steps towards that either from me or other member of the community) but
I think, having an embedded solution comes with the merit of adding the
capability of persistent statistical information out of the box.
> Improving fault tolerancy of the QuestDB backed metrics repository
> ------------------------------------------------------------------
>
> Key: NIFI-12236
> URL: https://issues.apache.org/jira/browse/NIFI-12236
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Reporter: Simon Bence
> Assignee: Simon Bence
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Based on the related discussion on the dev email list, the QuestDB handling
> of the metrics repository needs to be improved to have better fault tolerance
> in order to be possible to use as a viable option for default metrics data
> store. This should primarily focus on handling unexpeted database events like
> corrupted database or loss of space on the disk. Any issues should be handled
> with an attempt to keep the database service healthy but in case of that is
> impossible, the priority is to keep NiFi and the core services running, even
> with the price of metrics collection / presentation outage.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
