[
https://issues.apache.org/jira/browse/NIFI-14599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17954242#comment-17954242
]
Aljaz Hericko commented on NIFI-14599:
--------------------------------------
We are encountering the same issue and were planning on creating an issue. We
are running multiple instance of NiFi and it happens on some of those as it
depends on some factors.
We are using AwsSecretsManagerParameterProvider and from our investigation it
is not related to the parameter provider implementation as parameter provider
works and returns what is expected.
>From debugging the code the issue seems to be more easily reproducible in 2.0
>due to another bug with parameter deletion not working, which was recently
>reported - https://issues.apache.org/jira/browse/NIFI-14269
Since parameter are not removed from the parameter context it tries to update
them on every attempt and having multiple "broken" parameter contexts then
makes it so it attempts to update multiple parameter contexts on each attempt.
As the parameter deletion does not work it can be updated infinite number of
times and there will be no change on it. Due to the this bug every time we try
to update the parameter contexts we have multiple parameter contexts, which are
attempted to be updated. The first parameter context update increases the
revision by 1 and then the second update fails as it has a miss match in
revision versions. This is why the revision version goes up by 1 on each
attempt as one update goes through, but since deletion does not work it can
always "update" the same parameter context. As we have 20+ parameter context on
a server if there are multiple parameter contexts with an issue we are not able
to update any parameter context on the server using that parameter provider.
For now our workaround is to create a new parameter provider for a context we
want to update and is a huge problem and time sink for us.
For reproduction case there are 2 scenarios we found:
1. Delete a secret from existing parameter provider
2. Restart the server - the parameter loses the "value":"provided:parameter" in
flow.json configuration on synchronisation
3. After restart the parameter is in limbo and every attempted update fails on
the parameter context
4. This also breakes a different parameter context on the same parameter
provider with the error above
Initially we thought this was the only scenario so we cleaned up the flow.json
on every server by removing the problematic parameter by hand and internally
agreed to stop deleting parameters from secret manager and it worked for some
time, but it seems like there is another scenario which we assume is related to
flow configuration, flow registry and the parameter context information saved
on the flow as those parameters are appended to the parameter context.
Simplest way to see if this issue will occur on update is if after clicking on
"Fetch Parameters" on the parameter provider the Apply button can be clicked
immediately (if parameter context works as expected and does not have any
updates the button is grayed out until something is changed on the modal).
After the update is executed the fetch parameter also throws an error if we try
refetching the parameters.
> Parameter provider invalid revision issue
> -----------------------------------------
>
> Key: NIFI-14599
> URL: https://issues.apache.org/jira/browse/NIFI-14599
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.4.0
> Reporter: Yuanhao Zhu
> Priority: Major
>
> We recently upgrade to nifi 2.4.0 from 1.x and have noticed something weird
> about the parameter provider in 2.x. From time to time, for some reason, the
> parameter provider will complain about invalid revision given when fetching
> for parameters with nifi-cli. Specifically we are using
> AzureKeyvaultParameterProvider. And once this happens, the only way to get
> rid of it is the re-creation of the parameter provider.
> See the screenshot
> !https://community.cloudera.com/t5/image/serverpage/image-id/45570i701517E88FD12C48/image-size/large/is-moderation-mode/true?v=v2&px=999!
> And it looks like the revision manager was messed up that every time the new
> revision is always one version below, we are running nifi in a standalone
> mode and consecutively triggering of the parameter fetch will constantly get
> the invalid revision(each time the revision is up by 1) until the parameter
> provider is recreated, even restarting nifi won't help. I'm also quite sure
> there is only one person sending request to fetch for parameter and attempt
> to change the parameter provider at the same time so no concurrent operation.
>
> And once this problem happens, fetching parameters from UI also does not
> work, it would just fail silently and the value of the secrets remains
> unchanged.
>
> Also FYI, this never happened in 1.x
>
> Another FYI, if we restart the nifi container, it does not go away, and the
> revision number in the complaint will start over from 0
>
> Still have no idea how to reproduce it though, but will keep trying and log
> the activities in this ticket
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
