Hi Josef, Thank you for raising the attention to this.
The storageLocation is a new concept to generalise the information used to locate nested flows in a versioned flow. In general it can be there and does not cause issues, as of now NiFi falls back the previous way to locate nested flows. As Joe mentioned, there is a fix in 1.19, but I need to double check if it relates to this situation. Please give me some time, I will find you back. Regards, Bence > On 2022. Nov 28., at 17:17, Joe Witt <joe.w...@gmail.com> wrote: > > Josef > > Sorry for the challenges you've hit there. I do think in 1.18 we had a > bug/regression as we refactored our registry client. That issue should be > resolved in 1.19 which just went live thanks to > https://issues.apache.org/jira/browse/NIFI-10787 > <https://issues.apache.org/jira/browse/NIFI-10787>. However, I am not > positive if this will solve the scenario you've hit now but please if able > try it out. > > Thanks > > On Mon, Nov 28, 2022 at 8:50 AM <josef.zahn...@swisscom.com > <mailto:josef.zahn...@swisscom.com>> wrote: > Hi guys > > > > We had the following bug twice already and it broke the whole sync with the > NiFi registry for the Flow/PG. First time was directly after we have upgraded > from NiFi 1.15.3 to 1.18.0, but we ignored it as we thought it could be > because of the upgrade, however it occurred again after a few NiFi Registry > commits on NiFi 1.18.0… The error was the following when we tried to change > the flow version or when we tried to start from scratch with that version > from the NiFi Registry, so the version was broken in the NiFi Registry: > > > > <image001.png> > > > > > > We investigated the last NiFi Registry Commit and we saw in our GIT repo (we > sync the “flow_storage” to GIT) on line 4847 that NiFi removed a > “storageLocation” from another PG which makes no sense at all. We changed > nothing there and especially why should NiFi remove only the storageLocation > line… We have one specialty, as we have nested NiFi Registry flows, so one of > the flows where the storageLocation has been removed was such a nested flow. > > > > > > <image002.png> > > > > > > Luckily we were able to resolve the error. We tried to add the line and > commit it to GIT plus we dropped the the database to repopulate the DB, > however it was broken again after a commit from NiFi. So we tried to manually > create a new fake version on bucket.yml in the corresponding bucket folder > and added as well the line in the snapshot again. We dropped then the DB and > restarted NiFi Registry and voilà it was working again. > > > > However it was a nightmare to get it working again as the flow was completely > broken, we couldn’t checkout the affected version at all. > > > > Any thoughts on this? Shall I fill a Jira Ticket? The problem is, we can’t > really reproduce it, it looks like it happens randomly. As you could imagine, > we can’t share our template as it contains a lot of confidential material. > > > > Cheers Josef > > >
