Hi Josef, Thanks for your patience!
I took a deeper look on what you were writing. In general, this is a sign for the case where the registry client cannot find the nested flow. As far as I understood in your case this happens when you are having a sync. Based on this you may be hitting https://issues.apache.org/jira/browse/NIFI-10874 <https://issues.apache.org/jira/browse/NIFI-10874> which is not part of NiFi 1.19.0. The next release should contain the fix that addresses your issue. Regards, Bence > On 2022. Nov 28., at 17:48, Simon Bence <[email protected]> wrote: > > Hi Josef, > > Thank you for raising the attention to this. > > The storageLocation is a new concept to generalise the information used to > locate nested flows in a versioned flow. In general it can be there and does > not cause issues, as of now NiFi falls back the previous way to locate nested > flows. As Joe mentioned, there is a fix in 1.19, but I need to double check > if it relates to this situation. Please give me some time, I will find you > back. > > Regards, > Bence > >> On 2022. Nov 28., at 17:17, Joe Witt <[email protected]> wrote: >> >> Josef >> >> Sorry for the challenges you've hit there. I do think in 1.18 we had a >> bug/regression as we refactored our registry client. That issue should be >> resolved in 1.19 which just went live thanks to >> https://issues.apache.org/jira/browse/NIFI-10787. However, I am not >> positive if this will solve the scenario you've hit now but please if able >> try it out. >> >> Thanks >> >> On Mon, Nov 28, 2022 at 8:50 AM <[email protected]> wrote: >> Hi guys >> >> >> >> We had the following bug twice already and it broke the whole sync with the >> NiFi registry for the Flow/PG. First time was directly after we have >> upgraded from NiFi 1.15.3 to 1.18.0, but we ignored it as we thought it >> could be because of the upgrade, however it occurred again after a few NiFi >> Registry commits on NiFi 1.18.0… The error was the following when we tried >> to change the flow version or when we tried to start from scratch with that >> version from the NiFi Registry, so the version was broken in the NiFi >> Registry: >> >> >> >> <image001.png> >> >> >> >> >> >> We investigated the last NiFi Registry Commit and we saw in our GIT repo (we >> sync the “flow_storage” to GIT) on line 4847 that NiFi removed a >> “storageLocation” from another PG which makes no sense at all. We changed >> nothing there and especially why should NiFi remove only the storageLocation >> line… We have one specialty, as we have nested NiFi Registry flows, so one >> of the flows where the storageLocation has been removed was such a nested >> flow. >> >> >> >> >> >> <image002.png> >> >> >> >> >> >> Luckily we were able to resolve the error. We tried to add the line and >> commit it to GIT plus we dropped the the database to repopulate the DB, >> however it was broken again after a commit from NiFi. So we tried to >> manually create a new fake version on bucket.yml in the corresponding bucket >> folder and added as well the line in the snapshot again. We dropped then the >> DB and restarted NiFi Registry and voilà it was working again. >> >> >> >> However it was a nightmare to get it working again as the flow was >> completely broken, we couldn’t checkout the affected version at all. >> >> >> >> Any thoughts on this? Shall I fill a Jira Ticket? The problem is, we can’t >> really reproduce it, it looks like it happens randomly. As you could >> imagine, we can’t share our template as it contains a lot of confidential >> material. >> >> >> >> Cheers Josef >> >> >> >
