I believe as soon as this is merged Joe Witt is planning to prepare the 1.19.1 RC candidate and open the vote. We should have a release by next week unless something comes up
On Dec 1, 2022 at 07:44:29, [email protected] wrote: > Hi Bence > > > > Ok got it, thanks a lot, So we will try to sync as less as possible. Let’s > hope that we will see a NiFi 1.19.1 soon… > > > > Cheers Josef > > > > *From: *Simon Bence <[email protected]> > *Reply to: *"[email protected]" <[email protected]> > *Date: *Thursday, 1 December 2022 at 13:42 > *To: *"[email protected]" <[email protected]> > *Subject: *Re: NiFi Registry Bug which brakes the flow sync with NiFi > 1.18.0 (and same version of the registry) on nested flows > > > > Hi Josef, > > > > I would like to inform you that there is an adjustment for the ticket I > mentioned in my previous mail, which is needed for the proper behaviour of > the function in every case. The change is in PR phase but I hope it will be > merged soon: https://github.com/apache/nifi/pull/6741 I would not > recommend to go forward without this additional change! > > > > Regards, > > Bence > > > > On 2022. Nov 29., at 16:21, Simon Bence <[email protected]> wrote: > > > > Hi Josef, > > > > Your welcome! In your case it looks like the information is lost during > transformations of the internal representation. > > > > For technical details, here you can take a look on the code changes: > https://github.com/apache/nifi/commit/df2147829742129c039b37c5d6f4f11aa54785a2 > > > > Regards, > > Bence > > > > On 2022. Nov 29., at 16:08, <[email protected]> < > [email protected]> wrote: > > > > Hi Bence, Joe > > > > Thank you guys for your fast response. > > > > @Bence you are right, the whole issue seems to be related to nested flows > which we try to sync. We are relying heavily on this as we are doing the > integration on one NiFi system and then sync it to production via the NiFi > Registry. Our parent flow contains multiple nested flows. > > > > Ok, as NIFI-10874 will be included in the next release we will skip the > 1.19.0 and wait for 1.19.1 / 1.20.0. Sadly it will probably take more than > a few days until we see the next release... Do you have an idea why the > flow with the nested flow could lose the storageLocation? Because, we just > change a small thing in one of the nested flows and suddenly one of the > other nested flows have been losing it’s storageLocation in the parent flow > and from this point the whole flow was broken. Now we are scared to change > anything as it could brake anytime again. > > > > Cheers Josef > > > > > > > > *From: *Simon Bence <[email protected]> > *Reply to: *"[email protected]" <[email protected]> > *Date: *Tuesday, 29 November 2022 at 13:31 > *To: *"[email protected]" <[email protected]> > *Subject: *Re: NiFi Registry Bug which brakes the flow sync with NiFi > 1.18.0 (and same version of the registry) on nested flows > > > > Hi Josef, > > > > Thanks for your patience! > > > > I took a deeper look on what you were writing. In general, this is a sign > for the case where the registry client cannot find the nested flow. As far > as I understood in your case this happens when you are having a sync. Based > on this you may be hitting > https://issues.apache.org/jira/browse/NIFI-10874 which is not part of > NiFi 1.19.0. The next release should contain the fix that addresses your > issue. > > > > Regards, > > Bence > > > > > On 2022. Nov 28., at 17:48, Simon Bence <[email protected]> wrote: > > Hi Josef, > > Thank you for raising the attention to this. > > The storageLocation is a new concept to generalise the information used to > locate nested flows in a versioned flow. In general it can be there and > does not cause issues, as of now NiFi falls back the previous way to locate > nested flows. As Joe mentioned, there is a fix in 1.19, but I need to > double check if it relates to this situation. Please give me some time, I > will find you back. > > Regards, > Bence > > > > On 2022. Nov 28., at 17:17, Joe Witt <[email protected]> wrote: > > Josef > > Sorry for the challenges you've hit there. I do think in 1.18 we had a > bug/regression as we refactored our registry client. That issue should be > resolved in 1.19 which just went live thanks to > https://issues.apache.org/jira/browse/NIFI-10787. However, I am not > positive if this will solve the scenario you've hit now but please if able > try it out. > > Thanks > > On Mon, Nov 28, 2022 at 8:50 AM <[email protected]> wrote: > Hi guys > > > > We had the following bug twice already and it broke the whole sync with > the NiFi registry for the Flow/PG. First time was directly after we have > upgraded from NiFi 1.15.3 to 1.18.0, but we ignored it as we > thought it could be because of the upgrade, however it occurred again after > a few NiFi Registry commits on NiFi 1.18.0… The error was the following > when we tried to change the flow version or when we tried to start from > scratch with that version from the NiFi Registry, so the version was broken > in the NiFi Registry: > > > > <image001.png> > > > > > > We investigated the last NiFi Registry Commit and we saw in our GIT repo > (we sync the “flow_storage” to GIT) on line 4847 that NiFi removed a > “storageLocation” from another PG which makes no sense at all. We changed > nothing there and especially why should NiFi remove only the > storageLocation line… We have one specialty, as we have nested NiFi > Registry flows, so one of the flows where the storageLocation has > been removed was such a nested flow. > > > > > > <image002.png> > > > > > > Luckily we were able to resolve the error. We tried to add the line and > commit it to GIT plus we dropped the the database to repopulate the DB, > however it was broken again after a commit from NiFi. So we tried to > manually create a new fake version on bucket.yml in the corresponding > bucket folder and added as well the line in the snapshot again. We dropped > then the DB and restarted NiFi Registry and voilà it was working again. > > > > However it was a nightmare to get it working again as the flow was > completely broken, we couldn’t checkout the affected version at all. > > > > Any thoughts on this? Shall I fill a Jira Ticket? The problem is, we can’t > really reproduce it, it looks like it happens randomly. As you could > imagine, we can’t share our template as it contains a lot of > confidential material. > > > > Cheers Josef > > > > >
