I've been going through mailing list archives, and looking at blog posts around Wait/Notify, and these don't seem to be the solution for my use case.
My basic use case is as follows. I have 4 DB tables, 3 of which are id/name pairs (office name, city, state), and the 4th table joins the 3 ids together to a new id which is used elsewhere in the database. Using NiFi to injest data from a different database system, we have to verify if that office is active, and if it isn't active or non-existent, create a new record, as well as any of the other 3 tables necessary. The first step, then, is to join the 4 tables together to search for the name fields, and if the join comes back with a row, use that top level id as an attribute. No problem, works fine. (FetchDatabaseTable -> AvroToJson -> EvaluateJsonPath, etc.) If the join comes back empty, I need to insert rows for the 3 pieces and then join them together. Ideally, this would be a flow of 3 PutSqls, then a connection back to the top level search of the database. (Currently I'm using a modified custom processor, LookupAttributeFromSQL, that Brett Ryan did in January 18th, before he worked on a SQLLookupService.) The problem is that I could have 2 records coming in with the same pieces of information, and because it's flow based, the check for the 4 table join will come up empty on the second record before the first record is done creating the 4 table records. I've investigated the Wait/Notify pattern, but the odd part for me is that you need to have a separate "initialization" of the Wait/Notify release signal indicator (https://gist.github.com/ijokarumawak/9e1a4855934f2bb9661f88ca625bd244) and that seems "hack-ish" to me. With all of that said, I was curious if there was a way to have a back pressure value of 1 into the Process Group, so that if there is a flow file anywhere in the Process Group, the flow file is unable to enter that Process Group? That way, the creation of the 4 records could be inside a process group, and no other flowfile can enter until that first flowfile has exited. Thanks for any insight, John
