Hi Nissim, I'm wondering if we need to store the set of sensitiveDynamicPropertyNames in flow.json or flow.xml. Currently when loading the flow we are recreating this set by looking at the property values and seeing which properties are dynamic and have an encrypted value. The problem you are highlighting is that once the component is ghosted, the flow.json/flow.xml now has all encrypted values and there is no way to get back to the original set of sensitiveDynamicPropertyNames. For flow.json it would likely be in VersionedConfigurableExtension where the other property fields are. For flow.xml, it would require updates to the serialization/deserialization. When serializing the sensitiveDynamicPropertyNames, it would have to always retain the set as submitted when the processor was created/updated, and not base it off the current component's descriptors, otherwise it will have the same issue.
Curious to see what others think. Thanks, Bryan On Fri, Jun 30, 2023 at 10:46 AM Nissim Shiman <nshi...@yahoo.com.invalid> wrote: > Hello apache nifi devs, > > I have run into an issue that appears to require a different way of > handling ghost components than we are currently, and would like input > before proceeding further. > > > Specifically, this is related to > https://issues.apache.org/jira/browse/NIFI-11570 > > > Besides the example in the ticket, a good case where this issue can be > seen is with the ExecuteScript processor with two dynamic properties, one > sensitive and one not. On recovery from ghost component, both properties > will be sensitive. > > > The challenge is that on recovery from ghost component there no way of > knowing if a dynamic property is sensitive or not, so it is left in the > existing sensitive/encrypted state. > > > There are solutions, but each is a shift from how nifi is currently > working. > > 1. Leave unencrypted properties in the clear when entering ghost > component mode. This would work, but we'd lose protections of why > encryption was done in the first place, so probably a non-starter. > > 2. When encrypting properties in ghost mode, have a custom encryption tag > in flow.json.gz/flow.xml.gz. i.e. instead of sensitive values starting > with enc{ they would start with enc-ghost{ This will help when exiting > ghost mode where we could now more easily determine why a value was > encrypted in the first place, but is introducing new complexities in > keeping track of why something is encrypted > > > 3. Leave handling of ghost components as is, but for a given component > only allow Dynamic Properties to be non-sensitive or senstive, but not > both. This would affect ExecuteScript, InvokeHTTP (and others) which > currently allow both and would be rolling back considerable work that was > done to allow this, and we'd lose existing functionality, so also not > likely. > > > 4. Leave as is. It is not so common to have components in ghost state, > and only some of them allow Dynamic Properties so this is an acceptable > risk considering the alternatives. > > I am interested in community feedback and/or other suggestions to the > current situation. > > > Thanks, > > > Nissim Shiman > > > > > >
