Thanks for answering my questions Andy, Below are the sensitive properties:
# Provenance Repository Properties nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository nifi.provenance.repository.debug.frequency=1_000_000 *nifi.provenance.repository.encryption.key=fbRg/ZgK7U8qJcrU||4nI1n1aRD0Tooq7TLSTyVDhkmX8* nifi.provenance.repository.encryption.key.protected=aes/gcm/256 nifi.provenance.repository.encryption.key.provider.location= nifi.provenance.repository.encryption.key.id= # security properties # *nifi.sensitive.props.key=jtZiGY+mZyHPQIc1||/IJnMQBBXKN7VNkwMf6Oo7vZmAs* nifi.sensitive.props.key.protected=aes/gcm/256 nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL nifi.sensitive.props.provider=BC nifi.sensitive.props.additional.keys= nifi.security.keystore=/opt/certs/keystore.jks nifi.security.keystoreType=JKS *nifi.security.keystorePasswd=GuuOm4fyK6yvo76H||av/NQmH7Hw8qK9k0NOMRSjp08tw+walt4D5JLpYPiCHG/Z7DDq5QZ+ui/dKOXxtapH76Gjpt3hMwmP0* nifi.security.keystorePasswd.protected=aes/gcm/256 *nifi.security.keyPasswd=y4spsJvsy5Fzc3Uq||Q1vMntNgfLLMMSJuyPNn8+9aHlH+banQy82Ly0qrLWf6hNUTNgA+akyh86rlf2J5XZCONL3JCLX6mY0* nifi.security.keyPasswd.protected=aes/gcm/256 nifi.security.truststore=/opt/certs/truststore.jks nifi.security.truststoreType=JKS *nifi.security.truststorePasswd=9r+fyOSjRUXQLcZG||YwAtPYorADqHSKFUmU4H3SbyqvYqqYNZiGidgCOUCibPdP2jiEAMGtLt5xyFsMcNPm5Pye2qXEioLR8* nifi.security.truststorePasswd.protected=aes/gcm/256 These properties are generated by the toolkit. I using the same value for nifi.sensitive.props.key value and the nifi.provenance.repository.encryption.key, I was not aware they should be different. Could be that the problem? Juan On Mon, 9 Dec 2019 at 08:20, Andy LoPresto <[email protected]> wrote: > Hi Juan, > > The error you are getting is saying that one of the protected properties > is not of the expected format. While the Sensitive Property Provider > mechanism is extensible (see NIFI-5481 [1] for additional options being > added), the only natively supported one in 1.10.0 is AES/GCM encryption. > This requires the sensitive properties to be in the format > > > Wl9bXjSWX5DXs4Gm||EDnf18wwAAMJFckgNNfkRWiA4daSDWJCuRvSsbe99AaefQrkpmSqehJtyJGgEbhn402zSyztXi1EGPU > > Where the segment preceding the “||” delimiter is the Base64-encoded 16 > byte initialization vector (IV), which is random and unique for each > property, and the segment following the delimiter is the Base64-encoded > cipher text. > > The error states that when NiFi tries to decrypt one of the five encrypted > properties (it does not specify which in this case), it is not encoded in > the proper form. Assuming you are using a strong key > for nifi.bootstrap.sensitive.key in conf/bootstrap.conf, you can share the > nifi.properties file with the encoded and encrypted values with this list > to be verified for format, as no one will be able to decrypt them. However, > if you do not wish to share them, please validate that they are all of the > format specified above and encrypted with the same key that is present in > bootstrap.conf. > > Another thing I noted is that you are replacing the > nifi.sensitive.props.key value and the > nifi.provenance.repository.encryption.key value with the same environment > variable. These keys should not have the same value. The provenance > repository key is designed to protect the provenance repository on disk and > be rotated/migrated automatically. The formatting and provision of these > keys is documented in the User Guide [2]. The key can be present in > plaintext (raw hexadecimal encoding) or encrypted as any other sensitive > configuration value in the nifi.properties file. > > The nifi.sensitive.props.key value is a password or other key derivation > material used by NiFi to derive a strong key to encrypt the sensitive > _property_ values - this means things like database passwords, FTP server > passwords, keystore passwords, etc. that the NiFi flow uses and persists in > an encrypted format in the flow.xml.gz file. > > If you believe the sensitive properties key you are injecting into the > file is in the correct format (encoded as described above), check the value > of your master key to ensure it is the same key that encrypted that value. > If you are injecting a plaintext value like > “my_bad_sensitive_props_password”, you must remove the master key from the > bootstrap.conf file and ensure there is no sibling property present called > NiFi.sensitive.props.key.protected which indicates that the value must be > decrypted. > > I.e. the existing section like: > > > nifi.sensitive.props.key=xPqEWK8a34r19J4z||UOFzOfZE/NQK4Xua8WWblf1/Ld+Pf7eQ1zg0U/qYW2sPwxyhhOXWwQmrUft6qA > nifi.sensitive.props.key.protected=aes/gcm/128 > > Should change to look like: > > nifi.sensitive.props.key=my_bad_sensitive_props_password > NiFi.sensitive.props.key.protected= # or remove this line entirely > > > [1] https://github.com/apache/nifi/pull/3672 > [2] > https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#encrypted-provenance > > > Andy LoPresto > [email protected] > *[email protected] <[email protected]>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Dec 8, 2019, at 8:01 PM, Juan Pablo Gardella < > [email protected]> wrote: > > Hello all, > > I am trying to protect plain text passwords. I am using the latest docker > image (1.10.0), and edited manually nifi.sensitive.props.key as below > > sed -i -e > "s|^nifi.sensitive.props.key=.*$|nifi.sensitive.props.key=${NIFI_SENSITIVE_PROPS_KEY}|" > /opt/nifi/nifi-current/conf/nifi.properties > sed -i -e > "s|^nifi.provenance.repository.encryption.key=.*$|nifi.provenance.repository.encryption.key=${NIFI_SENSITIVE_PROPS_KEY}|" > /opt/nifi/nifi-current/conf/nifi.properties > > (this command for some reason does not update the file inside the > Dockerfile, I have to do inside the container). > > After updated that property, I run following command inside the container: > > bash /opt/nifi/nifi-toolkit-current/bin/encrypt-config.sh -n > /opt/nifi/nifi-current/conf/nifi.properties -b > /opt/nifi/nifi-current/conf/bootstrap.conf -a > /opt/nifi/nifi-current/conf/authorizers.xml -l > /opt/nifi/nifi-current/conf/login-identity-providers.xml > > It prompts to put a master password and after that, I restart[1] the > container but it failed to start with below error: > > nifi | 2019-12-08 18:57:31,777 INFO [main] > o.a.nifi.properties.NiFiPropertiesLoader Loaded 162 properties from > /opt/nifi/nifi-current/./conf/nifi.properties > *nifi | 2019-12-08 18:57:31,933 INFO [main] > o.a.n.properties.ProtectedNiFiProperties There are 5 protected properties > of 5 sensitive properties (100%)* > nifi | 2019-12-08 18:57:31,935 ERROR [main] org.apache.nifi.NiFi > Failure to launch NiFi due to java.lang.IllegalArgumentException: There was > an issue decrypting protected properties > nifi | java.lang.IllegalArgumentException: There was an issue > decrypting protected properties > nifi | at org.apache.nifi.NiFi.initializeProperties(NiFi.java:341) > nifi | at > org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:309) > nifi | at org.apache.nifi.NiFi.main(NiFi.java:300) > nifi | Caused by: java.lang.IllegalArgumentException: The cipher > text does not contain the delimiter || -- it should be of the form > Base64(IV) || Base64(cipherText) > nifi | at > org.apache.nifi.properties.AESSensitivePropertyProvider.unprotect(AESSensitivePropertyProvider.java:217) > nifi | at > org.apache.nifi.properties.ProtectedNiFiProperties.unprotectValue(ProtectedNiFiProperties.java:524) > nifi | at > org.apache.nifi.properties.ProtectedNiFiProperties.getUnprotectedProperties(ProtectedNiFiProperties.java:343) > nifi | at > org.apache.nifi.properties.NiFiPropertiesLoader.load(NiFiPropertiesLoader.java:209) > nifi | at > org.apache.nifi.properties.NiFiPropertiesLoader.load(NiFiPropertiesLoader.java:223) > nifi | at > org.apache.nifi.properties.NiFiPropertiesLoader.loadDefault(NiFiPropertiesLoader.java:130) > nifi | at > org.apache.nifi.properties.NiFiPropertiesLoader.get(NiFiPropertiesLoader.java:241) > nifi | at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > nifi | at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > nifi | at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > nifi | at java.lang.reflect.Method.invoke(Method.java:498) > nifi | at org.apache.nifi.NiFi.initializeProperties(NiFi.java:336) > nifi | ... 2 common frames omitted > > Any idea why it is failing? > > Thanks, > Juan > > [1] Actually, after that command two entries are generated to > nifi.provenance.repository.encryption.key= in the file, one with the plain > text and the other encrypted. I have to remove manually the plain text one. > > >
