Hi again, Sorry for not following up but other priorities came ahead…
Basically it’s still not working, I’ve tried several combinations and I still keep getting: “Failed to process Flow Configuration [./conf/flow.xml.gz] org.apache.nifi.encrypt.EncryptionException: Decryption Failed with Algorithm [AES/GCM/NoPadding]” After reading some documentation, for this purpose, assuming that the password configured in the nifi.properties file is “PasswordUsedOnNifiProperties”… I’ve tried and failed: File: nifi.properties nifi.sensitive.props.key=PasswordUsedOnNifiProperties nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL Cmd: ./nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256 ./nifi.sh set-sensitive-properties-key PasswordUsedOnNifiProperties I’ve tried and failed by setting algorithm to empty string: File: nifi.properties nifi.sensitive.props.key=PasswordUsedOnNifiProperties nifi.sensitive.props.algorithm= Cmd: ./nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256 ./nifi.sh set-sensitive-properties-key PasswordUsedOnNifiProperties I’ve tried and failed using the new toolkit (I was using toolkit version 1.13.3): Cmd: /apps/nifi-toolkit-1.18.0/bin/encrypt-config.sh -f /apps/nifi-1.18.0/conf/flow.xml.gz -n /apps/nifi-1.18.0/conf/nifi.properties -s PasswordUsedOnNifiProperties -A NIFI_ARGON2_AES_GCM_256 -x -v I’ve tried and failed doing the same but generating new files to debug: Cmd: /apps/nifi-toolkit-1.18.0/bin/encrypt-config.sh -f /apps/nifi-1.18.0/conf/flow.json.gz -g /apps/nifi-1.18.0/conf/flow2.json.gz -n /apps/nifi-1.18.0/conf/nifi.properties -o /apps/nifi-1.18.0/conf/nifi2.properties -A NIFI_PBKDF2_AES_GCM_256 -s PasswordUsedOnNifiProperties -x -v In this last one I noticed that the file flow2.json.gz got its passwords encrypted differently and a longer encrypt also. I’m kind of wondering if I can use this last command to generate these files on the side and then manipulate the nifi.properties file by changing the algorithm to NIFI_PBKDF2_AES_GCM_256 since it seems it is already encrypted despite the known errors/warnings, then I would rename these new files to the older ones and start nifi with: 1. New flow.json.gz file (apparently encrypted with NIFI_PBKDF2_AES_GCM_256 algorithm) 2. New nifi.properties file (with nifi.sensitive.props.algorithm property manipulated to NIFI_PBKDF2_AES_GCM_256) Since, unfortunately<https://www.google.com/search?sxsrf=ALiCzsYtztPHersBtg21lqlpGlc7DZ9CUw:1669302812266&q=unfortunately&spell=1&sa=X&ved=2ahUKEwiq4pbJjcf7AhXKzqQKHfK8DNwQkeECKAB6BAgGEAE>, Im getting nowhere with this and I need to migrate to version 1.18.0 in order to apply the bugfix that changes the serverConnectorFactory.setNeedClientAuth(wantClientAuth) to serverConnectorFactory.setWantClientAuth(wantClientAuth) I am needing help in a consequent situation. In order to reduce the size of the log generated from the deprecation warnings (WARN [Flow Service Tasks Thread-1] d.o.a.n.s.u.c.NiFiLegacyCipherProvider Insecure Cipher Provider Algorithm [PBEWITHMD5AND256BITAES-CBC-OPENSSL] generate salt requested) I’ve tried to LEVEL OFF that Warning from stateless-logback.xml file without success. On the tag <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> and <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender"> I changed the tag <pattern> so that I could see the full class name (although without success also…) <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> to <pattern>%date %level [%thread] %logger{140} %msg%n</pattern> On the stateless-logback.xml I inserted the following: <logger name="org.apache.nifi" level="INFO"/> <logger name="deprecation.org.apache.nifi.security.util.crypto.NiFiLegacyCipherProvider" level="OFF" /> It’s not working and I don’t understand why, the class name seems to be correct but I keep getting the same WARN. Sorry for the long email… Regards. Tiago Sebastião From: Tiago Luís Sebastião (DSI) Sent: 28 de outubro de 2022 09:48 To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: RE: NiFi 1.18.0 Sensitive Property broken after Upgrade Hi David, It’s a standalone deployment and runs directly on the server. Yes the command updated the flow.xml.gz/flow.json.gz and nifi.properties settings. Maybe I messed up the nifi.sensitive.props.key, I’ll run some more tests. Thanks for your help. Tiago From: David Handermann [mailto:exceptionfact...@apache.org] Sent: 27 de outubro de 2022 16:50 To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: NiFi 1.18.0 Sensitive Property broken after Upgrade Hi Tiago, The initial warning for the Insecure Cipher Provider Algorithm indicates the use of the deprecated setting as mentioned previously. The set-sensitive-properties-algorithm command looks correct, and should have updated the flow.xml.gz, flow.json.gz, and nifi.properties settings. The Decryption Failed message indicates that the nifi.sensitive.props.key value does not match the value used to encrypt the flow configuration, or that the algorithm does not match. Can you provide some additional details about the NiFi installation? Is this a standalone or clustered deployment, and is it running in a containerized environment, or directly on a server? Regards, David Handermann On Thu, Oct 27, 2022 at 10:35 AM Tiago Luís Sebastião (DSI) <tiago.luis.sebast...@cgd.pt<mailto:tiago.luis.sebast...@cgd.pt>> wrote: Hi all, I'm having the same “problem”. I upgraded nifi version from 1.17.0 to 1.18.0 and that same warning started to appear 500k times a day. " WARN [Flow Service Tasks Thread-1] d.o.a.n.s.u.c.NiFiLegacyCipherProvider Insecure Cipher Provider Algorithm [PBEWITHMD5AND256BITAES-CBC-OPENSSL] generate salt requested " A already had nifi.sensitive.props.key value defined from when we migrated to 1.15.3. With Nifi STOPPED and without changing any configuration on nifi.properties I executed the following: ./nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256 No errors found there, then I started Nifi and received the following errors: " WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down. org.apache.nifi.encrypt.EncryptionException: Decryption Failed with Algorithm [AES/GCM/NoPadding] " Since Nifi could not start anymore I reversed it... Now Im kind of stuck with this warning... Anyone knows what Im doing wrong? Tiago From: David Handermann [mailto:exceptionfact...@apache.org<mailto:exceptionfact...@apache.org>] Sent: 19 de outubro de 2022 13:41 To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: NiFi 1.18.0 Sensitive Property broken after Upgrade Hi Mike, The deprecation warning is not related to NIFI-10567 or Sensitive Dynamic Properties. Deprecation logging is a new feature added in NiFi 1.18.0 to highlight components and features that are targeted for removal in future major releases. The current administrator's guide has more details on deprecation logging. [1] Deprecation warnings do not impact operational behavior, but they do identify configuration settings that should be changed. In this particular case, the deprecation is related to the use of the insecure algorithm. NiFi 1.14.0 and following introduced new Sensitive Properties Key Algorithm settings, which should be used instead of the historical default value indicated in the warning. The new default value is NIFI_PBKDF2_AES_GCM_256, additional supported options are listed in the administrator's guide, [2] along with the command that can be run to update the Sensitive Properties Key Algorithm. [3] Feel free to follow up if you have additional questions. Regards, David Handermann [1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#deprecation-logging [2] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#property-encryption-algorithms [3] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#updating-the-sensitive-properties-algorithm On Wed, Oct 19, 2022 at 7:28 AM Mike S <88msha...@gmail.com<mailto:88msha...@gmail.com>> wrote: I upgraded from 1.16.2 to 1.18.0 and now see this warning in the log file. WARN [Flow Service Tasks Thread-1] d.o.a.n.s.u.c.NiFiLegacyCipherProvider Insecure Cipher Provider Algorithm [PBEWITHMD5AND256BITAES-CBC-OPENSSL] generate salt requested org.apache.nifi.deprecation.log.DeprecationException: Reference Class [org.apache.nifi.security.util.crypto.NiFiLegacyCipherProvider] ClassLoader [org.apache.nifi.nar.NarClassLoader[./work/nar/framework/nifi-framework-nar-1.18.0.nar-unpacked]] I read this here. NIFI-10567<https://issues.apache.org/jira/browse/NIFI-10567> Corrects the parsing of Sensitive Dynamic Properties read from the XML version of the flow configuration, in absence of the JSON version. The issue surfaces when upgrading to NiFi 1.17.0 or 1.18.0 from a version older than 1.16.0. The issue also requires the presence of a Parameter Context with a Sensitive value assigned to a component with a Sensitive Property. Upgrading from 1.16.0 and following is not a problem. It appears that all my ListS3 processors using sensitive properties are working. Is this related since 1.16.2 has the latest flow.json.gz file? Mike
