Hi Brett, I believe the section you want is here — PKCS #5 Password Based Encryption [1]. There is also brief discussion of the method here [2] and full documentation here [3].
I don’t have my Ruby environment set up right now, but basically in the example from the first link, the “pass_phrase” is the value of “nifi.sensitive.props.key”, the salt is nil I believe, and the cipher is instantiated with AES-256-CBC unless you have changed the value of “nifi.sensitive.props.algorithm”. The digest param defaults to MD5, and you’ll need to provide an iterations value of 0 (or 1 — it happens once, but when invoking it in Java, the value needs to be 0 for some reason). Play around with these values and one combination will match. The plaintext is then the raw value of the property you want to encrypt. Check out this test [4] to see code which verifies the new NiFi implementation is compatible with the legacy key derivation, and this test [5] to see an example of verification of Ruby key derivation using PBKDF2 [5]. [1] http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL.html#module-OpenSSL-label-PKCS+-235+Password-based+Encryption [2] http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL/Cipher.html#class-OpenSSL::Cipher-label-Choosing+a+key [3] http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL/Cipher.html#method-i-pkcs5_keyivgen <http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL/Cipher.html#method-i-pkcs5_keyivgen> [4] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/groovy/org/apache/nifi/processors/standard/util/crypto/OpenSSLPKCS5CipherProviderGroovyTest.groovy#L150 <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/groovy/org/apache/nifi/processors/standard/util/crypto/OpenSSLPKCS5CipherProviderGroovyTest.groovy#L150> [5] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/groovy/org/apache/nifi/processors/standard/util/crypto/PBKDF2CipherProviderGroovyTest.groovy#L303 <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/groovy/org/apache/nifi/processors/standard/util/crypto/PBKDF2CipherProviderGroovyTest.groovy#L303> Andy LoPresto [email protected] [email protected] PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jul 22, 2016, at 8:41 AM, Hite, Brett <[email protected]> wrote: > > I’ve been banging my head on this for a while now so reaching out again. Ruby > and encryption are topics I can learn more of for sure! > > The Ruby code you have below was helpful. Maybe this is my naiveté with Ruby, > but I can’t find a way to use EVP_BytesToKey or PBKDF1 with the OpenSSL > module [1]. Any tips or resources that you would recommend? > > [1] http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL.html > <http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL.html> > > Thanks, > > Brett Hite > [email protected] <mailto:[email protected]> > > From: Andy LoPresto [mailto:[email protected]] > Sent: Thursday, July 14, 2016 2:46 PM > To: [email protected] > Subject: Re: Passwords in EncryptContent > > Happy to help, Brett. I always like seeing people use the software in a > secure manner. If you can, you may want to publish your tool on GitHub. While > you don’t have to submit it back for inclusion in NiFi itself (and if it’s a > Ruby tool, it may not be correct for inclusion), there are many people who > share their personal extensions and tools for administering NiFi with the > public. > > I remembered I had written some Ruby code using OpenSSL for key derivation > and encryption verification for an earlier ticket, so take a look here [1] > for some examples that may help. Basically, you switch out the PBKDF2 > invocation with the EVP_BytesToKey (aka PKCS #5 v1.5 PBKDF1) and use an empty > salt, and change the cipher to AES-256-CBC. > > [1] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/openssl_pbkdf2.rb > > <https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/resources/openssl_pbkdf2.rb> > > > Andy LoPresto > [email protected] <mailto:[email protected]> > [email protected] <mailto:[email protected]> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jul 14, 2016, at 2:23 PM, Hite, Brett <[email protected] > <mailto:[email protected]>> wrote: > > Hi Andy, > > I think I need a little time to review your post, but this sounds exactly > like what I was looking for. I was looking for a way to create the encrypted > value stored within the “enc{ … }” tag. Thank you for translating my question > and for the quick response! > > Brett Hite > [email protected] <mailto:[email protected]> > > From: Andy LoPresto [mailto:[email protected] > <mailto:[email protected]>] > Sent: Thursday, July 14, 2016 2:14 PM > To: [email protected] <mailto:[email protected]> > Subject: Re: Passwords in EncryptContent > > Hi Brett, > > I’m not sure I understand your question completely, so let me try to describe > it and you can correct me where I get it wrong. > > You have some deployment system which uses a Ruby process to replace tokens > in a flow template with the “real” values, and one of the values that needs > to be set is the password used by an EncryptContent processor configured with > password-based encryption. (This much makes sense to me). > > What I am confused by is your reference to “hash values”. While in many > situations (most web applications, user databases, etc.) cryptographic > hashing is the correct way to protect a password or other sensitive value > when persisting to disk, this is only appropriate if the raw sensitive value > does not need to be retrieved. However, in this scenario, the password must > be usable in raw form to derive the key to encrypt content, so it cannot be > stored in a “hash value” format (irreversible), but rather encrypted > (reversible). > > In order to persist the encrypted form of this password, you need to run the > same encryption algorithm and use the same key as NiFi does. These are > exposed to you in nifi.properties using the keys “nifi.sensitive.props.key” > and > “nifi.sensitive.props.algorithm”. By default the key is blank, and the > algorithm is “PBEWITHMD5AND256BITAES-CBC-OPENSSL” — in English, that’s > Password-Based Encryption using a single iteration of MD5 digest over the > password (the previous property) and salt (none in this case), taking the > resulting 32 hexadecimal characters (16 bytes) as the first half of a 256 bit > (32 byte) key, then calculating the MD5 of this value concatenated with the > raw password and raw salt again as the second half. [1][2] That key is now > used with AES-256 in CBC mode [3] to encrypt the raw sensitive values and > persist them in the form “enc{hex-encoded-ciphertext}” in the flow (see below > for example). If you feel at this point that the default key derivation > function is not sufficiently strong, know that I agree with you and have > opened a Jira to increase the strength of this process [4]. > > Anyway, to answer (what I believe is your question), you can write Ruby code > to populate your template with the encrypted value by retrieving the > sensitive properties key from nifi.properties, use the Ruby OpenSSL bindings > [5] to derive the key and encrypt the password, and then encode it in > hexadecimal and wrap it with the “enc{“ and “}” tags. > > I would also suggest you look at the Variable Registry [6][7], upcoming > encrypted configuration files [8], and deterministic templates [9][10], as > these may provide an easier way to perform what you are looking for, or at > least inform your next steps if you wish to keep your Ruby template system > and move forward in a compatible manner. > > If this didn’t answer your question (or raised others), please reply and I’ll > filter my thoughts through someone with a more human understanding of the > system. > > > [1] https://www.openssl.org/docs/manmaster/crypto/EVP_BytesToKey.html > <https://www.openssl.org/docs/manmaster/crypto/EVP_BytesToKey.html> > [2] http://security.stackexchange.com/a/117654/16485 > <http://security.stackexchange.com/a/117654/16485> > [3] > https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Block_Chaining_.28CBC.29 > > <https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Cipher_Block_Chaining_.28CBC.29> > [4] https://issues.apache.org/jira/browse/NIFI-1465 > <https://issues.apache.org/jira/browse/NIFI-1465> > [5] http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL.html > <http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL.html> > [6] https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry > <https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry> > [7] https://github.com/apache/nifi/pull/5 > <https://github.com/apache/nifi/pull/5>71 > [8] https://issues.apache.org/jira/browse/NIFI- > <https://issues.apache.org/jira/browse/NIFI->1831 > [9] > https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows > > <https://cwiki.apache.org/confluence/display/NIFI/Configuration+Management+of+Flows> > > <processor> > <id>206f32b5-9f72-4762-a1d5-711f8669db60</id> > <name>EncryptContent</name> > <position x="1101.0" y="176.0"/> > <styles/> > <comment/> > <class>org.apache.nifi.processors.standard.EncryptContent</class> > ... > <property> > <name>Encryption Algorithm</name> > <value>AES_GCM</value> > </property> > <property> > <name>Password</name> > > <value>enc{3832072F21CB7448E3C268287AF7F2F38359EB7F7E0F22CCFF2432EBC358335A0616B5CDCC895827FE0B1478901D364A}</value> > </property> > </processor> > > > > Andy LoPresto > [email protected] <mailto:[email protected]> > [email protected] <mailto:[email protected]> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jul 14, 2016, at 1:23 PM, Hite, Brett <[email protected] > <mailto:[email protected]>> wrote: > > Hello, > > I have a flow file that is created from a Ruby template file (flow.xml.erb). > The template contains variables that the user can set that then get populated > when NiFi is set up. I have an EncryptContent processor and would like to > create a template variable for the Password property. Ideally, the user would > say “password = some_password” and the template variable would evaluate to > the hash value stored in the actual flow file. > > Is there a way that I can calculate the hash value given a plain text > password? I’ve looked around and haven’t found too much. The NiFi > Administration Guide has an Encryption Configuration section that doesn’t > quite answer my question. > > Thanks, > > Brett Hite > [email protected] <mailto:[email protected]>
signature.asc
Description: Message signed with OpenPGP using GPGMail
