[
https://issues.apache.org/jira/browse/NIFI-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434409#comment-16434409
]
ASF GitHub Bot commented on NIFI-4942:
--------------------------------------
Github user alopresto commented on the issue:
https://github.com/apache/nifi/pull/2628
Here are some instructions and expected outputs to demonstrate that the
tool works as intended:
```
# CD to $NIFI_HOME/conf because secure_hash.key must be written to
immediate directory
# Populate sensitive properties in nifi.properties in order for something
to be encrypted
sed 's/asswd=/&thisIsABadPassword/' nifi.properties
>nifi-sensitive.properties
# Initial encryption of nifi.properties
../../../../../nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT/bin/encrypt-config.sh
-v \
-b bootstrap.conf \
-n nifi-sensitive.properties \
-o nifi-encrypted.properties \
-p passwordpassword
# Example hashes for "passwordpassword"
#
secureHashKey=$s0$100801$H8N5sEErC9hOVpQLxUt+oA$RrwImM1uWD59KuA1AxFamK7oPHlnI1uBXEN2lt4CpbM
#
secureHashPassword=$s0$100801$dZ04VTEBHxTR8tb6j29q/w$mXsXKxvd3nYXXOSoxobO7gkLaLAdz2dZRqAvPNfOzWE
# Verify secure_hash.key file generated and populated w/ both key and
password hash
more secure_hash.key
# Derived key for "passwordpassword"
#
nifi.bootstrap.sensitive.key=A2EA52795B33AB2F21C93E7E820D08369F1448478C877F4C710D6E85FD904AE6
# Verify bootstrap.conf file updated with master key value
more bootstrap.conf
# Verify encryption of sensitive properties occurred
more nifi-sensitive.properties | grep 'assw'
more nifi-encrypted.properties | grep 'assw'
# Migration using raw password
../../../../../nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT/bin/encrypt-config.sh
-v -m \
-b bootstrap.conf \
-n nifi-encrypted.properties \
-o nifi-migrated.properties \
-p thisIsABadPassword \
-w passwordpassword
# Example hashes for "thisIsABadPassword"
#
secureHashKey=$s0$100801$Y5rcY+pECpOBw5JBT1esMw$OEfnR/cze9u6ZjHMbd6NzvQltz2cC0qskSH8XeiXcp4
#
secureHashPassword=$s0$100801$rxjtgO5m859l6aI1xHIjpA$jAqTpGrJNiTkcIei6HtbCuZmhkPnqDlC3G4RjxRtf18
# Migration using hashed password (single quote escape hash to avoid
dollar-sign variable evaluation)
../../../../../nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT/bin/encrypt-config.sh
-v -m \
-b bootstrap.conf \
-n nifi-migrated.properties \
-o nifi-migrated-from-hash.properties \
-p thisIsABadPassword2 \
-z
'$s0$100801$rxjtgO5m859l6aI1xHIjpA$jAqTpGrJNiTkcIei6HtbCuZmhkPnqDlC3G4RjxRtf18'
# Example output
hw12203:.../nifi/nifi-assembly/target/nifi-1.7.0-SNAPSHOT-bin/nifi-1.7.0-SNAPSHOT/conf
(NIFI-4942) alopresto
🔓 174714s @ 14:37:45 $
../../../../../nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT/bin/encrypt-config.sh
-v \
> -m -b bootstrap.conf \
> -n nifi-migrated.properties \
> -o nifi-migrated-from-hash.properties \
> -p thisIsABadPassword2 \
> -z
'$s0$100801$rxjtgO5m859l6aI1xHIjpA$jAqTpGrJNiTkcIei6HtbCuZmhkPnqDlC3G4RjxRtf18'
Listening for transport dt_socket at address: 8000
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of
nifi.properties
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: bootstrap.conf:
bootstrap.conf
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src) nifi.properties:
nifi-migrated.properties
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties:
nifi-migrated-from-hash.properties
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src)
login-identity-providers.xml: null
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest)
login-identity-providers.xml: null
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src) authorizers.xml:
null
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest) authorizers.xml:
null
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src) flow.xml.gz:
null
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz:
null
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Key migration mode activated
2018/04/11 14:38:28 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Secure hash argument present
2018/04/11 14:38:39 INFO [main]
org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 151 properties from
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.7.0-SNAPSHOT-bin/nifi-1.7.0-SNAPSHOT/conf/nifi-migrated.properties
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 151 properties from
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.7.0-SNAPSHOT-bin/nifi-1.7.0-SNAPSHOT/conf/nifi-migrated.properties
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ProtectedNiFiProperties: There are 3 protected
properties of 5 sensitive properties (100%)
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Loaded NiFiProperties instance
with 147 properties
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Protected
nifi.security.keyPasswd with aes/gcm/256 ->
RDHo9b1qNOhGsvke||lOPiNxqlII0l6V07KsKQKSXgaZPmphdKLnL9QQB9LsGLLQ
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Updated protection key
nifi.security.keyPasswd.protected
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Protected
nifi.security.keystorePasswd with aes/gcm/256 ->
/IefK+XgVTpqGSFt||5O8oPFVXMYc6T1WK99++EWJ92V1TIlF9M7+H7xDnno6+mw
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Updated protection key
nifi.security.keystorePasswd.protected
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Protected
nifi.security.truststorePasswd with aes/gcm/256 ->
NyF6m6sg7ZH+eZF1||/ex+7LMFmD9iZ964k/BwlzA+Gd9z6Uuv5S5xv2oHQc9ynQ
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Updated protection key
nifi.security.truststorePasswd.protected
2018/04/11 14:39:03 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Final result: 150 keys
including 3 protected keys
# Example hashes for "thisIsABadPassword2"
#
secureHashKey=$s0$100801$HdGObf10x+ou+Vru5kns4A$U7kDSAwpoyvrWg7P9iRZxersUPln51TQ2IyKJYkiuTI
#
secureHashPassword=$s0$100801$h+9JoIzRU8ENNrKLECVPSQ$rZoWTxMe0er7EwW+ZtaWc1TF5N1oV19y1LXK4AjgZ/A
# Migration using hashed key
../../../../../nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT/bin/encrypt-config.sh
-v -m \
-b bootstrap.conf \
-n nifi-migrated-from-hash.properties \
-o nifi-migrated-from-hash-using-key.properties \
-p thisIsABadPassword3 \
-y
'$s0$100801$HdGObf10x+ou+Vru5kns4A$U7kDSAwpoyvrWg7P9iRZxersUPln51TQ2IyKJYkiuTI'
# Example hashes for "thisIsABadPassword3"
#
secureHashKey=$s0$100801$t7L8nq9Bc2gtbCnodjKG+A$i4z4dFHnV7CL3LlC5yt8N56me4pQUMx1Z0buZw0MsHA
#
secureHashPassword=$s0$100801$SDL+y7LI1HHxeBo4SEe1ig$OtfQWc4cHtTd0msxbgYytVjS0mB2VDm/z8S78sBjcGI
# Migration using "correct" hashed password but with incorrect master key
../../../../../nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT/bin/encrypt-config.sh
-v -m \
-b bootstrap.conf \
-n nifi-migrated.properties \
-o nifi-migrated-from-hash.properties \
-p thisIsABadPassword2 \
-z
'$s0$100801$t7L8nq9Bc2gtbCnodjKG+A$i4z4dFHnV7CL3LlC5yt8N56me4pQUMx1Z0buZw0MsHA'
# Example output
hw12203:.../nifi/nifi-assembly/target/nifi-1.7.0-SNAPSHOT-bin/nifi-1.7.0-SNAPSHOT/conf
(NIFI-4942) alopresto
🔓 175013s @ 14:42:44 $
../../../../../nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT/bin/encrypt-config.sh
-v -m \
> -b bootstrap.conf \
> -n nifi-migrated.properties \
> -o nifi-migrated-from-hash.properties \
> -p thisIsABadPassword2 \
> -z
'$s0$100801$t7L8nq9Bc2gtbCnodjKG+A$i4z4dFHnV7CL3LlC5yt8N56me4pQUMx1Z0buZw0MsHA'
Listening for transport dt_socket at address: 8000
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of
nifi.properties
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: bootstrap.conf:
bootstrap.conf
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src) nifi.properties:
nifi-migrated.properties
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties:
nifi-migrated-from-hash.properties
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src)
login-identity-providers.xml: null
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest)
login-identity-providers.xml: null
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src) authorizers.xml:
null
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest) authorizers.xml:
null
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (src) flow.xml.gz:
null
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz:
null
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Key migration mode activated
2018/04/11 14:45:58 INFO [main]
org.apache.nifi.properties.ConfigEncryptionTool: Secure hash argument present
2018/04/11 14:46:15 INFO [main]
org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 151 properties from
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.7.0-SNAPSHOT-bin/nifi-1.7.0-SNAPSHOT/conf/nifi-migrated.properties
2018/04/11 14:46:21 ERROR [main]
org.apache.nifi.properties.ConfigEncryptionTool: Encountered an error
java.security.InvalidKeyException: The provided hashed key/password is not
correct
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
at
org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
at
org.apache.nifi.properties.ConfigEncryptionTool.getMigrationKey(ConfigEncryptionTool.groovy:553)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
at
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at
org.apache.nifi.properties.ConfigEncryptionTool.main(ConfigEncryptionTool.groovy:1677)
at org.apache.nifi.properties.ConfigEncryptionTool$main.call(Unknown
Source)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at
org.apache.nifi.toolkit.encryptconfig.LegacyMode.run(LegacyMode.groovy:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:169)
at
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
at
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at
org.apache.nifi.toolkit.encryptconfig.EncryptConfigMain.main(EncryptConfigMain.groovy:109)
The provided hashed key/password is not correct
usage: org.apache.nifi.properties.ConfigEncryptionTool [-h] [-v] [-n
<file>] [-o <file>] [-l <file>] [-i <file>] [-a <file>] [-u <file>] [-f <file>]
[-g <file>]
[-b <file>] [-k <keyhex>] [-e <keyhex>] [-p <password>] [-w
<password>] [-y <hashed_keyhex>] [-z <hashed_password>] [-r] [-m] [-x] [-s
<password|keyhex>]
[-A <algorithm>] [-P <algorithm>] [--currentHashParams]
This tool reads from a nifi.properties and/or login-identity-providers.xml
file with plain sensitive configuration values, prompts the user for a master
key,
and encrypts each value. It will replace the plain value with the protected
value in the same file (or write to a new file if specified). It can also be
used to
migrate already-encrypted values in those files or in flow.xml.gz to be
encrypted with a new key.
-h,--help Show usage information (this
message)
-v,--verbose Sets verbose mode (default
false)
-n,--niFiProperties <file> The nifi.properties file
containing unprotected config values (will be overwritten unless -o is
specified)
-o,--outputNiFiProperties <file> The destination
nifi.properties file containing protected config values (will not modify input
nifi.properties)
-l,--loginIdentityProviders <file> The
login-identity-providers.xml file containing unprotected config values (will be
overwritten unless -i is
specified)
-i,--outputLoginIdentityProviders <file> The destination
login-identity-providers.xml file containing protected config values (will not
modify input
login-identity-providers.xml)
-a,--authorizers <file> The authorizers.xml file
containing unprotected config values (will be overwritten unless -u is
specified)
-u,--outputAuthorizers <file> The destination
authorizers.xml file containing protected config values (will not modify input
authorizers.xml)
-f,--flowXml <file> The flow.xml.gz file currently
protected with old password (will be overwritten unless -g is specified)
-g,--outputFlowXml <file> The destination flow.xml.gz
file containing protected config values (will not modify input flow.xml.gz)
-b,--bootstrapConf <file> The bootstrap.conf file to
persist master key
-k,--key <keyhex> The raw hexadecimal key to use
to encrypt the sensitive properties
-e,--oldKey <keyhex> The old raw hexadecimal key to
use during key migration
-p,--password <password> The password from which to
derive the key to use to encrypt the sensitive properties
-w,--oldPassword <password> The old password from which to
derive the key during migration
-y,--secureHashKey <hashed_keyhex> The old securely-hashed
hexadecimal key to authenticate during key migration (see NiFi Admin Guide)
-z,--secureHashPassword <hashed_password> The old securely-hashed
password to authenticate during key migration (see NiFi Admin Guide)
-r,--useRawKey If provided, the secure
console will prompt for the raw key value in hexadecimal form
-m,--migrate If provided, the
nifi.properties and/or login-identity-providers.xml sensitive properties will
be re-encrypted with
a new key
-x,--encryptFlowXmlOnly If provided, the properties in
flow.xml.gz will be re-encrypted with a new key but the nifi.properties and/or
login-identity-providers.xml
files will not be modified
-s,--propsKey <password|keyhex> The password or key to use to
encrypt the sensitive processor properties in flow.xml.gz
-A,--newFlowAlgorithm <algorithm> The algorithm to use to
encrypt the sensitive processor properties in flow.xml.gz
-P,--newFlowProvider <algorithm> The security provider to use
to encrypt the sensitive processor properties in flow.xml.gz
--currentHashParams Returns the current salt and
cost params used to store the hashed key/password
Java home: /Users/alopresto/.jenv/versions/1.8
NiFi Toolkit home:
/Users/alopresto/Workspace/nifi/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.7.0-SNAPSHOT-bin/nifi-toolkit-1.7.0-SNAPSHOT
hw12203:.../nifi/nifi-assembly/target/nifi-1.7.0-SNAPSHOT-bin/nifi-1.7.0-SNAPSHOT/conf
(NIFI-4942) alopresto
🔓 175232s @ 14:46:23 $
```
> NiFi Toolkit - Allow migration of master key without previous password
> ----------------------------------------------------------------------
>
> Key: NIFI-4942
> URL: https://issues.apache.org/jira/browse/NIFI-4942
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
> Affects Versions: 1.5.0
> Reporter: Yolanda M. Davis
> Assignee: Andy LoPresto
> Priority: Major
>
> Currently the encryption cli in nifi toolkit requires that, in order to
> migrate from one master key to the next, the previous master key or password
> should be provided. In cases where the provisioning tool doesn't have the
> previous value available this becomes challenging to provide and may be prone
> to error. In speaking with [~alopresto]Â we can allow toolkit to support a
> mode of execution such that the master key can be updated without requiring
> the previous password. Also documentation around it's usage should be updated
> to be clear in describing the purpose and the type of environment where this
> command should be used (admin only access etc).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
