exceptionfactory commented on a change in pull request #5154:
URL: https://github.com/apache/nifi/pull/5154#discussion_r654806383
##########
File path:
nifi-commons/nifi-sensitive-property-provider/src/test/java/org/apache/nifi/properties/StandardSensitivePropertyProviderFactoryTest.java
##########
@@ -45,32 +48,38 @@
private static final String AD_HOC_KEY_HEX =
"123456789ABCDEFFEDCBA98765432101";
private static Path tempConfDir;
- private static Path mockBootstrapConf;
- private static Path mockNifiProperties;
+ private static Path bootstrapConf;
+ private static Path hashicorpVaultBootstrapConf;
+ private static Path nifiProperties;
private static NiFiProperties niFiProperties;
@BeforeClass
public static void initOnce() throws IOException {
Security.addProvider(new BouncyCastleProvider());
tempConfDir = Files.createTempDirectory("conf");
- mockBootstrapConf = Files.createTempFile("bootstrap",
".conf").toAbsolutePath();
+ bootstrapConf = Files.createTempFile("bootstrap",
".conf").toAbsolutePath();
+ hashicorpVaultBootstrapConf =
Files.createTempFile("bootstrap-hashicorp-vault", ".conf").toAbsolutePath();
Review comment:
This value appears to be causing failures on the Windows automated build
due to forward slashes being treated as escape characters when writing out the
temporary file contents. One approach used in
`SetSingleUserCredentials.getNiFiProperties()` is Apache Commons IO
`FilenameUtils.separatorsToUnix()`, which changes separators to forward slashes
on all platforms.
##########
File path:
nifi-commons/nifi-property-utils/src/main/java/org/apache/nifi/properties/AbstractBootstrapPropertiesLoader.java
##########
@@ -74,13 +76,27 @@ public String extractKeyFromBootstrapFile() throws
IOException {
* @throws IOException If the file is not readable
*/
public BootstrapProperties loadBootstrapProperties(final String
bootstrapPath) throws IOException {
- final Properties properties = new Properties();
final Path bootstrapFilePath =
getBootstrapFile(bootstrapPath).toPath();
- try (final InputStream bootstrapInput =
Files.newInputStream(bootstrapFilePath)) {
+ return loadBootstrapProperties(bootstrapFilePath,
getApplicationPrefix());
+ }
+
+ /**
+ * Loads a properties file into a BootstrapProperties object.
+ * @param bootstrapPath The path to the properties file
+ * @param propertyPrefix The property prefix to enforce
+ * @return The BootstrapProperties
+ * @throws IOException If the properties file could not be read
+ */
+ public static BootstrapProperties loadBootstrapProperties(final Path
bootstrapPath, final String propertyPrefix) throws IOException {
+ Objects.requireNonNull(bootstrapPath, "Bootstrap path must be
provided");
+ Objects.requireNonNull(propertyPrefix, "Property prefix must be
provided");
+
+ final Properties properties = new Properties();
+ try (final InputStream bootstrapInput =
Files.newInputStream(bootstrapPath)) {
properties.load(bootstrapInput);
- return new BootstrapProperties(getApplicationPrefix(), properties,
bootstrapFilePath);
+ return new BootstrapProperties(propertyPrefix, properties,
bootstrapPath);
} catch (final IOException e) {
- logger.error("Cannot read from bootstrap.conf file at {}",
bootstrapFilePath);
+ logger.debug("Cannot read from bootstrap.conf file at {}",
bootstrapPath);
Review comment:
Instead of changing this log method and others from error to debug,
removing the log and including the path in the exception message seems like a
more concise approach.
##########
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/HashiCorpVaultTransitSensitivePropertyProvider.java
##########
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+
+/**
+ * Uses the HashiCorp Vault Transit Secrets Engine to encrypt sensitive values
at rest.
+ */
+public class HashiCorpVaultTransitSensitivePropertyProvider extends
AbstractHashiCorpVaultSensitivePropertyProvider {
+ private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+ private static final String TRANSIT_PATH = "vault.transit.path";
+
+ HashiCorpVaultTransitSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) {
+ super(bootstrapProperties);
+ }
+
+ @Override
+ protected String getSecretsEnginePath(final BootstrapProperties
vaultBootstrapProperties) {
+ if (vaultBootstrapProperties == null) {
+ return null;
+ }
+ final String transitPath =
vaultBootstrapProperties.getProperty(TRANSIT_PATH);
+ // Validate transit path
+ try {
+
PropertyProtectionScheme.fromIdentifier(getProtectionScheme().getIdentifier(transitPath));
+ } catch (IllegalArgumentException e) {
+ throw new SensitivePropertyProtectionException(TRANSIT_PATH + "
contains unsupported characters");
Review comment:
It would be helpful to include the actual `transitPath` value, as well
as the `IllegalArgumentException` as the cause or part of the exception message.
##########
File path: nifi-docs/src/main/asciidoc/toolkit-guide.adoc
##########
@@ -466,6 +470,40 @@ The following are available options when targeting NiFi
Registry using the `--ni
* `-I`,`--outputIdentityProvidersXml <file>` The destination
_identity-providers.xml_ file containing protected config values.
* `--decrypt` Can be used with `-r` to
decrypt a previously encrypted NiFi Registry Properties file. Decrypted content
is printed to STDOUT.
+=== Protection Schemes
+The protection scheme can be selected during encryption using the
`--protectionScheme` flag. During migration, the former protection scheme is
specified using the `--oldProtectionScheme` flag. This distinction allows a
set of protected configuration files to be migrated not only to a new key, but
to a completely different protection scheme.
+
+==== AES_GCM
+The default protection scheme, `AES-G/CM` simply encrypts sensitive properties
and marks their protection as either `aes/gcm/256` or `aes/gcm/256` as
appropriate. This protection is all done within NiFi itself.
+
+==== HASHICORP_VAULT_TRANSIT
+This protection scheme uses HashiCorp Vault's Transit Secrets Engine
(https://www.vaultproject.io/docs/secrets/transit) to outsource encryption to a
configured Vault server. All HashiCorp Vault configuration is stored in the
`bootstrap-hashicorp-vault.conf` file, as referenced in the `bootstrap.conf` of
a NiFi or NiFi Registry instance. Therefore, when using the
HASHICORP_VAULT_TRANSIT protection scheme, the
`nifi(.registry)?.bootstrap.protection.hashicorp.vault.conf` property in the
`bootstrap.conf` specified using the `-b` flag must be available to the Encrypt
Configuration Tool and must be configured as follows:
+
+===== Required properties
+[options="header,footer"]
+|===
+|Property Name|Description|Default
+|`vault.uri`|The HashiCorp Vault URI (e.g., `https://vault-server:8200`). If
not set, this provider will be disabled.|_none_
+|`vault.authPropertiesFilename`|Filename of a properties file containing Vault
authentication properties. See the `Authentication-specific property keys`
section of
https://docs.spring.io/spring-vault/docs/2.3.x/reference/html/#vault.core.environment-vault-configuration";
for all authentication property keys. If not set, this provider will be
disabled.|_none_
+|`vault.transit.path`|The HashiCorp Vault `path` specifying the Transit
Secrets Engine (e.g., `nifi-transit`). Valid characters include alphanumeric,
dash, and underscore.|_none_
+|===
+
+===== Optional properties
+[options="header,footer"]
+|===
+|Property Name|Description|Default
+|`vault.connection.timeout`|The connection timeout of the Vault client|`5 secs`
+|`vault.read.timeout`|The read timeout of the Vault client|`15 secs`
+|`vault.ssl.enabledCipherSuites`|A comma-separated list of the enabled TLS
cipher suites|_none_
+|`vault.ssl.enabledProtocols`|A comma-separated list of the enabled TLS
protocols|_none_
+|`vault.ssl.key-store`|Path to a keystore. Required if the Vault server is
TLS-enabled|_none_
+|`vault.ssl.key-store-type`|Keystore type (JKS or PKCS12). Required if the
Vault server is TLS-enabled|_none_
Review comment:
Can the BCFKS format be supported here, or is this dependent on the
Spring Vault code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
