[GitHub] [nifi] s9514171 commented on pull request #4905: NIFI-8332 Service account impersonation support for GCP credential co…
s9514171 commented on pull request #4905: URL: https://github.com/apache/nifi/pull/4905#issuecomment-880370626 Hi @pvillard31 sorry for ping you, does this pull request need any changes it's running on our servers for a while and runs well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jfrazee commented on pull request #5216: NIFI-8783 Moved SingleUserAuthorizer Login Provider check
jfrazee commented on pull request #5216: URL: https://github.com/apache/nifi/pull/5216#issuecomment-880301186 @bbende @exceptionfactory Thanks. After working through it, it's just a superficially similar thing -- any userGroupProvider present (uncommented) must have a valid configuration whether or not it or the managed-authorizer is being used. There's been no change in behavior, just new ways to bump into it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5216: NIFI-8783 Moved SingleUserAuthorizer Login Provider check
exceptionfactory commented on pull request #5216: URL: https://github.com/apache/nifi/pull/5216#issuecomment-880294530 Thanks for the feedback @jfrazee. As @bbende mentioned, this particular check is specific to the `SingleUserAuthorizer`. The purpose is to prevent the `SingleUserAuthorizer` from being used with a Login Identity Provider other than `SingleUserLoginIdentityProvider`. The `setProperties()` method that receives the instance of `NiFiProperties` gets called before `initialize()`, so moving the logic to `initialize()` allows the Authorizer to check its own identifier and thus be defined without necessarily being enabled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] github-actions[bot] commented on pull request #4856: NIFI-8275 Add NET_BIND_SERVICE capability to Java in Docker image
github-actions[bot] commented on pull request #4856: URL: https://github.com/apache/nifi/pull/4856#issuecomment-880289292 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bbende commented on pull request #5216: NIFI-8783 Moved SingleUserAuthorizer Login Provider check
bbende commented on pull request #5216: URL: https://github.com/apache/nifi/pull/5216#issuecomment-880259482 @jfrazee this issue was specially for single-user-group authorizer because it was doing a check to ensure that it can only be used with the single-user-group login identity provider. In general though, anything declared in the provider files should be able to be instantiated and configured, even if it is not directly used in NiFi properties. So if any other provider has an issue like this we should address that too with additional JIRAs/PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jfrazee commented on pull request #5216: NIFI-8783 Moved SingleUserAuthorizer Login Provider check
jfrazee commented on pull request #5216: URL: https://github.com/apache/nifi/pull/5216#issuecomment-880252565 @exceptionfactory There was a thread in Slack where someone noted similar behavior for the AzureGraphUserGroupProvider so I also checked the LdapUserGroupProvider and all will try to initialize whether or not they're being used resulting in startup failures, so I don't think this is a SingleUserAuthorizer issue alone. I checked some commits back in April and this wasn't happening. Not sure where the problem is yet though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] simonbence commented on a change in pull request #4948: NIFI-8273 Adding Scripted Record processors
simonbence commented on a change in pull request #4948:
URL: https://github.com/apache/nifi/pull/4948#discussion_r669899338
##
File path:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedPartitionRecord.java
##
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.script;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.Restricted;
+import org.apache.nifi.annotation.behavior.Restriction;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.RequiredPermission;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.record.PushBackRecordSet;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.RecordSet;
+
+import javax.script.ScriptEngine;
+import javax.script.ScriptException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.function.BiFunction;
+
+@EventDriven
+@SideEffectFree
+@Tags({"record", "partition", "script", "groovy", "jython", "python",
"segment", "split", "group", "organize"})
+@CapabilityDescription("Receives Record-oriented data (i.e., data that can be
read by the configured Record Reader) and evaluates the user provided script
against "
++ "each record in the incoming flow file. Each record is then grouped
with other records sharing the same partition and a FlowFile is created for
each groups of records. " +
+"Two records shares the same partition if the evaluation of the script
results the same return value for both. Those will be considered as part of the
same partition.")
+@Restricted(restrictions = {
+@Restriction(requiredPermission = RequiredPermission.EXECUTE_CODE,
+explanation = "Provides operator the ability to execute
arbitrary code assuming all permissions that NiFi has.")
+})
+@WritesAttributes({
+@WritesAttribute(attribute = "partition", description = "The partition
of the outgoing flow file."),
+@WritesAttribute(attribute = "mime.type", description = "Sets the
mime.type attribute to the MIME Type specified by the Record Writer"),
+@WritesAttribute(attribute = "record.count", description = "The number
of records within the flow file."),
+@WritesAttribute(attribute = "record.error.message", description =
"This attribute provides on failure the error message encountered by the Reader
or Writer."),
+@WritesAttribute(attribute = "fragment.index", description = "A one-up
number that indicates the ordering of the partitioned FlowFiles that were
created from a single parent FlowFile"),
+@WritesAttribute(attribute = "fragment.count", description = "The
number of partitioned FlowFiles generated from the parent FlowFile")
+})
+@SeeAlso(classNames = {
+"org.apache.nifi.processors.script.ScriptedTransformRecord",
+
[jira] [Commented] (NIFI-8742) Unable to view FlowFile Content in cluster mode
[
https://issues.apache.org/jira/browse/NIFI-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380815#comment-17380815
]
Matt Burgess commented on NIFI-8742:
I wasn't able to reproduce this on the latest main branch (slightly post-1.14.0
release)
> Unable to view FlowFile Content in cluster mode
> ---
>
> Key: NIFI-8742
> URL: https://issues.apache.org/jira/browse/NIFI-8742
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: Mark Payne
>Priority: Critical
>
> When I create some content and List Queue I can see the FlowFile in the
> queue. I can then download it. However, when I attempt to view it, I get a
> TimeoutException:
> {code:java}
> 2021-06-25 18:08:55,958 WARN [Replicate Request Thread-1]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
> /nifi-api/flowfile-queues/452afb8c-017a-1000--46f5f263/flowfiles/907f11da-666f-428f-9582-b9afb0ac107a/content
> to localhost:8481 due to java.net.SocketTimeoutException: timeout
> 2021-06-25 18:08:55,962 WARN [Replicate Request Thread-1]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> java.net.SocketTimeoutException: timeout
> at okio.SocketAsyncTimeout.newTimeoutException(JvmOkio.kt:143)
> at okio.AsyncTimeout.access$newTimeoutException(AsyncTimeout.kt:162)
> at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:335)
> at okio.RealBufferedSource.indexOf(RealBufferedSource.kt:427)
> at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.kt:320)
> at okhttp3.internal.http1.HeadersReader.readLine(HeadersReader.kt:29)
> at
> okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:178)
> at
> okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:106)
> at
> okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:79)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
> at
> okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
> at
> okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
> at
> okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
> at
> okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
> at
> okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
> at
> okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
> at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
> at
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:136)
> at
> org.apache.nifi.cluster.coordination.http.replication.okhttp.OkHttpReplicationClient.replicate(OkHttpReplicationClient.java:130)
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:640)
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:832)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
> at sun.security.ssl.InputRecord.read(InputRecord.java:503)
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:990)
> at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:948)
> at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
> at okio.InputStreamSource.read(JvmOkio.kt:90)
> at okio.AsyncTimeout$source$1.read(AsyncTimeout.kt:129)
> ... 26 common frames
[jira] [Updated] (NIFI-8783) SingleUserAuthorizer fails start up when not used
[ https://issues.apache.org/jira/browse/NIFI-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8783: --- Status: Patch Available (was: Open) > SingleUserAuthorizer fails start up when not used > - > > Key: NIFI-8783 > URL: https://issues.apache.org/jira/browse/NIFI-8783 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0, 1.15.0 >Reporter: Bryan Bende >Assignee: David Handermann >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > The SingleUserAuthorizer has code that checks to see if the > SinglueUserLoginIdentityProvider is configured, and fails start up if it > isn't. This is good to ensure they are only used together. > The issue is, if the SinglueUserAuthorizer is defined in authorizers.xml, but > it is not the configured authorizer in nifi.properties, it still performs > this check and fails start up. The check should be improved to account for > this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8782) Add Rate-Limiting for Access Token Requests
[ https://issues.apache.org/jira/browse/NIFI-8782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Gough updated NIFI-8782: --- Fix Version/s: 1.15.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add Rate-Limiting for Access Token Requests > --- > > Key: NIFI-8782 > URL: https://issues.apache.org/jira/browse/NIFI-8782 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, Security >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: authentication, jetty, security > Fix For: 1.15.0 > > Time Spent: 40m > Remaining Estimate: 0h > > The NiFi Jetty Server currently relies on the Jetty [Denial of Service > Filter|https://www.eclipse.org/jetty/documentation/jetty-9/index.html#dos-filter] > to provide configurable rate-limiting for HTTP requests. The DoSFilter > applies to all requests and setting to the limit too low can cause unexpected > problems during system administration or data transfer. > When configured with a Login Identity Provider, Access Token requests support > authenticating users against the specified provider. The number of Access > Token requests from a given remote address should be minimal and predictable > based on the expected number of authorized users. Introducing a separate > configuration property and targeted filter for Access Token requests will > allow the NiFi Jetty Server to reject excessive numbers of authentication > attempts while permitting higher numbers of requests to other resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8782) Add Rate-Limiting for Access Token Requests
[ https://issues.apache.org/jira/browse/NIFI-8782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380798#comment-17380798 ] ASF subversion and git services commented on NIFI-8782: --- Commit c668d3df1baa4dd40f727aaa2bc1fab697520913 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c668d3d ] NIFI-8782 Added Rate-Limiting for Access Token Requests - Added Jetty DoSFilter configured for /access/token - Added nifi.web.max.access.token.requests.per.second property with default value of 25 Signed-off-by: Nathan Gough This closes #5215. > Add Rate-Limiting for Access Token Requests > --- > > Key: NIFI-8782 > URL: https://issues.apache.org/jira/browse/NIFI-8782 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, Security >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: authentication, jetty, security > Time Spent: 40m > Remaining Estimate: 0h > > The NiFi Jetty Server currently relies on the Jetty [Denial of Service > Filter|https://www.eclipse.org/jetty/documentation/jetty-9/index.html#dos-filter] > to provide configurable rate-limiting for HTTP requests. The DoSFilter > applies to all requests and setting to the limit too low can cause unexpected > problems during system administration or data transfer. > When configured with a Login Identity Provider, Access Token requests support > authenticating users against the specified provider. The number of Access > Token requests from a given remote address should be minimal and predictable > based on the expected number of authorized users. Introducing a separate > configuration property and targeted filter for Access Token requests will > allow the NiFi Jetty Server to reject excessive numbers of authentication > attempts while permitting higher numbers of requests to other resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] thenatog closed pull request #5215: NIFI-8782 Added Rate-Limiting for Access Token Requests
thenatog closed pull request #5215: URL: https://github.com/apache/nifi/pull/5215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #5215: NIFI-8782 Added Rate-Limiting for Access Token Requests
thenatog commented on pull request #5215: URL: https://github.com/apache/nifi/pull/5215#issuecomment-880108460 +1 Tested this and got a HTTP 429 when sending too many requests in 1 second. Checked that other contexts were not affected by the changes, and the code changes look good to me, will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202: URL: https://github.com/apache/nifi/pull/5202#discussion_r669807702 ## File path: nifi-nar-bundles/nifi-grpc-bundle/nifi-grpc-processors/pom.xml ## @@ -75,6 +75,13 @@ language governing permissions and limitations under the License. --> nifi-mock 1.14.0-SNAPSHOT test + Review comment: Adjusting the comment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669807125
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java
##
@@ -173,8 +173,14 @@ public NiFiProperties load(final File file) {
.getSupportedSensitivePropertyProviders()
.forEach(protectedNiFiProperties::addSensitivePropertyProvider);
}
-
-return protectedNiFiProperties.getUnprotectedProperties();
+NiFiProperties props =
protectedNiFiProperties.getUnprotectedProperties();
+if (protectedNiFiProperties.hasProtectedKeys()) {
+// releases resources used by SPP
Review comment:
Removing the comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202: URL: https://github.com/apache/nifi/pull/5202#discussion_r669806838 ## File path: nifi-docs/src/main/asciidoc/toolkit-guide.adoc ## @@ -699,6 +719,39 @@ for each phase (old vs. new), and any combination is sufficient: In order to change the protection scheme (e.g., migrating from AES encryption to Vault encryption), specify the `--protectionScheme` and `--oldProtectionScheme` in the migration command. +The following is an example of the commands for protection scheme migration from AES_GCM to AWS_KMS then back. Execute these commands at the `nifi` directory with the `nifi-toolkit` directory as a sibling directory. In addition, make sure to update `bootstrap-aws.conf` with your AWS KMS Key ARN/ID and have your credentials and region for AWS configured. Review comment: Making the changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202: URL: https://github.com/apache/nifi/pull/5202#discussion_r669806075 ## File path: nifi-docs/src/main/asciidoc/toolkit-guide.adoc ## @@ -504,6 +504,26 @@ This protection scheme uses HashiCorp Vault's Transit Secrets Engine (https://ww |`vault.ssl.trust-store-password`|Truststore password. Required if the Vault server is TLS-enabled|_none_ |=== + AWS_KMS +This protection scheme uses AWS Key Management Service, or AWS KMS for short (https://aws.amazon.com/kms/) for encryption/decryption. All AWS KMS configuration/credentials details are to be stored in the `bootstrap-aws.conf` file, as referenced in the `bootstrap.conf` of a NiFi or NiFi Registry instance. If the configuration/credentials details are not fully specified in `bootstrap-aws.conf`, then the protection scheme will attempt to use the default AWS credentials/configuration chain. Therefore, when using the AWS_KMS protection scheme, the `nifi(.registry)?.bootstrap.protection.aws.kms.conf` property in the `bootstrap.conf` specified using the `-b` flag must be available to the Encrypt Configuration Tool and must be configured as follows: + += Required properties +[options="header,footer"] +|=== +|Property Name|Description|Default +|`aws.kms.key.id`|The key id or ARN to be used by AWS KMS to identify the key used for encryption/decryption.|_none_ Review comment: Making the changes ## File path: nifi-docs/src/main/asciidoc/toolkit-guide.adoc ## @@ -504,6 +504,26 @@ This protection scheme uses HashiCorp Vault's Transit Secrets Engine (https://ww |`vault.ssl.trust-store-password`|Truststore password. Required if the Vault server is TLS-enabled|_none_ |=== + AWS_KMS +This protection scheme uses AWS Key Management Service, or AWS KMS for short (https://aws.amazon.com/kms/) for encryption/decryption. All AWS KMS configuration/credentials details are to be stored in the `bootstrap-aws.conf` file, as referenced in the `bootstrap.conf` of a NiFi or NiFi Registry instance. If the configuration/credentials details are not fully specified in `bootstrap-aws.conf`, then the protection scheme will attempt to use the default AWS credentials/configuration chain. Therefore, when using the AWS_KMS protection scheme, the `nifi(.registry)?.bootstrap.protection.aws.kms.conf` property in the `bootstrap.conf` specified using the `-b` flag must be available to the Encrypt Configuration Tool and must be configured as follows: + += Required properties +[options="header,footer"] +|=== +|Property Name|Description|Default +|`aws.kms.key.id`|The key id or ARN to be used by AWS KMS to identify the key used for encryption/decryption.|_none_ +|=== + += Optional properties +== All of the following must be configured, or will be ignored entirely. +[options="header,footer"] +|=== +|Property Name|Description|Default +|`aws.region`|The region to configure AWS KMS Client with for encryption/decryption.|_none_ Review comment: Making the changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202: URL: https://github.com/apache/nifi/pull/5202#discussion_r669804912 ## File path: nifi-docs/src/main/asciidoc/toolkit-guide.adoc ## @@ -504,6 +504,26 @@ This protection scheme uses HashiCorp Vault's Transit Secrets Engine (https://ww |`vault.ssl.trust-store-password`|Truststore password. Required if the Vault server is TLS-enabled|_none_ |=== + AWS_KMS +This protection scheme uses AWS Key Management Service, or AWS KMS for short (https://aws.amazon.com/kms/) for encryption/decryption. All AWS KMS configuration/credentials details are to be stored in the `bootstrap-aws.conf` file, as referenced in the `bootstrap.conf` of a NiFi or NiFi Registry instance. If the configuration/credentials details are not fully specified in `bootstrap-aws.conf`, then the protection scheme will attempt to use the default AWS credentials/configuration chain. Therefore, when using the AWS_KMS protection scheme, the `nifi(.registry)?.bootstrap.protection.aws.kms.conf` property in the `bootstrap.conf` specified using the `-b` flag must be available to the Encrypt Configuration Tool and must be configured as follows: Review comment: Will make the changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669804229
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/StandardSensitivePropertyProviderFactory.java
##
@@ -105,6 +105,9 @@ public SensitivePropertyProvider getProvider(final
PropertyProtectionScheme prot
switch (protectionScheme) {
case AES_GCM:
return providerMap.computeIfAbsent(protectionScheme, s -> new
AESSensitivePropertyProvider(keyHex));
+// Other providers may choose to pass getBootstrapProperties()
into the constructor
Review comment:
Will remove it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202: URL: https://github.com/apache/nifi/pull/5202#discussion_r669803749 ## File path: nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/SensitivePropertyProvider.java ## @@ -56,4 +56,10 @@ * @return the raw value to be used by the application */ String unprotect(String protectedValue) throws SensitivePropertyProtectionException; + +/** + * Cleans up resources that may have been allocated/used by an SPP implementation + * Note: If there is nothing to be done, then this function is a no-op (i.e. no cleanup necessary) Review comment: Will remove this part of the comment -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669800473
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669800300
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669799880
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669798868
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669797862
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669797245
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669797080
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669796600
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669794639
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669794527
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669794244
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] exceptionfactory opened a new pull request #5216: NIFI-8783 Moved SingleUserAuthorizer Login Provider check
exceptionfactory opened a new pull request #5216:
URL: https://github.com/apache/nifi/pull/5216
Description of PR
NIFI-8783 Moves the check for `SingleUserLoginIdentityProvider` from
`setProperties()` to `initialize()` in `SingleUserAuthorizer`. This approach
allows the `SingleUserAuthorizer` definition to exist in `authorizers.xml` but
continues to prevent it from being used without the
`SingleUserLoginIdentityProvider` being configured as the selected Login
Identity Provider.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669793563
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669790392
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[jira] [Updated] (NIFI-8783) SingleUserAuthorizer fails start up when not used
[ https://issues.apache.org/jira/browse/NIFI-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8783: --- Affects Version/s: 1.15.0 1.14.0 > SingleUserAuthorizer fails start up when not used > - > > Key: NIFI-8783 > URL: https://issues.apache.org/jira/browse/NIFI-8783 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.14.0, 1.15.0 >Reporter: Bryan Bende >Assignee: David Handermann >Priority: Minor > > The SingleUserAuthorizer has code that checks to see if the > SinglueUserLoginIdentityProvider is configured, and fails start up if it > isn't. This is good to ensure they are only used together. > The issue is, if the SinglueUserAuthorizer is defined in authorizers.xml, but > it is not the configured authorizer in nifi.properties, it still performs > this check and fails start up. The check should be improved to account for > this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8754) CSVReader with InferSchema ignoring 1st non header row, even with necessary properties set
[ https://issues.apache.org/jira/browse/NIFI-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380743#comment-17380743 ] Matt Burgess commented on NIFI-8754: The docs kind of mention that if you have a header line, the choices for Schema Access Strategy are "Infer Schema" and "Use String Fields From Header". However it doesn't seem to mention the reverse, that if you have "Infer Schema" then you must have a header line. We can keep this Jira open to make the docs more clear and to make Treat First Line As Header a dependent property that only shows up when the Schema Access Strategy value will honor it. > CSVReader with InferSchema ignoring 1st non header row, even with necessary > properties set > -- > > Key: NIFI-8754 > URL: https://issues.apache.org/jira/browse/NIFI-8754 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 >Reporter: K Sanghavi >Priority: Major > Labels: bug > Attachments: CSVReaderBug.xml, csvreader-config.jpg, > csvreader-flow.jpg, csvreader-input.jpg, csvreader-output.jpg > > > When using CSVReader controller service in any Record based processors of > 1.13.2, is not working as expected in InferSchema mode, where it ignores the > 1st non header row even with *Treat First Line as Header* property set to > *false*. But with External schema reference the same controller service is > working as expected. (Attached images for SplitRecord with CSVReader) > This setup was tested in 1.11.4, and with same configuration for CSVReader > with InferSchema was giving expected results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669789260
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669789032
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[jira] [Assigned] (NIFI-8783) SingleUserAuthorizer fails start up when not used
[ https://issues.apache.org/jira/browse/NIFI-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann reassigned NIFI-8783: -- Assignee: David Handermann > SingleUserAuthorizer fails start up when not used > - > > Key: NIFI-8783 > URL: https://issues.apache.org/jira/browse/NIFI-8783 > Project: Apache NiFi > Issue Type: Bug >Reporter: Bryan Bende >Assignee: David Handermann >Priority: Minor > > The SingleUserAuthorizer has code that checks to see if the > SinglueUserLoginIdentityProvider is configured, and fails start up if it > isn't. This is good to ensure they are only used together. > The issue is, if the SinglueUserAuthorizer is defined in authorizers.xml, but > it is not the configured authorizer in nifi.properties, it still performs > this check and fails start up. The check should be improved to account for > this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-6042) Inferring schema from CSV produces fields in wrong order
[
https://issues.apache.org/jira/browse/NIFI-6042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess resolved NIFI-6042.
Resolution: Fixed
I can't reproduce this with the latest NiFi, there were some changes made from
1.9.0 and on to the inference stuff, closing this as OBE. If you find it's
still an issue, please reopen the case.
> Inferring schema from CSV produces fields in wrong order
>
>
> Key: NIFI-6042
> URL: https://issues.apache.org/jira/browse/NIFI-6042
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.9.0
>Reporter: Bryan Bende
>Priority: Minor
>
> I created a CsvReader with Infer Schema strategy and had a CSV with:
> {code:java}
> first,last
> bryan,bende{code}
> The schema produced had the fields as "last,first" which then was sent to the
> writer and wrote the columns as "last,first".
> I would expect CSV to retain the order of the columns in this case.
> It also seems like when selecting "Infer Schema" you should also be forced to
> select "Treat First Line as Header" as true, since inferring is always going
> to use the first line to get the field names.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669788160
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669787168
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669786483
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
Review comment:
Making the change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669786171
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669785906
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669784757
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669784671
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669783961
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
+logger.debug("Credentials/Configuration provided in
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669783200
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
+awsBootstrapProperties =
getAWSBootstrapProperties(bootstrapProperties);
+if (awsBootstrapProperties != null) {
+loadRequiredAWSProperties(awsBootstrapProperties);
+}
+}
+
+/**
+ * Initializes the KMS Client to be used for encrypt, decrypt and other
interactions with AWS KMS.
+ * First attempts to use credentials/configuration in bootstrap-aws.conf.
+ * If credentials/configuration in bootstrap-aws.conf is not fully
configured,
+ * attempt to initialize credentials using default AWS
credentials/configuration chain.
+ * Note: This does not verify if credentials are valid.
+ */
+private final void initializeClient() {
+if (awsBootstrapProperties == null) {
+logger.warn("Cannot initialize client if awsBootstrapProperties is
null");
+return;
+}
+final String accessKeyId =
awsBootstrapProperties.getProperty(ACCESS_KEY_PROPS_NAME);
+final String secretKeyId =
awsBootstrapProperties.getProperty(SECRET_KEY_PROPS_NAME);
+final String region =
awsBootstrapProperties.getProperty(REGION_KEY_PROPS_NAME);
+
+if (StringUtils.isNotBlank(accessKeyId) &&
StringUtils.isNotBlank(secretKeyId) && StringUtils.isNotBlank(region)) {
Review comment:
Making the change!
##
File
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669781688
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AWSSensitivePropertyProvider.java
##
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.properties;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.properties.BootstrapProperties.BootstrapPropertyKey;
+
+import org.bouncycastle.util.encoders.Base64;
+import org.bouncycastle.util.encoders.DecoderException;
+import org.bouncycastle.util.encoders.EncoderException;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
+import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
+import software.amazon.awssdk.core.SdkBytes;
+import software.amazon.awssdk.core.exception.SdkClientException;
+import software.amazon.awssdk.regions.Region;
+import software.amazon.awssdk.services.kms.KmsClient;
+import software.amazon.awssdk.services.kms.model.DecryptRequest;
+import software.amazon.awssdk.services.kms.model.DecryptResponse;
+import software.amazon.awssdk.services.kms.model.DescribeKeyRequest;
+import software.amazon.awssdk.services.kms.model.DescribeKeyResponse;
+import software.amazon.awssdk.services.kms.model.EncryptRequest;
+import software.amazon.awssdk.services.kms.model.EncryptResponse;
+import software.amazon.awssdk.services.kms.model.KeyMetadata;
+import software.amazon.awssdk.services.kms.model.KmsException;
+
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Paths;
+
+public class AWSSensitivePropertyProvider extends
AbstractSensitivePropertyProvider {
+private static final Logger logger =
LoggerFactory.getLogger(AWSSensitivePropertyProvider.class);
+
+private static final String AWS_PREFIX = "aws";
+private static final String ACCESS_KEY_PROPS_NAME = "aws.access.key.id";
+private static final String SECRET_KEY_PROPS_NAME = "aws.secret.key.id";
+private static final String REGION_KEY_PROPS_NAME = "aws.region";
+private static final String KMS_KEY_PROPS_NAME = "aws.kms.key.id";
+
+private static final Charset PROPERTY_CHARSET = StandardCharsets.UTF_8;
+
+private final BootstrapProperties awsBootstrapProperties;
+private KmsClient client;
+private String keyId;
+
+
+AWSSensitivePropertyProvider(final BootstrapProperties
bootstrapProperties) throws SensitivePropertyProtectionException {
+super(bootstrapProperties);
+// if either awsBootstrapProperties or keyId is loaded as null values,
then isSupported will return false
Review comment:
I will just omit this then, the isSupported function calls another
function that has the description of what to check. Making the changes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a change in pull request #5202: NIFI-6325 Added AWS Sensitive Property Provider
emiliosetiadarma commented on a change in pull request #5202:
URL: https://github.com/apache/nifi/pull/5202#discussion_r669780912
##
File path:
nifi-commons/nifi-sensitive-property-provider/src/main/java/org/apache/nifi/properties/AESSensitivePropertyProvider.java
##
@@ -257,4 +257,12 @@ public static int getMinCipherTextLength() {
public static String getDelimiter() {
return DELIMITER;
}
+
+/**
+ * No cleanup necessary
+ */
+@Override
+public void cleanUp() {
+return;
Review comment:
Making the changes!
##
File path: nifi-assembly/NOTICE
##
@@ -1013,6 +1013,13 @@ The following binary components are provided under the
Apache Software License v
Since product implements StAX API, it has dependencies to StAX API
classes.
+ (ASLv2) AWS SDK for Java 2.0
+ The following NOTICE information applies:
+Copyright 2010-2019 Amazon.com, Inc. or its affiliates. All Rights
Reserved.
Review comment:
Making the changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-8784) The Docker start script variable NIFI_WEB_PROXY_HOST is ignored when using single user authorization
Joey Frazee created NIFI-8784:
-
Summary: The Docker start script variable NIFI_WEB_PROXY_HOST is
ignored when using single user authorization
Key: NIFI-8784
URL: https://issues.apache.org/jira/browse/NIFI-8784
Project: Apache NiFi
Issue Type: Bug
Components: Docker
Affects Versions: 1.14.0
Reporter: Joey Frazee
Assignee: Joey Frazee
In the Docker start scripts nifi.web.proxy.host is only set when sourcing
secure.sh. This only happens when AUTH=tls or AUTH=ldap so the exposed
environment variable NIFI_WEB_PROXY_HOST is ignored for the default single user
authorization and echoes the following:
{{NIFI_WEB_PROXY_HOST was set but NiFi is not configured to run in a secure
mode. Will not update nifi.web.proxy.host.}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8754) CSVReader with InferSchema ignoring 1st non header row, even with necessary properties set
[ https://issues.apache.org/jira/browse/NIFI-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380678#comment-17380678 ] Matt Burgess commented on NIFI-8754: Not sure when/if this behavior changed, but IMO this is the expected behavior. With Infer Schema for CSV, it needs the header line to determine the field names, so no matter what Treat First Line as Header is set to, Infer Schema will treat the first line as the header. When using an External Schema, the Treat First Line as Header is honored since you can supply the field names via the schema. > CSVReader with InferSchema ignoring 1st non header row, even with necessary > properties set > -- > > Key: NIFI-8754 > URL: https://issues.apache.org/jira/browse/NIFI-8754 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.13.2 >Reporter: K Sanghavi >Priority: Major > Labels: bug > Attachments: CSVReaderBug.xml, csvreader-config.jpg, > csvreader-flow.jpg, csvreader-input.jpg, csvreader-output.jpg > > > When using CSVReader controller service in any Record based processors of > 1.13.2, is not working as expected in InferSchema mode, where it ignores the > 1st non header row even with *Treat First Line as Header* property set to > *false*. But with External schema reference the same controller service is > working as expected. (Attached images for SplitRecord with CSVReader) > This setup was tested in 1.11.4, and with same configuration for CSVReader > with InferSchema was giving expected results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1605) Renewing AWS credentials not working when using instance profiles
[ https://issues.apache.org/jira/browse/MINIFICPP-1605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Gyimesi resolved MINIFICPP-1605. -- Resolution: Fixed > Renewing AWS credentials not working when using instance profiles > - > > Key: MINIFICPP-1605 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1605 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gabor Gyimesi >Assignee: Gabor Gyimesi >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > When using AWS instance profile for AWS processors through AWS Credentials > Service the credentials can expire after a few days. According to the AWS > developer support (https://github.com/aws/aws-sdk-cpp/issues/1684) when using > instance profiles we should not cache the credentials as due to a time skew > problem the credentials may expire on AWS side even though it can be > non-expired when checked on client side. > When using default credentials chain we should let the AWS SDK handle the > caching and renewing the credentials and not cache manually on client side. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8767) Conduct 1.14.0 Release
[ https://issues.apache.org/jira/browse/NIFI-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380661#comment-17380661 ] ASF subversion and git services commented on NIFI-8767: --- Commit 1891541 from Joe Witt in branch 'site/trunk' [ https://svn.apache.org/r1891541 ] NIFI-8767 just getting pure apache nifi 1.14.0 links updated > Conduct 1.14.0 Release > -- > > Key: NIFI-8767 > URL: https://issues.apache.org/jira/browse/NIFI-8767 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Trivial > Fix For: 1.14.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] thenatog commented on pull request #5215: NIFI-8782 Added Rate-Limiting for Access Token Requests
thenatog commented on pull request #5215: URL: https://github.com/apache/nifi/pull/5215#issuecomment-879955721 Reviewing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8767) Conduct 1.14.0 Release
[ https://issues.apache.org/jira/browse/NIFI-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380648#comment-17380648 ] ASF subversion and git services commented on NIFI-8767: --- Commit fcbf1d5f975dd984e34f3a543b9480c779b0dc2f in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=fcbf1d5 ] NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 > Conduct 1.14.0 Release > -- > > Key: NIFI-8767 > URL: https://issues.apache.org/jira/browse/NIFI-8767 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Trivial > Fix For: 1.14.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8767) Conduct 1.14.0 Release
[ https://issues.apache.org/jira/browse/NIFI-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380650#comment-17380650 ] ASF subversion and git services commented on NIFI-8767: --- Commit 73563328522b70ef0b024196b2518c9276bf62ae in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7356332 ] Merge branch 'NIFI-8767-RC2' > Conduct 1.14.0 Release > -- > > Key: NIFI-8767 > URL: https://issues.apache.org/jira/browse/NIFI-8767 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Trivial > Fix For: 1.14.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8767) Conduct 1.14.0 Release
[ https://issues.apache.org/jira/browse/NIFI-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380649#comment-17380649 ] ASF subversion and git services commented on NIFI-8767: --- Commit 97feacc18146e95d86070ac6a51c9f89c215e45b in nifi's branch refs/heads/main from Joe Witt [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=97feacc ] NIFI-8767-RC2 prepare for next development iteration > Conduct 1.14.0 Release > -- > > Key: NIFI-8767 > URL: https://issues.apache.org/jira/browse/NIFI-8767 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Trivial > Fix For: 1.14.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8783) SingleUserAuthorizer fails start up when not used
Bryan Bende created NIFI-8783: - Summary: SingleUserAuthorizer fails start up when not used Key: NIFI-8783 URL: https://issues.apache.org/jira/browse/NIFI-8783 Project: Apache NiFi Issue Type: Bug Reporter: Bryan Bende The SingleUserAuthorizer has code that checks to see if the SinglueUserLoginIdentityProvider is configured, and fails start up if it isn't. This is good to ensure they are only used together. The issue is, if the SinglueUserAuthorizer is defined in authorizers.xml, but it is not the configured authorizer in nifi.properties, it still performs this check and fails start up. The check should be improved to account for this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8782) Add Rate-Limiting for Access Token Requests
[ https://issues.apache.org/jira/browse/NIFI-8782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-8782: --- Status: Patch Available (was: Open) > Add Rate-Limiting for Access Token Requests > --- > > Key: NIFI-8782 > URL: https://issues.apache.org/jira/browse/NIFI-8782 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI, Security >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: authentication, jetty, security > Time Spent: 10m > Remaining Estimate: 0h > > The NiFi Jetty Server currently relies on the Jetty [Denial of Service > Filter|https://www.eclipse.org/jetty/documentation/jetty-9/index.html#dos-filter] > to provide configurable rate-limiting for HTTP requests. The DoSFilter > applies to all requests and setting to the limit too low can cause unexpected > problems during system administration or data transfer. > When configured with a Login Identity Provider, Access Token requests support > authenticating users against the specified provider. The number of Access > Token requests from a given remote address should be minimal and predictable > based on the expected number of authorized users. Introducing a separate > configuration property and targeted filter for Access Token requests will > allow the NiFi Jetty Server to reject excessive numbers of authentication > attempts while permitting higher numbers of requests to other resources. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] exceptionfactory opened a new pull request #5215: NIFI-8782 Added Rate-Limiting for Access Token Requests
exceptionfactory opened a new pull request #5215:
URL: https://github.com/apache/nifi/pull/5215
Description of PR
NIFI-8782 Adds a new NiFi configuration property named
`nifi.web.max.access.token.requests.per.second` to support rate-limiting for
HTTP Access Token requests to `/nifi-api/access/token`. Leveraging the Jetty
`DoSFilter`, the configuration property supports rejecting requests with HTTP
429 responses when the number of requests exceeds the configured maximum per
second.
The property uses a default value of 25 requests per second, following the
default value from `DoSFilter`, which should be sufficient for a reasonable
number of interactive authentication attempts. This instance of the `DoSFilter`
also leverages the existing configuration property that supports excluding
selected remote addresses from rate-limiting.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on JDK 8?
- [X] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [X] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #1130: MINIFICPP-1605 Always refresh AWS credentials through default credentials chain
arpadboda closed pull request #1130: URL: https://github.com/apache/nifi-minifi-cpp/pull/1130 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits opened a new pull request #1131: MINIFICPP-1606 Fix the reading of large flow files
fgerlits opened a new pull request #1131: URL: https://github.com/apache/nifi-minifi-cpp/pull/1131 https://issues.apache.org/jira/browse/MINIFICPP-1606 Since `ProcessSession::read()` returns the size of the flow file, and its return type was `int`, we still couldn't handle flow files larger than around 2 GB, even after #1015, #1028 and #1083. I have changed the return type to `int64_t`, which I think fixes the issue. It could have been `size_t` as in #1028 and #1083, but most of the existing code around this (eg. `InputStreamCallback::process()`) use `int64_t`, so this looked like a more logical choice, requiring a smaller code change. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
