[GitHub] [nifi-minifi-cpp] adamdebreceni commented on pull request #974: MINIFICPP-1449 Add pause and resume command to C2
adamdebreceni commented on pull request #974: URL: https://github.com/apache/nifi-minifi-cpp/pull/974#issuecomment-759273848 > > Wonder if tasks need to have the property to indicate paused state. > > So currently I think we have a dedicated threadpool for c2 (heartbeating). Because of this the flowcontroller's threadpool can be completely paused. > > In this case the scheduling agent and the tasks don't need to be impacted, only the threadpool workers top picking up tasks while the flowcontroller is paused. > > What do you think? > > That's actually what I wanted to go with first, but a problem occurred when I implemented it. When the ThreadPool dequeues the worker queue and sees that the dequeued task is in stopped state it does not re-enqueue the task, but simply throws it away. The task remains in the task states map, but changing its state has no effect afterwards. This is why I wanted to distinguish a stopped task from a paused task by moving the paused tasks to separate map when they are processed and later re-enqueueing them when they are resumed thus keeping the original behavior of the stopped tasks. I feel like there are two contending "pause/resume" operation semantics, one regarding the FlowController (@arpadboda ), and one regarding the processors (@lordgamez) 1. which one is the current PR about? (it seem to me, that the first one) 2. if the first one, do we plan on extending it to support the second one? 3. if the first one, and we don't plan on supporting processor-level pause/resume why doesn't a ThreadPool simply stop dequeuing tasks on pause This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] sjyang18 opened a new pull request #4754: NIFI-7417: GetAzureCosmosDBRecord processor
sjyang18 opened a new pull request #4754:
URL: https://github.com/apache/nifi/pull/4754
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Functionally it is equivalent to GetMongoRecord Processor in
nifi-mongodb-bundle, but this processor will use the Azure Cosmos native sql
API.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on JDK 8?
- [X] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [X] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
thenatog commented on pull request #4753: URL: https://github.com/apache/nifi/pull/4753#issuecomment-759047320 Looks like the admin guide was not merged correctly when I rebased initially, I will fix that up. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jfrazee commented on a change in pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
jfrazee commented on a change in pull request #4753:
URL: https://github.com/apache/nifi/pull/4753#discussion_r556097342
##
File path:
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java
##
@@ -1540,6 +1540,12 @@ public boolean isZooKeeperTlsConfigurationPresent() {
&&
getProperty(NiFiProperties.ZOOKEEPER_SECURITY_TRUSTSTORE_PASSWD) != null;
}
+public boolean isTlsConfigurationPresent() {
+return
StringUtils.isNotBlank(getProperty(NiFiProperties.SECURITY_KEYSTORE))
+&& getProperty(NiFiProperties.SECURITY_KEYSTORE_PASSWD) != null
+&&
StringUtils.isNotBlank(getProperty(NiFiProperties.SECURITY_TRUSTSTORE));
Review comment:
We made a stronger assumption in `isZooKeeperTlsConfigurationPresent()`
that truststorePasswd was present. I think what you're doing here, allowing
empty truststorePasswd, is correct but should we change
`isZooKeeperTlsConfigurationPresent()`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
thenatog commented on pull request #4753: URL: https://github.com/apache/nifi/pull/4753#issuecomment-759015855 Some instructions on configuration: **Generated TLS configuration:** ./tls-toolkit.sh standalone -n 'nifi[0-2].com' -C 'CN=three-node-client-cert,OU=NIFI' -O -o security_output --subjectAlternativeNames localhost,127.0.0.1 **Relevant nifi.properties:** Node 1 example (basically the same on all nodes unless keystore passwords are different): nifi.security.keystore=./conf/keystore.jks nifi.security.keystoreType=jks nifi.security.keystorePasswd=password nifi.security.keyPasswd=password nifi.security.truststore=./conf/truststore.jks nifi.security.truststoreType=jks nifi.security.truststorePasswd=password nifi.security.user.authorizer=managed-authorizer nifi.zookeeper.connect.string=nifi0.com:2281,nifi1.com:2282,nifi2.com:2283 nifi.zookeeper.connect.timeout=10 secs nifi.zookeeper.session.timeout=10 secs nifi.zookeeper.root.node=/nifi nifi.zookeeper.client.secure=true nifi.state.management.configuration.file=./conf/state-management.xml nifi.state.management.provider.local=local-provider nifi.state.management.provider.cluster=zk-provider nifi.state.management.embedded.zookeeper.start=true nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties **Zookeeper state directory with myid file for each node in:** node0/state/zookeeper/myid (contents of "1") node1/state/zookeeper/myid (contents of "2") node2/state/zookeeper/myid (contents of "3") **zookeeper.properties (Almost identical other than secureClientPort. If running on different machines then the port can be the same and the file will be identical across nodes):** Node 1: secureClientPort=2281 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 server.1=nifi0.com:2880:3880 server.2=nifi1.com:2881:3881 server.3=nifi2.com:2882:3882 Node 2: secureClientPort=2282 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 server.1=nifi0.com:2880:3880 server.2=nifi1.com:2881:3881 server.3=nifi2.com:2882:3882 Node 3: secureClientPort=2283 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 server.1=nifi0.com:2880:3880 server.2=nifi1.com:2881:3881 server.3=nifi2.com:2882:3882 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jfrazee commented on a change in pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
jfrazee commented on a change in pull request #4753:
URL: https://github.com/apache/nifi/pull/4753#discussion_r556103586
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/state/server/ZooKeeperStateServer.java
##
@@ -136,13 +170,14 @@ private void startDistributed() throws IOException {
quorumPeer.setInitLimit(quorumPeerConfig.getInitLimit());
quorumPeer.setSyncLimit(quorumPeerConfig.getSyncLimit());
quorumPeer.setQuorumVerifier(quorumPeerConfig.getQuorumVerifier(),
false);
-quorumPeer.setCnxnFactory(connectionFactory);
quorumPeer.setZKDatabase(new
ZKDatabase(quorumPeer.getTxnFactory()));
quorumPeer.setLearnerType(quorumPeerConfig.getPeerType());
quorumPeer.setSyncEnabled(quorumPeerConfig.getSyncEnabled());
quorumPeer.setQuorumListenOnAllIPs(quorumPeerConfig.getQuorumListenOnAllIPs());
+quorumPeer.setSslQuorum(quorumPeerConfig.isSslQuorum());
quorumPeer.start();
+
Review comment:
```suggestion
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7991) Flow Configuration History displays "annotation data not found/available" from "Advanced" changes.
[ https://issues.apache.org/jira/browse/NIFI-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-7991: --- Fix Version/s: 1.13.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Flow Configuration History displays "annotation data not found/available" > from "Advanced" changes. > -- > > Key: NIFI-7991 > URL: https://issues.apache.org/jira/browse/NIFI-7991 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Tim Smith >Assignee: Tim Smith >Priority: Minor > Fix For: 1.13.0 > > Time Spent: 50m > Remaining Estimate: 0h > > On making changes through UpdateAttributes Advanced tab, discovered that > "AnnotationData" is presented in "Flow Configuration History" as "annotation > data not found/available" in both previous and new values by > ProcessorAuditor. UpdateAttribute represents the annotation data as XML. > ProcessorAuditor, at a minimum, should attempt to represent new and previous > values as the difference between the XML nodes or as String values, if not in > XML format. Default value presented is not useful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7991) Flow Configuration History displays "annotation data not found/available" from "Advanced" changes.
[ https://issues.apache.org/jira/browse/NIFI-7991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263697#comment-17263697 ] ASF subversion and git services commented on NIFI-7991: --- Commit 0b966cdb141a89d5cef49e93653ace83217752bf in nifi's branch refs/heads/main from tlsmith [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0b966cd ] NIFI-7991 Flow Configuration History displays "annotation data not found/available" from "Advanced" changes Signed-off-by: Matthew Burgess This closes #4668 > Flow Configuration History displays "annotation data not found/available" > from "Advanced" changes. > -- > > Key: NIFI-7991 > URL: https://issues.apache.org/jira/browse/NIFI-7991 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Tim Smith >Assignee: Tim Smith >Priority: Minor > Time Spent: 40m > Remaining Estimate: 0h > > On making changes through UpdateAttributes Advanced tab, discovered that > "AnnotationData" is presented in "Flow Configuration History" as "annotation > data not found/available" in both previous and new values by > ProcessorAuditor. UpdateAttribute represents the annotation data as XML. > ProcessorAuditor, at a minimum, should attempt to represent new and previous > values as the difference between the XML nodes or as String values, if not in > XML format. Default value presented is not useful. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 closed pull request #4668: NIFI-7991 Flow Configuration History displays "annotation data not fo…
mattyb149 closed pull request #4668: URL: https://github.com/apache/nifi/pull/4668 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] jfrazee commented on a change in pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
jfrazee commented on a change in pull request #4753:
URL: https://github.com/apache/nifi/pull/4753#discussion_r556097342
##
File path:
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java
##
@@ -1540,6 +1540,12 @@ public boolean isZooKeeperTlsConfigurationPresent() {
&&
getProperty(NiFiProperties.ZOOKEEPER_SECURITY_TRUSTSTORE_PASSWD) != null;
}
+public boolean isTlsConfigurationPresent() {
+return
StringUtils.isNotBlank(getProperty(NiFiProperties.SECURITY_KEYSTORE))
+&& getProperty(NiFiProperties.SECURITY_KEYSTORE_PASSWD) != null
+&&
StringUtils.isNotBlank(getProperty(NiFiProperties.SECURITY_TRUSTSTORE));
Review comment:
We made a stronger assumption in `isZooKeeperTlsConfigurationPresent()`
that truststorePasswd was present. I think what you're doing here, allowing
empty truststorePasswd, is correct but should we change
`isZooKeeperTlsConfigurationPresent()`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-8138) Add property sensitivity in UI property tooltip
[ https://issues.apache.org/jira/browse/NIFI-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-8138: - Fix Version/s: 1.13.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add property sensitivity in UI property tooltip > --- > > Key: NIFI-8138 > URL: https://issues.apache.org/jira/browse/NIFI-8138 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Assignee: Matt Burgess >Priority: Trivial > Labels: easyfix, ui > Fix For: 1.13.0 > > Attachments: Screenshot 2021-01-12 at 23.06.10.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > When hovering the question mark next to a property name, it's currently > possible to see the expression language scope associated to this property but > not if this property is sensitive or not. This is only displayed in the usage > documentation of the processor. It would be useful for users to also have > this information available in the tooltip. > !Screenshot 2021-01-12 at 23.06.10.png|width=417,height=208! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] jfrazee commented on pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
jfrazee commented on pull request #4753: URL: https://github.com/apache/nifi/pull/4753#issuecomment-758982220 Thanks @thenatog for bringing this back. I'll start looking at it right now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog commented on pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
thenatog commented on pull request #4753: URL: https://github.com/apache/nifi/pull/4753#issuecomment-758978443 For this PR I have tested: NiFi secured 3 node cluster with a single secured Zookeeper instance. NiFi secured 3 node cluster with a secured 3 node Zookeeper quorum. NiFi insecure 3 node cluster with insecure 3 node Zookeeper quorum. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] thenatog opened a new pull request #4753: NIFI-7356 - Enable TLS for embedded Zookeeper when NiFi has TLS enabled
thenatog opened a new pull request #4753:
URL: https://github.com/apache/nifi/pull/4753
NIFI-7356 - Addresses PR feedback.
NIFI-7356 - Additional changes from PR feedback.
NIFI-7356 - Adding integration tests for ZooKeeperStateServer for TLS.
NIFI-7356 - TLS + Zookeeper now working with single and quorum. Needs code
cleanup, need to fix IT tests and docs.
NIFI-7356 - Fixed up tests and removed some irrelevant ones. Refactored some
of ZooKeeperStateServer. Tested successfully with a secure and insecure 3 node
NiFi + Quorum.
NIFI-7356 - Checkstyle fixes.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [x] Have you written or updated unit tests to verify your changes?
- [x] Have you verified that the full build is successful on JDK 8?
- [x] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-7356) Enable TLS for embedded Zookeeper when NiFi has TLS enabled
[
https://issues.apache.org/jira/browse/NIFI-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough reassigned NIFI-7356:
--
Assignee: Nathan Gough (was: Troy Melhase)
> Enable TLS for embedded Zookeeper when NiFi has TLS enabled
> ---
>
> Key: NIFI-7356
> URL: https://issues.apache.org/jira/browse/NIFI-7356
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Configuration, Configuration Management, Security
>Reporter: Troy Melhase
>Assignee: Nathan Gough
>Priority: Major
> Fix For: 1.13.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> If embedded ZK has TLS properties in the {{zookeeper.properties}} file, these
> will be used. If however, this file does not populate those properties, and
> NiFi does have TLS properties configured ({{nifi.security.keyStore}}, etc.),
> these values will be used to override the ZK plaintext connection listener to
> create a TLS connection listener.
> If the {{zookeeper.properties}} file has an incomplete configuration (i.e.
> keystore password set but no keystore path), startup should fail with a clear
> error message indicating the missing properties and how to resolve.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] jfrazee commented on a change in pull request #4367: NIFI-7573: Add an Azure Active Directory (AAD) User Group Provider
jfrazee commented on a change in pull request #4367:
URL: https://github.com/apache/nifi/pull/4367#discussion_r556083164
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-graph-authorizer/src/main/java/org/apache/nifi/authorization/azure/AzureGraphUserGroupProvider.java
##
@@ -0,0 +1,412 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.authorization.azure;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import com.google.gson.JsonObject;
+import com.microsoft.graph.core.ClientException;
+import com.microsoft.graph.models.extensions.DirectoryObject;
+import com.microsoft.graph.models.extensions.IGraphServiceClient;
+import com.microsoft.graph.options.Option;
+import com.microsoft.graph.options.QueryOption;
+import com.microsoft.graph.requests.extensions.GraphServiceClient;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesPage;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesRequest;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesRequestBuilder;
+import com.microsoft.graph.requests.extensions.IGroupCollectionPage;
+import com.microsoft.graph.requests.extensions.IGroupCollectionRequest;
+import com.microsoft.graph.requests.extensions.IGroupCollectionRequestBuilder;
+
+import org.apache.nifi.authorization.AuthorizerConfigurationContext;
+import org.apache.nifi.authorization.Group;
+import org.apache.nifi.authorization.User;
+import org.apache.nifi.authorization.UserAndGroups;
+import org.apache.nifi.authorization.UserGroupProvider;
+import org.apache.nifi.authorization.UserGroupProviderInitializationContext;
+import org.apache.nifi.authorization.exception.AuthorizationAccessException;
+import org.apache.nifi.authorization.exception.AuthorizerCreationException;
+import org.apache.nifi.authorization.exception.AuthorizerDestructionException;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.util.FormatUtils;
+import org.apache.nifi.util.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The AzureGraphUserGroupProvider provides support for retrieving users and
+ * groups from Azure Activy Driectory (AAD) using graph rest-api & SDK.
+ */
+public class AzureGraphUserGroupProvider implements UserGroupProvider {
+private final static Logger logger =
LoggerFactory.getLogger(AzureGraphUserGroupProvider.class);
+private Set groupFilterSet;
+private int pageSize;
+private String claimForUserName;
+
+private final ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(2);
+
+public static final String REFRESH_DELAY_PROPERTY = "REFRESH_DELAY";
+private static final long MINIMUM_SYNC_INTERVAL_MILLISECONDS = 10_000;
+public static final String AUTHORITY_ENDPOINT_PROPERTY =
"AUTHORITY_ENDPOINT";
+public static final String TENANT_ID_PROPERTY = "TENANT_ID";
+public static final String APP_REG_CLIENT_ID_PROPERTY =
"APP_REG_CLIENT_ID";
+public static final String APP_REG_CLIENT_SECRET_PROPERTY =
"APP_REG_CLIENT_SECRET";
+// comma separate list of group names to search from AAD
+public static final String GROUP_FILTER_LIST_PROPERTY =
"GROUP_FILTER_LIST_INCLUSION";
+// group filter with startswith
+public static final String GROUP_FILTER_PREFIX_PROPERTY =
"GROUP_FILTER_PREFIX";
+// client side group filter 'endswith' operator, due to support limiation
of
+// azure graph rest-api
+public static final String GROUP_FILTER_SUFFIX_PROPERTY =
"GROUP_FILTER_SUFFIX";
+// client side group filter 'contains' operator, due to support limiation
of
+// azure graph rest-api
+public static final String GROUP_FILTER_SUBSTRING_PROPERTY =
"GROUP_FILTER_SUBSTRING";
+public static final String PAGE_SIZE_PROPERTY =
[GitHub] [nifi] jfrazee commented on a change in pull request #4367: NIFI-7573: Add an Azure Active Directory (AAD) User Group Provider
jfrazee commented on a change in pull request #4367:
URL: https://github.com/apache/nifi/pull/4367#discussion_r556083164
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-graph-authorizer/src/main/java/org/apache/nifi/authorization/azure/AzureGraphUserGroupProvider.java
##
@@ -0,0 +1,412 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.authorization.azure;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import com.google.gson.JsonObject;
+import com.microsoft.graph.core.ClientException;
+import com.microsoft.graph.models.extensions.DirectoryObject;
+import com.microsoft.graph.models.extensions.IGraphServiceClient;
+import com.microsoft.graph.options.Option;
+import com.microsoft.graph.options.QueryOption;
+import com.microsoft.graph.requests.extensions.GraphServiceClient;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesPage;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesRequest;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesRequestBuilder;
+import com.microsoft.graph.requests.extensions.IGroupCollectionPage;
+import com.microsoft.graph.requests.extensions.IGroupCollectionRequest;
+import com.microsoft.graph.requests.extensions.IGroupCollectionRequestBuilder;
+
+import org.apache.nifi.authorization.AuthorizerConfigurationContext;
+import org.apache.nifi.authorization.Group;
+import org.apache.nifi.authorization.User;
+import org.apache.nifi.authorization.UserAndGroups;
+import org.apache.nifi.authorization.UserGroupProvider;
+import org.apache.nifi.authorization.UserGroupProviderInitializationContext;
+import org.apache.nifi.authorization.exception.AuthorizationAccessException;
+import org.apache.nifi.authorization.exception.AuthorizerCreationException;
+import org.apache.nifi.authorization.exception.AuthorizerDestructionException;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.util.FormatUtils;
+import org.apache.nifi.util.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The AzureGraphUserGroupProvider provides support for retrieving users and
+ * groups from Azure Activy Driectory (AAD) using graph rest-api & SDK.
+ */
+public class AzureGraphUserGroupProvider implements UserGroupProvider {
+private final static Logger logger =
LoggerFactory.getLogger(AzureGraphUserGroupProvider.class);
+private Set groupFilterSet;
+private int pageSize;
+private String claimForUserName;
+
+private final ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(2);
+
+public static final String REFRESH_DELAY_PROPERTY = "REFRESH_DELAY";
+private static final long MINIMUM_SYNC_INTERVAL_MILLISECONDS = 10_000;
+public static final String AUTHORITY_ENDPOINT_PROPERTY =
"AUTHORITY_ENDPOINT";
+public static final String TENANT_ID_PROPERTY = "TENANT_ID";
+public static final String APP_REG_CLIENT_ID_PROPERTY =
"APP_REG_CLIENT_ID";
+public static final String APP_REG_CLIENT_SECRET_PROPERTY =
"APP_REG_CLIENT_SECRET";
+// comma separate list of group names to search from AAD
+public static final String GROUP_FILTER_LIST_PROPERTY =
"GROUP_FILTER_LIST_INCLUSION";
+// group filter with startswith
+public static final String GROUP_FILTER_PREFIX_PROPERTY =
"GROUP_FILTER_PREFIX";
+// client side group filter 'endswith' operator, due to support limiation
of
+// azure graph rest-api
+public static final String GROUP_FILTER_SUFFIX_PROPERTY =
"GROUP_FILTER_SUFFIX";
+// client side group filter 'contains' operator, due to support limiation
of
+// azure graph rest-api
+public static final String GROUP_FILTER_SUBSTRING_PROPERTY =
"GROUP_FILTER_SUBSTRING";
+public static final String PAGE_SIZE_PROPERTY =
[GitHub] [nifi] jfrazee commented on a change in pull request #4367: NIFI-7573: Add an Azure Active Directory (AAD) User Group Provider
jfrazee commented on a change in pull request #4367:
URL: https://github.com/apache/nifi/pull/4367#discussion_r556081137
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-graph-authorizer/src/main/java/org/apache/nifi/authorization/azure/AzureGraphUserGroupProvider.java
##
@@ -0,0 +1,412 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.authorization.azure;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import com.google.gson.JsonObject;
+import com.microsoft.graph.core.ClientException;
+import com.microsoft.graph.models.extensions.DirectoryObject;
+import com.microsoft.graph.models.extensions.IGraphServiceClient;
+import com.microsoft.graph.options.Option;
+import com.microsoft.graph.options.QueryOption;
+import com.microsoft.graph.requests.extensions.GraphServiceClient;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesPage;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesRequest;
+import
com.microsoft.graph.requests.extensions.IDirectoryObjectCollectionWithReferencesRequestBuilder;
+import com.microsoft.graph.requests.extensions.IGroupCollectionPage;
+import com.microsoft.graph.requests.extensions.IGroupCollectionRequest;
+import com.microsoft.graph.requests.extensions.IGroupCollectionRequestBuilder;
+
+import org.apache.nifi.authorization.AuthorizerConfigurationContext;
+import org.apache.nifi.authorization.Group;
+import org.apache.nifi.authorization.User;
+import org.apache.nifi.authorization.UserAndGroups;
+import org.apache.nifi.authorization.UserGroupProvider;
+import org.apache.nifi.authorization.UserGroupProviderInitializationContext;
+import org.apache.nifi.authorization.exception.AuthorizationAccessException;
+import org.apache.nifi.authorization.exception.AuthorizerCreationException;
+import org.apache.nifi.authorization.exception.AuthorizerDestructionException;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.util.FormatUtils;
+import org.apache.nifi.util.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * The AzureGraphUserGroupProvider provides support for retrieving users and
+ * groups from Azure Activy Driectory (AAD) using graph rest-api & SDK.
+ */
+public class AzureGraphUserGroupProvider implements UserGroupProvider {
+private final static Logger logger =
LoggerFactory.getLogger(AzureGraphUserGroupProvider.class);
+private Set groupFilterSet;
+private int pageSize;
+private String claimForUserName;
+
+private final ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(2);
+
+public static final String REFRESH_DELAY_PROPERTY = "REFRESH_DELAY";
+private static final long MINIMUM_SYNC_INTERVAL_MILLISECONDS = 10_000;
+public static final String AUTHORITY_ENDPOINT_PROPERTY =
"AUTHORITY_ENDPOINT";
+public static final String TENANT_ID_PROPERTY = "TENANT_ID";
+public static final String APP_REG_CLIENT_ID_PROPERTY =
"APP_REG_CLIENT_ID";
+public static final String APP_REG_CLIENT_SECRET_PROPERTY =
"APP_REG_CLIENT_SECRET";
+// comma separate list of group names to search from AAD
+public static final String GROUP_FILTER_LIST_PROPERTY =
"GROUP_FILTER_LIST_INCLUSION";
+// group filter with startswith
+public static final String GROUP_FILTER_PREFIX_PROPERTY =
"GROUP_FILTER_PREFIX";
+// client side group filter 'endswith' operator, due to support limiation
of
+// azure graph rest-api
+public static final String GROUP_FILTER_SUFFIX_PROPERTY =
"GROUP_FILTER_SUFFIX";
+// client side group filter 'contains' operator, due to support limiation
of
+// azure graph rest-api
+public static final String GROUP_FILTER_SUBSTRING_PROPERTY =
"GROUP_FILTER_SUBSTRING";
+public static final String PAGE_SIZE_PROPERTY =
[jira] [Commented] (NIFI-8138) Add property sensitivity in UI property tooltip
[ https://issues.apache.org/jira/browse/NIFI-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263683#comment-17263683 ] ASF subversion and git services commented on NIFI-8138: --- Commit 2d6917918439153e8e20629efb3925dba4efdcf7 in nifi's branch refs/heads/main from Matt Burgess [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2d69179 ] NIFI-8138: Add Sensitive Property value to property tooltip Signed-off-by: Pierre Villard This closes #4752. > Add property sensitivity in UI property tooltip > --- > > Key: NIFI-8138 > URL: https://issues.apache.org/jira/browse/NIFI-8138 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Assignee: Matt Burgess >Priority: Trivial > Labels: easyfix, ui > Attachments: Screenshot 2021-01-12 at 23.06.10.png > > Time Spent: 20m > Remaining Estimate: 0h > > When hovering the question mark next to a property name, it's currently > possible to see the expression language scope associated to this property but > not if this property is sensitive or not. This is only displayed in the usage > documentation of the processor. It would be useful for users to also have > this information available in the tooltip. > !Screenshot 2021-01-12 at 23.06.10.png|width=417,height=208! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4752: NIFI-8138: Add Sensitive Property value to property tooltip
asfgit closed pull request #4752: URL: https://github.com/apache/nifi/pull/4752 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4752: NIFI-8138: Add Sensitive Property value to property tooltip
pvillard31 commented on pull request #4752: URL: https://github.com/apache/nifi/pull/4752#issuecomment-758920482 Reviewing... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8138) Add property sensitivity in UI property tooltip
[ https://issues.apache.org/jira/browse/NIFI-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-8138: --- Status: Patch Available (was: In Progress) > Add property sensitivity in UI property tooltip > --- > > Key: NIFI-8138 > URL: https://issues.apache.org/jira/browse/NIFI-8138 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Assignee: Matt Burgess >Priority: Trivial > Labels: easyfix, ui > Attachments: Screenshot 2021-01-12 at 23.06.10.png > > Time Spent: 10m > Remaining Estimate: 0h > > When hovering the question mark next to a property name, it's currently > possible to see the expression language scope associated to this property but > not if this property is sensitive or not. This is only displayed in the usage > documentation of the processor. It would be useful for users to also have > this information available in the tooltip. > !Screenshot 2021-01-12 at 23.06.10.png|width=417,height=208! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 opened a new pull request #4752: NIFI-8138: Add Sensitive Property value to property tooltip
mattyb149 opened a new pull request #4752:
URL: https://github.com/apache/nifi/pull/4752
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Adds a "Sensitive property: " line to property tooltips
(previously this information is only visible on the Usage page).
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [x] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-8138) Add property sensitivity in UI property tooltip
[ https://issues.apache.org/jira/browse/NIFI-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-8138: -- Assignee: Matt Burgess > Add property sensitivity in UI property tooltip > --- > > Key: NIFI-8138 > URL: https://issues.apache.org/jira/browse/NIFI-8138 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Assignee: Matt Burgess >Priority: Trivial > Labels: easyfix, ui > Attachments: Screenshot 2021-01-12 at 23.06.10.png > > > When hovering the question mark next to a property name, it's currently > possible to see the expression language scope associated to this property but > not if this property is sensitive or not. This is only displayed in the usage > documentation of the processor. It would be useful for users to also have > this information available in the tooltip. > !Screenshot 2021-01-12 at 23.06.10.png|width=417,height=208! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on a change in pull request #4750: NIFI-7989: Add Update Field Names and Record Writer to UpdateHiveTable processors
mattyb149 commented on a change in pull request #4750:
URL: https://github.com/apache/nifi/pull/4750#discussion_r556021114
##
File path:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive_1_1-processors/src/main/java/org/apache/nifi/processors/hive/UpdateHive_1_1Table.java
##
@@ -562,7 +686,35 @@ private synchronized void checkAndUpdateTableSchema(final
ProcessSession session
outputPath = tableLocation + "/" + String.join("/",
partitionColumnsLocationList);
}
-session.putAttribute(flowFile, ATTR_OUTPUT_PATH, outputPath);
+// If updating field names, return a new RecordSchema, otherwise
return null
+OutputMetadataHolder outputMetadataHolder;
+if (!tableCreated && updateFieldNames) {
+List inputRecordFields = schema.getFields();
+List outputRecordFields = new ArrayList<>();
+Map fieldMap = new HashMap<>();
+
+for (RecordField inputRecordField : inputRecordFields) {
+final String inputRecordFieldName =
inputRecordField.getFieldName();
+boolean found = false;
+for (String hiveColumnName : hiveColumns) {
+if
(inputRecordFieldName.equalsIgnoreCase(hiveColumnName)) {
Review comment:
Agreed, I will change the behavior and add the appropriate documentation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-8138) Add property sensitivity in UI property tooltip
Pierre Villard created NIFI-8138: Summary: Add property sensitivity in UI property tooltip Key: NIFI-8138 URL: https://issues.apache.org/jira/browse/NIFI-8138 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Pierre Villard Attachments: Screenshot 2021-01-12 at 23.06.10.png When hovering the question mark next to a property name, it's currently possible to see the expression language scope associated to this property but not if this property is sensitive or not. This is only displayed in the usage documentation of the processor. It would be useful for users to also have this information available in the tooltip. !Screenshot 2021-01-12 at 23.06.10.png|width=417,height=208! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] turcsanyip commented on a change in pull request #4750: NIFI-7989: Add Update Field Names and Record Writer to UpdateHiveTable processors
turcsanyip commented on a change in pull request #4750:
URL: https://github.com/apache/nifi/pull/4750#discussion_r555945721
##
File path:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive_1_1-processors/src/main/java/org/apache/nifi/processors/hive/UpdateHive_1_1Table.java
##
@@ -562,7 +686,35 @@ private synchronized void checkAndUpdateTableSchema(final
ProcessSession session
outputPath = tableLocation + "/" + String.join("/",
partitionColumnsLocationList);
}
-session.putAttribute(flowFile, ATTR_OUTPUT_PATH, outputPath);
+// If updating field names, return a new RecordSchema, otherwise
return null
+OutputMetadataHolder outputMetadataHolder;
+if (!tableCreated && updateFieldNames) {
+List inputRecordFields = schema.getFields();
+List outputRecordFields = new ArrayList<>();
+Map fieldMap = new HashMap<>();
+
+for (RecordField inputRecordField : inputRecordFields) {
+final String inputRecordFieldName =
inputRecordField.getFieldName();
+boolean found = false;
+for (String hiveColumnName : hiveColumns) {
+if
(inputRecordFieldName.equalsIgnoreCase(hiveColumnName)) {
Review comment:
If 'Update Fields Names' has been set to true, it always remap the
records. Even if all the columns names are identical and the conversion would
not be needed. It could be optimised to convert the records only if necessary.
In that case the record reader and writer must have the same data format
because some FFs will be converted while others will not. Actually, it is
already a requirement for a create + update scenario (create never converts)
and should be documented for the users.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-8094) Support BCFKS Keystore Type
[
https://issues.apache.org/jira/browse/NIFI-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-8094:
-
Fix Version/s: 1.13.0
> Support BCFKS Keystore Type
> ---
>
> Key: NIFI-8094
> URL: https://issues.apache.org/jira/browse/NIFI-8094
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions, Security
>Affects Versions: 1.12.1
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: FIPS, security
> Fix For: 1.13.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The [Bouncy Castle FIPS Key
> Store|https://cryptosense.com/blog/bouncycastle-keystore-security/] (BCFKS)
> format supports storage of certificates and private keys using AES-CCM and
> PBKDF2 algorithms, providing greater security than the standard JKS and
> PKCS12 implementations. Support for BCFKS can be implemented using Bouncy
> Castle security provider libraries that are already leveraged throughout the
> system.
> Initial support should include the ability to specify BCFKS as the key store
> and trust store type in standard properties files as well as the ability to
> select BCFKS in implementations of the SSLContextService.
> Extension components that do not use {{SSLContextService.createSSLContext()}}
> may need additional work, which should be addressed in related issues
> following this implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8094) Support BCFKS Keystore Type
[
https://issues.apache.org/jira/browse/NIFI-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-8094:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Support BCFKS Keystore Type
> ---
>
> Key: NIFI-8094
> URL: https://issues.apache.org/jira/browse/NIFI-8094
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions, Security
>Affects Versions: 1.12.1
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: FIPS, security
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The [Bouncy Castle FIPS Key
> Store|https://cryptosense.com/blog/bouncycastle-keystore-security/] (BCFKS)
> format supports storage of certificates and private keys using AES-CCM and
> PBKDF2 algorithms, providing greater security than the standard JKS and
> PKCS12 implementations. Support for BCFKS can be implemented using Bouncy
> Castle security provider libraries that are already leveraged throughout the
> system.
> Initial support should include the ability to specify BCFKS as the key store
> and trust store type in standard properties files as well as the ability to
> select BCFKS in implementations of the SSLContextService.
> Extension components that do not use {{SSLContextService.createSSLContext()}}
> may need additional work, which should be addressed in related issues
> following this implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8094) Support BCFKS Keystore Type
[
https://issues.apache.org/jira/browse/NIFI-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263452#comment-17263452
]
ASF subversion and git services commented on NIFI-8094:
---
Commit 7d76bcd5202a8680c952d3a19072087a971d0b69 in nifi's branch
refs/heads/main from exceptionfactory
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7d76bcd ]
NIFI-8094 Added support for BCFKS Keystore Type
NIFI-8094 Updated Administration Guide to include BCFKS
Signed-off-by: Nathan Gough
This closes #4729.
> Support BCFKS Keystore Type
> ---
>
> Key: NIFI-8094
> URL: https://issues.apache.org/jira/browse/NIFI-8094
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions, Security
>Affects Versions: 1.12.1
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: FIPS, security
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The [Bouncy Castle FIPS Key
> Store|https://cryptosense.com/blog/bouncycastle-keystore-security/] (BCFKS)
> format supports storage of certificates and private keys using AES-CCM and
> PBKDF2 algorithms, providing greater security than the standard JKS and
> PKCS12 implementations. Support for BCFKS can be implemented using Bouncy
> Castle security provider libraries that are already leveraged throughout the
> system.
> Initial support should include the ability to specify BCFKS as the key store
> and trust store type in standard properties files as well as the ability to
> select BCFKS in implementations of the SSLContextService.
> Extension components that do not use {{SSLContextService.createSSLContext()}}
> may need additional work, which should be addressed in related issues
> following this implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8094) Support BCFKS Keystore Type
[
https://issues.apache.org/jira/browse/NIFI-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263453#comment-17263453
]
ASF subversion and git services commented on NIFI-8094:
---
Commit 7d76bcd5202a8680c952d3a19072087a971d0b69 in nifi's branch
refs/heads/main from exceptionfactory
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7d76bcd ]
NIFI-8094 Added support for BCFKS Keystore Type
NIFI-8094 Updated Administration Guide to include BCFKS
Signed-off-by: Nathan Gough
This closes #4729.
> Support BCFKS Keystore Type
> ---
>
> Key: NIFI-8094
> URL: https://issues.apache.org/jira/browse/NIFI-8094
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework, Extensions, Security
>Affects Versions: 1.12.1
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Minor
> Labels: FIPS, security
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The [Bouncy Castle FIPS Key
> Store|https://cryptosense.com/blog/bouncycastle-keystore-security/] (BCFKS)
> format supports storage of certificates and private keys using AES-CCM and
> PBKDF2 algorithms, providing greater security than the standard JKS and
> PKCS12 implementations. Support for BCFKS can be implemented using Bouncy
> Castle security provider libraries that are already leveraged throughout the
> system.
> Initial support should include the ability to specify BCFKS as the key store
> and trust store type in standard properties files as well as the ability to
> select BCFKS in implementations of the SSLContextService.
> Extension components that do not use {{SSLContextService.createSSLContext()}}
> may need additional work, which should be addressed in related issues
> following this implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] thenatog closed pull request #4729: NIFI-8094 Added support for BCFKS Keystore Type
thenatog closed pull request #4729: URL: https://github.com/apache/nifi/pull/4729 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on pull request #974: MINIFICPP-1449 Add pause and resume command to C2
lordgamez commented on pull request #974: URL: https://github.com/apache/nifi-minifi-cpp/pull/974#issuecomment-758743776 > Wonder if tasks need to have the property to indicate paused state. > > So currently I think we have a dedicated threadpool for c2 (heartbeating). Because of this the flowcontroller's threadpool can be completely paused. > > In this case the scheduling agent and the tasks don't need to be impacted, only the threadpool workers top picking up tasks while the flowcontroller is paused. > > What do you think? That's actually what I wanted to go with first, but a problem occurred when I implemented it. When the ThreadPool dequeues the worker queue and sees that the dequeued task is in stopped state it does not re-enqueue the task, but simply throws it away. The task remains in the task states map, but changing its state has no effect afterwards. This is why I wanted to distinguish a stopped task from a paused task by moving the paused tasks to separate map when they are processed and later re-enqueueing them when they are resumed thus keeping the original behavior of the stopped tasks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-7906) Add graph processor with flexibility to query graph database conditioned on flowfile content and attributes
[ https://issues.apache.org/jira/browse/NIFI-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess resolved NIFI-7906. Resolution: Fixed > Add graph processor with flexibility to query graph database conditioned on > flowfile content and attributes > --- > > Key: NIFI-7906 > URL: https://issues.apache.org/jira/browse/NIFI-7906 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Levi Lentz >Assignee: Levi Lentz >Priority: Minor > Labels: graph > Fix For: 1.13.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > The current graph bundle currently does not allow you to query the graph > database (as defined in the GraphClientService) with attributes or content > available in the flow file. > > This functionality would allow uses to perform dynamic queries/mutations of > the underlying graph data based. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7906) Add graph processor with flexibility to query graph database conditioned on flowfile content and attributes
[ https://issues.apache.org/jira/browse/NIFI-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263429#comment-17263429 ] ASF subversion and git services commented on NIFI-7906: --- Commit 82c84492ced10b135242c86afd2e4ff1d0083ff9 in nifi's branch refs/heads/main from Mike Thomsen [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=82c8449 ] NIFI-7906 Fixed a bug in how record paths were handled. NIFI-7906 Fixed bugs in unit tests. NIFI-7906 Updated a few things based on a code review. NIFI-7906: Fixed typo in record.count attribute Signed-off-by: Matthew Burgess This closes #4739 > Add graph processor with flexibility to query graph database conditioned on > flowfile content and attributes > --- > > Key: NIFI-7906 > URL: https://issues.apache.org/jira/browse/NIFI-7906 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Levi Lentz >Assignee: Levi Lentz >Priority: Minor > Labels: graph > Fix For: 1.13.0 > > Time Spent: 7h 40m > Remaining Estimate: 0h > > The current graph bundle currently does not allow you to query the graph > database (as defined in the GraphClientService) with attributes or content > available in the flow file. > > This functionality would allow uses to perform dynamic queries/mutations of > the underlying graph data based. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7906) Add graph processor with flexibility to query graph database conditioned on flowfile content and attributes
[ https://issues.apache.org/jira/browse/NIFI-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263431#comment-17263431 ] ASF subversion and git services commented on NIFI-7906: --- Commit 82c84492ced10b135242c86afd2e4ff1d0083ff9 in nifi's branch refs/heads/main from Mike Thomsen [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=82c8449 ] NIFI-7906 Fixed a bug in how record paths were handled. NIFI-7906 Fixed bugs in unit tests. NIFI-7906 Updated a few things based on a code review. NIFI-7906: Fixed typo in record.count attribute Signed-off-by: Matthew Burgess This closes #4739 > Add graph processor with flexibility to query graph database conditioned on > flowfile content and attributes > --- > > Key: NIFI-7906 > URL: https://issues.apache.org/jira/browse/NIFI-7906 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Levi Lentz >Assignee: Levi Lentz >Priority: Minor > Labels: graph > Fix For: 1.13.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > The current graph bundle currently does not allow you to query the graph > database (as defined in the GraphClientService) with attributes or content > available in the flow file. > > This functionality would allow uses to perform dynamic queries/mutations of > the underlying graph data based. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7906) Add graph processor with flexibility to query graph database conditioned on flowfile content and attributes
[ https://issues.apache.org/jira/browse/NIFI-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263428#comment-17263428 ] ASF subversion and git services commented on NIFI-7906: --- Commit 82c84492ced10b135242c86afd2e4ff1d0083ff9 in nifi's branch refs/heads/main from Mike Thomsen [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=82c8449 ] NIFI-7906 Fixed a bug in how record paths were handled. NIFI-7906 Fixed bugs in unit tests. NIFI-7906 Updated a few things based on a code review. NIFI-7906: Fixed typo in record.count attribute Signed-off-by: Matthew Burgess This closes #4739 > Add graph processor with flexibility to query graph database conditioned on > flowfile content and attributes > --- > > Key: NIFI-7906 > URL: https://issues.apache.org/jira/browse/NIFI-7906 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Levi Lentz >Assignee: Levi Lentz >Priority: Minor > Labels: graph > Fix For: 1.13.0 > > Time Spent: 7h 40m > Remaining Estimate: 0h > > The current graph bundle currently does not allow you to query the graph > database (as defined in the GraphClientService) with attributes or content > available in the flow file. > > This functionality would allow uses to perform dynamic queries/mutations of > the underlying graph data based. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 closed pull request #4739: NIFI-7906 Fix a Cypher bug that was missed during initial review
mattyb149 closed pull request #4739: URL: https://github.com/apache/nifi/pull/4739 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7906) Add graph processor with flexibility to query graph database conditioned on flowfile content and attributes
[ https://issues.apache.org/jira/browse/NIFI-7906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17263430#comment-17263430 ] ASF subversion and git services commented on NIFI-7906: --- Commit 82c84492ced10b135242c86afd2e4ff1d0083ff9 in nifi's branch refs/heads/main from Mike Thomsen [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=82c8449 ] NIFI-7906 Fixed a bug in how record paths were handled. NIFI-7906 Fixed bugs in unit tests. NIFI-7906 Updated a few things based on a code review. NIFI-7906: Fixed typo in record.count attribute Signed-off-by: Matthew Burgess This closes #4739 > Add graph processor with flexibility to query graph database conditioned on > flowfile content and attributes > --- > > Key: NIFI-7906 > URL: https://issues.apache.org/jira/browse/NIFI-7906 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Levi Lentz >Assignee: Levi Lentz >Priority: Minor > Labels: graph > Fix For: 1.13.0 > > Time Spent: 7h 50m > Remaining Estimate: 0h > > The current graph bundle currently does not allow you to query the graph > database (as defined in the GraphClientService) with attributes or content > available in the flow file. > > This functionality would allow uses to perform dynamic queries/mutations of > the underlying graph data based. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on pull request #4739: NIFI-7906 Fix a Cypher bug that was missed during initial review
mattyb149 commented on pull request #4739: URL: https://github.com/apache/nifi/pull/4739#issuecomment-758734159 +1 LGTM, ran contrib-check and tried on a live system. I did notice a typo on the `records.count` attribute, I will change that on merge to read `record.count` for consistency with other record-based processors. Also it's interesting that if the fields used in EL in the query aren't defined fields in the input record, it went through correctly (possibly because a `null` or empty string was inserted and no results were returned from the otherwise-valid query). Might want to look into this as a follow-on to see if there's a way to detect whether a field is unset or set to null. Thanks for the improvements! Merging to main This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7356) Enable TLS for embedded Zookeeper when NiFi has TLS enabled
[
https://issues.apache.org/jira/browse/NIFI-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7356:
-
Fix Version/s: 1.13.0
> Enable TLS for embedded Zookeeper when NiFi has TLS enabled
> ---
>
> Key: NIFI-7356
> URL: https://issues.apache.org/jira/browse/NIFI-7356
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Configuration, Configuration Management, Security
>Reporter: Troy Melhase
>Assignee: Troy Melhase
>Priority: Major
> Fix For: 1.13.0
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> If embedded ZK has TLS properties in the {{zookeeper.properties}} file, these
> will be used. If however, this file does not populate those properties, and
> NiFi does have TLS properties configured ({{nifi.security.keyStore}}, etc.),
> these values will be used to override the ZK plaintext connection listener to
> create a TLS connection listener.
> If the {{zookeeper.properties}} file has an incomplete configuration (i.e.
> keystore password set but no keystore path), startup should fail with a clear
> error message indicating the missing properties and how to resolve.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8123) NiFi startup: add option to wait for NiFi to fully load before exiting startup script
[ https://issues.apache.org/jira/browse/NIFI-8123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-8123: - Fix Version/s: (was: 1.13.0) > NiFi startup: add option to wait for NiFi to fully load before exiting > startup script > - > > Key: NIFI-8123 > URL: https://issues.apache.org/jira/browse/NIFI-8123 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Gyori >Assignee: Peter Gyori >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Adding a flag like --wait-for-initialization so that NiFi can be started with > "bin/nifi.sh start --wait-for-initialization" so that the nifi.sh script > would not return until NiFi finishes scheduling all components. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1375) Ship Universal C Runtime DLLs
[ https://issues.apache.org/jira/browse/MINIFICPP-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1375. - Resolution: Fixed > Ship Universal C Runtime DLLs > - > > Key: MINIFICPP-1375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > MiNiFi C++ fails to start on older windows versions that don't have the > Universal C Runtime embedded or the respective update installed. Microsoft > allows for the redistribution of these DLLs and they only take up about 2 MB, > so we should ship them with the MSI for extra compatibility. > > [https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/] (see > point 6. near the bottom) > Redistributable files list: > [https://docs.microsoft.com/en-us/legal/windows-sdk/redist] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1375) Ship Universal C Runtime DLLs
[ https://issues.apache.org/jira/browse/MINIFICPP-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1375: Status: Reopened (was: Reopened) > Ship Universal C Runtime DLLs > - > > Key: MINIFICPP-1375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > MiNiFi C++ fails to start on older windows versions that don't have the > Universal C Runtime embedded or the respective update installed. Microsoft > allows for the redistribution of these DLLs and they only take up about 2 MB, > so we should ship them with the MSI for extra compatibility. > > [https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/] (see > point 6. near the bottom) > Redistributable files list: > [https://docs.microsoft.com/en-us/legal/windows-sdk/redist] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1398) EL date functions are not available on Windows
[
https://issues.apache.org/jira/browse/MINIFICPP-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved MINIFICPP-1398.
-
Resolution: Fixed
> EL date functions are not available on Windows
> --
>
> Key: MINIFICPP-1398
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1398
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> in Expression.h:
> {{#define EXPRESSION_LANGUAGE_USE_DATE}}
> {{// Disable date in EL for incompatible compilers}}
> {{#if \_\_GNUC\_\_ < 5}}
> {{#undef EXPRESSION_LANGUAGE_USE_DATE}}
> {{#endif}}
> I assume \_\_GNUC\_\_ is evaluated as 0 on Windows and that's the cause.
> update: due to problems with the date and timezone libraries on Windows, I'm
> going to add a fallback based on strftime for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #964: MINIFICPP-1443 Fix 'possible loss of data' warnings
szaszm commented on a change in pull request #964:
URL: https://github.com/apache/nifi-minifi-cpp/pull/964#discussion_r555746103
##
File path: extensions/rocksdb-repos/DatabaseContentRepository.cpp
##
@@ -94,7 +97,7 @@ void DatabaseContentRepository::Session::commit() {
if (outStream == nullptr) {
throw Exception(REPOSITORY_EXCEPTION, "Couldn't open the underlying
resource for append: " + resource.first->getContentFullPath());
}
-const auto size = resource.second->size();
+const int size = gsl::narrow(resource.second->size());
Review comment:
I know, it's fine :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #964: MINIFICPP-1443 Fix 'possible loss of data' warnings
fgerlits commented on a change in pull request #964:
URL: https://github.com/apache/nifi-minifi-cpp/pull/964#discussion_r555745253
##
File path: extensions/rocksdb-repos/DatabaseContentRepository.cpp
##
@@ -94,7 +97,7 @@ void DatabaseContentRepository::Session::commit() {
if (outStream == nullptr) {
throw Exception(REPOSITORY_EXCEPTION, "Couldn't open the underlying
resource for append: " + resource.first->getContentFullPath());
}
-const auto size = resource.second->size();
+const int size = gsl::narrow(resource.second->size());
Review comment:
@arpadboda was quicker than you :)
yes, this would have been an improvement, but I don't think it's essential,
so let's leave it as it is, for now
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request #975: MINIFICPP-1400 Create ListS3 processor
lordgamez opened a new pull request #975: URL: https://github.com/apache/nifi-minifi-cpp/pull/975 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #964: MINIFICPP-1443 Fix 'possible loss of data' warnings
arpadboda closed pull request #964: URL: https://github.com/apache/nifi-minifi-cpp/pull/964 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #964: MINIFICPP-1443 Fix 'possible loss of data' warnings
szaszm commented on a change in pull request #964:
URL: https://github.com/apache/nifi-minifi-cpp/pull/964#discussion_r546768707
##
File path: extensions/rocksdb-repos/DatabaseContentRepository.cpp
##
@@ -94,7 +97,7 @@ void DatabaseContentRepository::Session::commit() {
if (outStream == nullptr) {
throw Exception(REPOSITORY_EXCEPTION, "Couldn't open the underlying
resource for append: " + resource.first->getContentFullPath());
}
-const auto size = resource.second->size();
+const int size = gsl::narrow(resource.second->size());
Review comment:
I would narrow it at the last possible moment, as part of the condition
in this case. This is to simplify future changes that make narrowing different
or redundant.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on pull request #974: MINIFICPP-1449 Add pause and resume command to C2
lordgamez commented on pull request #974: URL: https://github.com/apache/nifi-minifi-cpp/pull/974#issuecomment-758556854 The C2PauseResumeTest is flaky because of the possible timings of the GET request and the HeartBeat messages, so it still needs to be stabilized. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #974: MINIFICPP-1449 Add pause and resume command to C2
lordgamez commented on a change in pull request #974:
URL: https://github.com/apache/nifi-minifi-cpp/pull/974#discussion_r555633171
##
File path: libminifi/include/utils/ThreadPool.h
##
@@ -344,11 +364,13 @@ class ThreadPool {
// notification for new delayed tasks that's before the current ones
std::condition_variable delayed_task_available_;
// map to identify if a task should be
- std::map task_status_;
+ std::map task_status_;
// manager mutex
std::recursive_mutex manager_mutex_;
// thread pool name
std::string name_;
+ // map of paused tasks moved from worker queue that should not run at the
time
+ std::map> paused_tasks_;
Review comment:
Thanks for the info, I was not aware of that. I modified the map to be
able to hold multiple tasks for a taskid in
[cd5817c](https://github.com/apache/nifi-minifi-cpp/pull/974/commits/cd5817cd7254305b382734d954ebc5b659453519)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #967: MINIFICPP-1398 MINIFICPP-1375 update date, fix EL date functions on windows, fix ucrt issues, fix vs2019 build issues
arpadboda closed pull request #967: URL: https://github.com/apache/nifi-minifi-cpp/pull/967 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #974: MINIFICPP-1449 Add pause and resume command to C2
adamdebreceni commented on a change in pull request #974:
URL: https://github.com/apache/nifi-minifi-cpp/pull/974#discussion_r86163
##
File path: libminifi/include/utils/ThreadPool.h
##
@@ -344,11 +364,13 @@ class ThreadPool {
// notification for new delayed tasks that's before the current ones
std::condition_variable delayed_task_available_;
// map to identify if a task should be
- std::map task_status_;
+ std::map task_status_;
// manager mutex
std::recursive_mutex manager_mutex_;
// thread pool name
std::string name_;
+ // map of paused tasks moved from worker queue that should not run at the
time
+ std::map> paused_tasks_;
Review comment:
there could be multiple tasks with the same `TaskId` when the
maxConcurrentTasks is greater than 1, this way we could discard tasks on a
pause + resume
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
