[jira] [Commented] (NIFI-11014) JWT token is rejected by NiFi when calling APIs
[ https://issues.apache.org/jira/browse/NIFI-11014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654238#comment-17654238 ] Irudya Raj commented on NIFI-11014: --- Thanks much for the details! Appreciate your input. > JWT token is rejected by NiFi when calling APIs > --- > > Key: NIFI-11014 > URL: https://issues.apache.org/jira/browse/NIFI-11014 > Project: Apache NiFi > Issue Type: Bug > Components: NiFi Stateless >Affects Versions: 1.15.3 > Environment: NiFi with Keycloak as OIDC provider. >Reporter: Irudya Raj >Priority: Major > > I have created oauth token using spring boot and transferred this token to > authorization header bearer. NiFi is configured with PS512 JWS algorithm via > nifi.security.user.oidc.preferred.jwsalgorithm property. But the API request > fails with message "nifi unable to validate the id token: signed jwt > rejected: another algorithm expected, or no matching key(s) found" > I am able to use NiFi web. Keycloak is configure to use PS512 algo for ID > token and access tokens. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] lizhizhou commented on a diff in pull request #6416: NIFI-10234 implement PutIoTDB
lizhizhou commented on code in PR #6416:
URL: https://github.com/apache/nifi/pull/6416#discussion_r1061072425
##
nifi-nar-bundles/nifi-iotdb-bundle/nifi-iotdb-processors/src/main/java/org/apache/nifi/processors/AbstractIoTDB.java:
##
@@ -0,0 +1,365 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processors.model.Field;
+import org.apache.nifi.processors.model.IoTDBSchema;
+import org.apache.iotdb.rpc.IoTDBConnectionException;
+import org.apache.iotdb.session.Session;
+import org.apache.iotdb.tsfile.file.metadata.enums.CompressionType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
+import org.apache.iotdb.tsfile.utils.Binary;
+import org.apache.iotdb.tsfile.write.record.Tablet;
+import org.apache.iotdb.tsfile.write.schema.MeasurementSchema;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.model.ValidationResult;
+import org.apache.nifi.serialization.record.DataType;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+
+public abstract class AbstractIoTDB extends AbstractProcessor {
+private static final int DEFAULT_IOTDB_PORT = 6667;
+
+protected static ObjectMapper mapper = new ObjectMapper();
+
+private static final String TIME_NAME = "timeName";
+private static final String FIELDS = "fields";
+private static final String TIME = "Time";
+
+private static final Map typeMap =
+new HashMap<>();
+
+private static final Map reversedTypeMap =
+new HashMap<>();
+
+static final Set supportedType =
+new HashSet<>();
+
+static final PropertyDescriptor IOTDB_HOST =
+new PropertyDescriptor.Builder()
+.name("Host")
+.description("The host of IoTDB.")
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.required(true)
+.build();
+
+static final PropertyDescriptor IOTDB_PORT =
+new PropertyDescriptor.Builder()
+.name("Port")
+.description("The port of IoTDB.")
+.defaultValue(String.valueOf(DEFAULT_IOTDB_PORT))
+.addValidator(StandardValidators.PORT_VALIDATOR)
+.required(true)
+.addValidator(StandardValidators.PORT_VALIDATOR)
+.build();
+
+static final PropertyDescriptor USERNAME =
+new PropertyDescriptor.Builder()
+.name("Username")
+.description("Username to access the IoTDB.")
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.required(true)
+.build();
+
+static final PropertyDescriptor PASSWORD =
+new PropertyDescriptor.Builder()
+.name("Password")
+.description("Password to access the IoTDB.")
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.required(true)
+.sensitive(true)
+.build();
+
+protected List descriptors = new ArrayList<>();
+
+protected final static Relationship REL_SUCCESS =
+new Relationship.Builder()
+
[jira] [Commented] (NIFI-11014) JWT token is rejected by NiFi when calling APIs
[ https://issues.apache.org/jira/browse/NIFI-11014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654208#comment-17654208 ] David Handermann commented on NIFI-11014: - The project documentation has a guide for [Securing NiFi with TLS|https://nifi.apache.org/docs/nifi-docs/html/walkthroughs.html#securing-nifi-with-tls]. NiFi supports X.509 certificate authentication whenever TLS is configured, regardless of the user-facing authentication strategy. If you already have HTTPS access configured, then the only requirement is obtaining authorized client certificates. The source of client certificates depends on whether you used to the NiFi TLS Toolkit, or some other certificate authority to provision NiFi server certificates. After obtaining client certificates, NiFi must authorize access based on the Subject Distinguished Name. The particulars depend on the User and Group Provider configuration specified in authorizers.xml. > JWT token is rejected by NiFi when calling APIs > --- > > Key: NIFI-11014 > URL: https://issues.apache.org/jira/browse/NIFI-11014 > Project: Apache NiFi > Issue Type: Bug > Components: NiFi Stateless >Affects Versions: 1.15.3 > Environment: NiFi with Keycloak as OIDC provider. >Reporter: Irudya Raj >Priority: Major > > I have created oauth token using spring boot and transferred this token to > authorization header bearer. NiFi is configured with PS512 JWS algorithm via > nifi.security.user.oidc.preferred.jwsalgorithm property. But the API request > fails with message "nifi unable to validate the id token: signed jwt > rejected: another algorithm expected, or no matching key(s) found" > I am able to use NiFi web. Keycloak is configure to use PS512 algo for ID > token and access tokens. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] bejancsaba commented on a diff in pull request #6820: NIFI-11023 Address socket communication instability between minifi pr…
bejancsaba commented on code in PR #6820:
URL: https://github.com/apache/nifi/pull/6820#discussion_r1060952136
##
minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/service/MiNiFiCommandSender.java:
##
@@ -61,6 +61,7 @@ public Optional sendCommand(String cmd, Integer port,
String... extraPar
LOGGER.debug("Connecting to MiNiFi instance");
socket.setSoTimeout(SOCKET_TIMEOUT);
socket.connect(new InetSocketAddress("localhost", port),
CONNECTION_TIMEOUT);
+socket.setSoTimeout(SOCKET_TIMEOUT);
Review Comment:
To be honest I'm not sure. In the old MiNiFi implementation there were both
setting. In the current RunNiFi.sendRequest code the same pattern is followed
similarly to NiFiRegistry. This got removed at a refactor when we moved code
here. However in another place I know @ferencerdei removed the second
setSoTimeout and it caused issues so it was added back. Here I haven't faced
any particular issues explicitly because of this just added it back for the
sake of consistency with the other places in the code. If you feel strongly
about it I can remove of course.
##
minifi/minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/bootstrap/BootstrapListener.java:
##
@@ -291,7 +293,7 @@ private void writeDump(OutputStream out) {
private void echoRequestCmd(String cmd, OutputStream out) {
try {
out.write((cmd + "\n").getBytes(StandardCharsets.UTF_8));
-out.flush();
+out.close();
Review Comment:
Yeah it is possible I tried to look around and it is basically a no brainer
that close should also do flush and it USUALLY does but it is not GUARANTEED. I
added flush back just to be sure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a diff in pull request #6820: NIFI-11023 Address socket communication instability between minifi pr…
mattyb149 commented on code in PR #6820:
URL: https://github.com/apache/nifi/pull/6820#discussion_r1060932185
##
minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/service/MiNiFiCommandSender.java:
##
@@ -61,6 +61,7 @@ public Optional sendCommand(String cmd, Integer port,
String... extraPar
LOGGER.debug("Connecting to MiNiFi instance");
socket.setSoTimeout(SOCKET_TIMEOUT);
socket.connect(new InetSocketAddress("localhost", port),
CONNECTION_TIMEOUT);
+socket.setSoTimeout(SOCKET_TIMEOUT);
Review Comment:
Do we need both calls to `setSoTimeout`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on a diff in pull request #6820: NIFI-11023 Address socket communication instability between minifi pr…
mattyb149 commented on code in PR #6820:
URL: https://github.com/apache/nifi/pull/6820#discussion_r1060931583
##
minifi/minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-runtime/src/main/java/org/apache/nifi/minifi/bootstrap/BootstrapListener.java:
##
@@ -291,7 +293,7 @@ private void writeDump(OutputStream out) {
private void echoRequestCmd(String cmd, OutputStream out) {
try {
out.write((cmd + "\n").getBytes(StandardCharsets.UTF_8));
-out.flush();
+out.close();
Review Comment:
Is it possible to lose this command if the buffer isn't flushed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
dan-s1 commented on PR #6813: URL: https://github.com/apache/nifi/pull/6813#issuecomment-1370182827 @exceptionfactory I was able to refactor most of the lenient code to avoid the `org.mockito.exceptions.misusing.UnnecessaryStubbingException` There is one class though which was too complex `org.apache.nifi.web.controller.ControllerSearchServiceTest` as it has 17 tests and it seems each test has different cause for `org.mockito.exceptions.misusing.UnnecessaryStubbingException`. So I will leave it as is. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6800: NIFI-10997 - Ensure auditing of process group / controller service op…
exceptionfactory commented on code in PR #6800:
URL: https://github.com/apache/nifi/pull/6800#discussion_r1060920376
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/audit/TestProcessGroupAuditor.java:
##
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.audit;
+
+import org.apache.nifi.action.Action;
+import org.apache.nifi.action.FlowChangeAction;
+import org.apache.nifi.action.Operation;
+import org.apache.nifi.admin.service.AuditService;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.authorization.user.NiFiUserDetails;
+import org.apache.nifi.authorization.user.StandardNiFiUser;
+import org.apache.nifi.connectable.ConnectableType;
+import org.apache.nifi.controller.FlowController;
+import org.apache.nifi.controller.ProcessorNode;
+import org.apache.nifi.controller.ScheduledState;
+import org.apache.nifi.controller.StandardProcessorNode;
+import org.apache.nifi.controller.flow.FlowManager;
+import org.apache.nifi.groups.ProcessGroup;
+import org.apache.nifi.web.dao.impl.StandardProcessGroupDAO;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.ArgumentCaptor;
+import org.mockito.Captor;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.boot.test.context.SpringBootTest;
+import org.springframework.context.annotation.EnableAspectJAutoProxy;
+import org.springframework.security.core.Authentication;
+import org.springframework.security.core.context.SecurityContext;
+import org.springframework.security.core.context.SecurityContextHolder;
+import org.springframework.test.context.junit.jupiter.SpringExtension;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+@SuppressWarnings("unused")
+@ExtendWith(SpringExtension.class)
+@SpringBootTest(classes = {
+StandardProcessGroupDAO.class,
+ProcessGroupAuditor.class
+})
Review Comment:
The `ExtendWith(SpringExtension.class)` annotation should be usable without
Spring Boot, and it should be possible to configure the `ProcessGroupAuditor`
through a custom Configuration-annotated class. The Spring Framework
documentation on [dependency injection of test
fixtures](https://docs.spring.io/spring-framework/docs/current/reference/html/testing.html#testcontext-fixture-di)
may provide some helpful background.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6689: [NIFI-10754] Initial check in of new getUri NIFI Expression Language …
exceptionfactory commented on PR #6689: URL: https://github.com/apache/nifi/pull/6689#issuecomment-1370171923 The latest PR build appears to be working as expected, it sounds like the other issues were unrelated intermittent failures. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6821: NIFI-11022 Add DecryptContent Compatibility Processors
exceptionfactory commented on PR #6821: URL: https://github.com/apache/nifi/pull/6821#issuecomment-1370170118 The [DecryptContent Flow](https://issues.apache.org/jira/secure/attachment/13054291/DecryptContent_Flow.json) attached to [NIFI-11022](https://issues.apache.org/jira/browse/NIFI-11022) includes multiple Process Group configurations with `EncryptContent` and the new `DecryptContent` Processors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11022) Add Decrypt Processors Compatible with EncryptContent Encoding
[
https://issues.apache.org/jira/browse/NIFI-11022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11022:
Attachment: DecryptContent_Flow.json
> Add Decrypt Processors Compatible with EncryptContent Encoding
> --
>
> Key: NIFI-11022
> URL: https://issues.apache.org/jira/browse/NIFI-11022
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions, Security
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Attachments: DecryptContent_Flow.json
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{EncryptContent}} Processors supports a wide variety of configuration
> options, enabling both encryption and decryption using various algorithms.
> Many of these algorithms are not secure according to modern cryptographic
> analysis, and existing secure options use a custom encoding format. New
> Processors should be added that support decrypting content according to these
> legacy and custom formats, which will enable deprecating {{EncryptContent}}
> for removal and replacement with other approaches.
> The majority allowable values for the {{Encryption Method}} property in
> {{EncryptContent}} come from the PKCS #5 Password-Based Cryptography
> Specification, described in [RFC
> 8018|https://www.rfc-editor.org/rfc/rfc8018]. These algorithm names start
> with {{PBE}} and incorporate a message digest function along with a cipher
> algorithm. Although these methods include AES, the key derivation process for
> all PBE algorithms follows the {{PBES1}} specification from RFC 8018, which
> is not secure or suitable for modern applications. The ability to decrypt
> older content is useful, but new content should not be encrypted using these
> methods.
> The PBE algorithms can be configured together with either the {{NiFi Legacy}}
> or {{OpenSSL EVP BytesToKey}} option for key derivation. The {{NiFI Legacy}}
> option derives from the [Jasypt|http://www.jasypt.org/] library, which
> provides a standard wrapper for PBE algorithms that defaults to 1000
> iterations of a selected digest algorithm. The {{OpenSSL EVP}} option
> supports compatibility with encryption operations implemented in the
> {{OpenSSL}} library and command.
> Advanced Key Derivation Functions include Argon2, bcrypt, PBKDF2, and scrypt,
> which can be used together with AES in Galois/Counter Mode (GCM) for
> authenticated encryption. These options provide much better security than the
> legacy PBE methods, but they rely on custom file encoding using byte
> delimiters that are specific to Apache NiFi. In addition, these Key
> Derivation Functions generate keys of 16 bytes, which supports AES with 128
> bit keys, but not AES with 256 bit keys. NiFi 0.5.0 added bcrypt, PBKDF2, and
> scrypt, and NiFi 1.12.0 added Argon2. Decrypting content according to the
> custom NiFi encoding should be supported, but other options should be
> evaluated separately for encryption in new flows. The salt parameter bytes
> associated with Argon2, bcrypt, and scrypt allow for detection of file
> encoding, which can enable new decryption processors to be configured without
> reference to a specific Key Derivation Function.
> Adding new decryption processors will enable clear separation of encryption
> and decryption operations, providing a compatible transition path for
> historical usage of {{EncryptContent}} without the need to continue
> supporting insecure encryption methods.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] greyp9 commented on a diff in pull request #6800: NIFI-10997 - Ensure auditing of process group / controller service op…
greyp9 commented on code in PR #6800: URL: https://github.com/apache/nifi/pull/6800#discussion_r1060892434 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/pom.xml: ## @@ -346,6 +346,12 @@ jersey-spring5 provided + +org.springframework.boot +spring-boot-test +2.7.4 Review Comment: > Is there a specific reason 2.7.4 was chosen? It appears that 2.7.4 depends on `spring-core` and `spring-context` 5.3.23 [1]. However, NiFi is already on `spring.version` 5.3.24. I think we bump to 2.7.6 which depends on 5.3.24. > > https://repo1.maven.org/maven2/org/springframework/boot/spring-boot/2.7.4/spring-boot-2.7.4.pom This could definitely be bumped, if we decide to go with things as is. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on a diff in pull request #6800: NIFI-10997 - Ensure auditing of process group / controller service op…
greyp9 commented on code in PR #6800: URL: https://github.com/apache/nifi/pull/6800#discussion_r1060891856 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/pom.xml: ## @@ -346,6 +346,12 @@ jersey-spring5 provided + +org.springframework.boot +spring-boot-test +2.7.4 Review Comment: > Currently only NiFi Registry contains Spring Boot dependencies. If this Spring Boot test dependency remains, recommend promoting the Spring Boot version from the NiFi Registry pom.xml to the root pom.xml for consistent versioning across the project. Sounds good. I could definitely do that, if we decide to keep the test as is. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on a diff in pull request #6800: NIFI-10997 - Ensure auditing of process group / controller service op…
greyp9 commented on code in PR #6800:
URL: https://github.com/apache/nifi/pull/6800#discussion_r1060890993
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/audit/TestProcessGroupAuditor.java:
##
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.audit;
+
+import org.apache.nifi.action.Action;
+import org.apache.nifi.action.FlowChangeAction;
+import org.apache.nifi.action.Operation;
+import org.apache.nifi.admin.service.AuditService;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.authorization.user.NiFiUserDetails;
+import org.apache.nifi.authorization.user.StandardNiFiUser;
+import org.apache.nifi.connectable.ConnectableType;
+import org.apache.nifi.controller.FlowController;
+import org.apache.nifi.controller.ProcessorNode;
+import org.apache.nifi.controller.ScheduledState;
+import org.apache.nifi.controller.StandardProcessorNode;
+import org.apache.nifi.controller.flow.FlowManager;
+import org.apache.nifi.groups.ProcessGroup;
+import org.apache.nifi.web.dao.impl.StandardProcessGroupDAO;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.ArgumentCaptor;
+import org.mockito.Captor;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.boot.test.context.SpringBootTest;
+import org.springframework.context.annotation.EnableAspectJAutoProxy;
+import org.springframework.security.core.Authentication;
+import org.springframework.security.core.context.SecurityContext;
+import org.springframework.security.core.context.SecurityContextHolder;
+import org.springframework.test.context.junit.jupiter.SpringExtension;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+@SuppressWarnings("unused")
+@ExtendWith(SpringExtension.class)
+@SpringBootTest(classes = {
+StandardProcessGroupDAO.class,
+ProcessGroupAuditor.class
+})
Review Comment:
Thanks, @exceptionfactory. Open to a different means of testing this
behavior. Maybe there is a different spring dependency that could accomplish
the same thing? Or maybe the test could be moved to a different module?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on pull request #6689: [NIFI-10754] Initial check in of new getUri NIFI Expression Language …
dan-s1 commented on PR #6689: URL: https://github.com/apache/nifi/pull/6689#issuecomment-1370134527 @exceptionfactory I noticed the build ci-workflow / Ubuntu Zulu JDK 11 HI failed but it failed on a unit test unrelated to my changes. Also the ci-workflow / Ubuntu Zulu JDK 17 EN timed out. Please advise. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11021) Upgrade Groovy to 3.0.14
[ https://issues.apache.org/jira/browse/NIFI-11021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654148#comment-17654148 ] ASF subversion and git services commented on NIFI-11021: Commit 9aecfc4e5b0f3da4cd2b445e32fe047876d4d8ba in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9aecfc4e5b ] NIFI-11021 Upgraded Groovy from 3.0.9 to 3.0.14 Signed-off-by: Matthew Burgess This closes #6819 > Upgrade Groovy to 3.0.14 > > > Key: NIFI-11021 > URL: https://issues.apache.org/jira/browse/NIFI-11021 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 0.5h > Remaining Estimate: 0h > > Groovy [3.0.14|https://groovy-lang.org/changelogs/changelog-3.0.14.html] > includes a number of bug fixes and transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11021) Upgrade Groovy to 3.0.14
[GitHub] [nifi] mattyb149 closed pull request #6819: NIFI-11021 Upgrade Groovy from 3.0.9 to 3.0.14
mattyb149 closed pull request #6819: NIFI-11021 Upgrade Groovy from 3.0.9 to 3.0.14 URL: https://github.com/apache/nifi/pull/6819 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] NissimShiman commented on pull request #6685: NIFI-10608 Copying Process Group now includes non-processor referenced controller services
NissimShiman commented on PR #6685: URL: https://github.com/apache/nifi/pull/6685#issuecomment-1370103926 Thank you very much @mattyb149 and @markobean for looking at this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11022) Add Decrypt Processors Compatible with EncryptContent Encoding
[
https://issues.apache.org/jira/browse/NIFI-11022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-11022:
Status: Patch Available (was: Open)
> Add Decrypt Processors Compatible with EncryptContent Encoding
> --
>
> Key: NIFI-11022
> URL: https://issues.apache.org/jira/browse/NIFI-11022
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions, Security
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The {{EncryptContent}} Processors supports a wide variety of configuration
> options, enabling both encryption and decryption using various algorithms.
> Many of these algorithms are not secure according to modern cryptographic
> analysis, and existing secure options use a custom encoding format. New
> Processors should be added that support decrypting content according to these
> legacy and custom formats, which will enable deprecating {{EncryptContent}}
> for removal and replacement with other approaches.
> The majority allowable values for the {{Encryption Method}} property in
> {{EncryptContent}} come from the PKCS #5 Password-Based Cryptography
> Specification, described in [RFC
> 8018|https://www.rfc-editor.org/rfc/rfc8018]. These algorithm names start
> with {{PBE}} and incorporate a message digest function along with a cipher
> algorithm. Although these methods include AES, the key derivation process for
> all PBE algorithms follows the {{PBES1}} specification from RFC 8018, which
> is not secure or suitable for modern applications. The ability to decrypt
> older content is useful, but new content should not be encrypted using these
> methods.
> The PBE algorithms can be configured together with either the {{NiFi Legacy}}
> or {{OpenSSL EVP BytesToKey}} option for key derivation. The {{NiFI Legacy}}
> option derives from the [Jasypt|http://www.jasypt.org/] library, which
> provides a standard wrapper for PBE algorithms that defaults to 1000
> iterations of a selected digest algorithm. The {{OpenSSL EVP}} option
> supports compatibility with encryption operations implemented in the
> {{OpenSSL}} library and command.
> Advanced Key Derivation Functions include Argon2, bcrypt, PBKDF2, and scrypt,
> which can be used together with AES in Galois/Counter Mode (GCM) for
> authenticated encryption. These options provide much better security than the
> legacy PBE methods, but they rely on custom file encoding using byte
> delimiters that are specific to Apache NiFi. In addition, these Key
> Derivation Functions generate keys of 16 bytes, which supports AES with 128
> bit keys, but not AES with 256 bit keys. NiFi 0.5.0 added bcrypt, PBKDF2, and
> scrypt, and NiFi 1.12.0 added Argon2. Decrypting content according to the
> custom NiFi encoding should be supported, but other options should be
> evaluated separately for encryption in new flows. The salt parameter bytes
> associated with Argon2, bcrypt, and scrypt allow for detection of file
> encoding, which can enable new decryption processors to be configured without
> reference to a specific Key Derivation Function.
> Adding new decryption processors will enable clear separation of encryption
> and decryption operations, providing a compatible transition path for
> historical usage of {{EncryptContent}} without the need to continue
> supporting insecure encryption methods.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6821: NIFI-11022 Add DecryptContent Compatibility Processors
exceptionfactory opened a new pull request, #6821: URL: https://github.com/apache/nifi/pull/6821 # Summary [NIFI-11022](https://issues.apache.org/jira/browse/NIFI-11022) Adds the following new Processors to support decryption of content encrypted using the `EncryptContent` Processor: - `DecryptContentCompatibilityMode` - `DecryptContentEncoded` A new `nifi-cipher-bundle` with `nifi-cipher-nar` module contains the new Processors. A new `nifi-security-crypto-key` module in `nifi-commons` provides standard interfaces and implementations of strong Key Derivation Functions using Bouncy Castle components. The purpose of the new Processors is to maintain components that are capable of decrypting historical information without needing to maintain encryption of insecure and custom formats. ## DecryptContentCompatibilityMode The `DecryptContentCompatibilityMode` Processor focuses on legacy Password-Based Encryption algorithms, many of which are defined in [RFC 8018](https://www.rfc-editor.org/rfc/rfc8018). The Processor supports a `Key Derivation Strategy` using either [OpenSSL EVP BytesToKey](https://www.openssl.org/docs/man3.0/man3/EVP_BytesToKey.html) or [Jasypt Java simplified encryption](http://www.jasypt.org/). These settings map to the OpenSSL and NiFi Legacy settings in the `EncryptContent` Processor. Neither of these key derivation strategies meet the security requirements of modern cryptography, so the `DecryptContentCompatibilityMode` Processor exists exclusively to provide compatibility with historical modes of encryption. This Processor relies on the Bouncy Castle Security Provider to support cipher operations, but avoids global registration of the provider. ## DecryptContentEncoded The `DecryptContentEncoded` Processor focuses on newer Key Derivation Functions, supporting Argon2, bcrypt, PBKDF2, and scrypt algorithms. The Processor supports selection of `Cipher Algorithm Mode` and `Cipher Algorithm Padding`, defaulting to `GCM` and `NoPadding` respectively. These settings match the default configuration of `EncryptContent` for the AES symmetric cipher. The `EncryptContent` Processor and supporting implementation components write encrypted content using a format that is specific to Apache NiFi, and also write the content header using a standard byte format specific to each support Key Derivation Function. The `DecryptContentEncoded` Processor leverages this standard byte formatting to determine the appropriate Key Derivation Function for decryption. The Processor supports a `Key Specification Format` property, defaulting to `PASSWORD`, but also allowing `RAW` to enable configuration of a raw key encoded as a hexadecimal string. Both the `DecryptContentEncoded ` Processor and the supporting `nifi-security-crypto-key` library include extensive tests to exercise byte format detection and cipher algorithm configurations. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #6819: NIFI-11021 Upgrade Groovy from 3.0.9 to 3.0.14
mattyb149 commented on PR #6819: URL: https://github.com/apache/nifi/pull/6819#issuecomment-1370027397 +1 LGTM, checked the release notes for any possible compatibility issues and found none. Will merge when Github Actions completes. Thanks for the upgrade! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
exceptionfactory commented on code in PR #6813: URL: https://github.com/apache/nifi/pull/6813#discussion_r1060789464 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/search/attributematchers/SearchableMatcherTest.java: ## @@ -25,14 +25,17 @@ import org.apache.nifi.search.SearchContext; import org.apache.nifi.search.SearchResult; import org.apache.nifi.search.Searchable; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.mockito.Mock; import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; import java.util.Collection; import java.util.HashSet; +@MockitoSettings(strictness = Strictness.LENIENT) Review Comment: Thanks, that sounds good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
dan-s1 commented on code in PR #6813: URL: https://github.com/apache/nifi/pull/6813#discussion_r1060786499 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/search/attributematchers/SearchableMatcherTest.java: ## @@ -25,14 +25,17 @@ import org.apache.nifi.search.SearchContext; import org.apache.nifi.search.SearchResult; import org.apache.nifi.search.Searchable; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.mockito.Mock; import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; import java.util.Collection; import java.util.HashSet; +@MockitoSettings(strictness = Strictness.LENIENT) Review Comment: @exceptionfactory I ended up using `@MockitoSettings(strictness = Strictness.LENIENT)` where there were multiple mocks and it was hard determining which one was causing the exception so I used this more as a "hammer" solution. The ones where I used `@Mock(lenient = true)` is where one or two mocks were involved so I can look at those again to see if I can remove the unnecessary stubbing. Does the sound reasonable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10608) Copied Processor Group no longer contains non-referenced Controller Services
[ https://issues.apache.org/jira/browse/NIFI-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654106#comment-17654106 ] ASF subversion and git services commented on NIFI-10608: Commit d229f3fd115be9801c1eaae8d0fab6e13f7f6893 in nifi's branch refs/heads/main from Nissim Shiman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d229f3fd11 ] NIFI-10608 Process Group copies include non-processor referenced controller services Signed-off-by: Matthew Burgess This closes #6685 > Copied Processor Group no longer contains non-referenced Controller Services > > > Key: NIFI-10608 > URL: https://issues.apache.org/jira/browse/NIFI-10608 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.3, 1.18.0 >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > A copied processor group does not contain controller service that original > processor group had if controller service had not been referenced to a > processor. > To reproduce: > create new PG -> right click -> Configure -> Controller Sevices -> Add any > controller service (AvroReader for example) > copy PG > new PG -> right click -> Configure -> Controller Services -> (new controller > service will not be listed/ i.e. AvroReader does not show up here) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10608) Copied Processor Group no longer contains non-referenced Controller Services
[ https://issues.apache.org/jira/browse/NIFI-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-10608: Fix Version/s: 1.20.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Copied Processor Group no longer contains non-referenced Controller Services > > > Key: NIFI-10608 > URL: https://issues.apache.org/jira/browse/NIFI-10608 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.3, 1.18.0 >Reporter: Nissim Shiman >Assignee: Nissim Shiman >Priority: Major > Fix For: 1.20.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > A copied processor group does not contain controller service that original > processor group had if controller service had not been referenced to a > processor. > To reproduce: > create new PG -> right click -> Configure -> Controller Sevices -> Add any > controller service (AvroReader for example) > copy PG > new PG -> right click -> Configure -> Controller Services -> (new controller > service will not be listed/ i.e. AvroReader does not show up here) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 closed pull request #6685: NIFI-10608 Copying Process Group now includes non-processor referenced controller services
mattyb149 closed pull request #6685: NIFI-10608 Copying Process Group now includes non-processor referenced controller services URL: https://github.com/apache/nifi/pull/6685 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #6685: NIFI-10608 Copying Process Group now includes non-processor referenced controller services
mattyb149 commented on PR #6685: URL: https://github.com/apache/nifi/pull/6685#issuecomment-1370017990 +1 LGTM, thanks for the review Mark! I too verified the expected behavior. Thanks for the fix! Merging to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
dan-s1 commented on code in PR #6813: URL: https://github.com/apache/nifi/pull/6813#discussion_r1060786499 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/search/attributematchers/SearchableMatcherTest.java: ## @@ -25,14 +25,17 @@ import org.apache.nifi.search.SearchContext; import org.apache.nifi.search.SearchResult; import org.apache.nifi.search.Searchable; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.mockito.Mock; import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; import java.util.Collection; import java.util.HashSet; +@MockitoSettings(strictness = Strictness.LENIENT) Review Comment: @exceptionfactory I ended up using `@MockitoSettings(strictness = Strictness.LENIENT)` where there were multiple mocks and it was hard determining which one was causing the exception so I used this more as a "hammer" solution. The ones where I used on `@Mock(lenient = true)` is where one or two mocks were involved so I can look at those again to see if I can remove the unnecessary stubbing. Does the sound reasonable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
exceptionfactory commented on code in PR #6813: URL: https://github.com/apache/nifi/pull/6813#discussion_r1060780901 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/search/attributematchers/SearchableMatcherTest.java: ## @@ -25,14 +25,17 @@ import org.apache.nifi.search.SearchContext; import org.apache.nifi.search.SearchResult; import org.apache.nifi.search.Searchable; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.mockito.Mock; import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; import java.util.Collection; import java.util.HashSet; +@MockitoSettings(strictness = Strictness.LENIENT) Review Comment: @dan-s1 The ideal solution is to remove the unnecessary stubbing instead of using the lenient mocking mode. If it ends up being too involved, it could be revisited as others noted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
dan-s1 commented on code in PR #6813: URL: https://github.com/apache/nifi/pull/6813#discussion_r1060774811 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/search/attributematchers/SearchableMatcherTest.java: ## @@ -25,14 +25,17 @@ import org.apache.nifi.search.SearchContext; import org.apache.nifi.search.SearchResult; import org.apache.nifi.search.Searchable; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.mockito.Mock; import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; import java.util.Collection; import java.util.HashSet; +@MockitoSettings(strictness = Strictness.LENIENT) Review Comment: @ChrisSamo632 I am not aware of what you are referring to in reference to Mockito functionality. But with the upgrade to JUnit5 the unit tests were failing with an exception like the following where it actually suggests using the lenient option which I did to get these unit cases working. ``` org.mockito.exceptions.misusing.UnnecessaryStubbingException: Unnecessary stubbings detected. Clean & maintainable test code requires zero unnecessary code. ... ... Please remove unnecessary stubbings or use 'lenient' strictness. More info: javadoc for UnnecessaryStubbingException class. ``` @MikeThomsen Is the scope of this ticket in addition to upgrading to Junit5 to remove unnecessary stubbing or is using lenient sufficient to get these unit tests working with JUnit5? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
dan-s1 commented on code in PR #6813: URL: https://github.com/apache/nifi/pull/6813#discussion_r1060774811 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/search/attributematchers/SearchableMatcherTest.java: ## @@ -25,14 +25,17 @@ import org.apache.nifi.search.SearchContext; import org.apache.nifi.search.SearchResult; import org.apache.nifi.search.Searchable; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.mockito.Mock; import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; import java.util.Collection; import java.util.HashSet; +@MockitoSettings(strictness = Strictness.LENIENT) Review Comment: @ChrisSamo632 I am not aware of what you are referring to in reference to Mockito functionality. But with the upgrade to JUnit5 the unit tests were failing with an exception like the following where it actually suggests using the lenient option which I did to get these unit cases working. ``` org.mockito.exceptions.misusing.UnnecessaryStubbingException: Unnecessary stubbings detected. Clean & maintainable test code requires zero unnecessary code. ... ... Please remove unnecessary stubbings or use 'lenient' strictness. More info: javadoc for UnnecessaryStubbingException class. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] dan-s1 commented on a diff in pull request #6813: [NIFI-11006] Upgraded from Junit4 to Junit5
dan-s1 commented on code in PR #6813: URL: https://github.com/apache/nifi/pull/6813#discussion_r1060774811 ## nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/test/java/org/apache/nifi/web/search/attributematchers/SearchableMatcherTest.java: ## @@ -25,14 +25,17 @@ import org.apache.nifi.search.SearchContext; import org.apache.nifi.search.SearchResult; import org.apache.nifi.search.Searchable; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; import org.mockito.Mock; import org.mockito.Mockito; +import org.mockito.junit.jupiter.MockitoSettings; +import org.mockito.quality.Strictness; import java.util.Collection; import java.util.HashSet; +@MockitoSettings(strictness = Strictness.LENIENT) Review Comment: @ChrisSamo632 I am not aware of what you are referring to in reference to Mockito functionality. But with the upgrade to JUnit5 in my cases the unit tests were failing with an exception like the following where it actually suggests using the lenient option which I did to get these unit cases working. ``` org.mockito.exceptions.misusing.UnnecessaryStubbingException: Unnecessary stubbings detected. Clean & maintainable test code requires zero unnecessary code. ... ... Please remove unnecessary stubbings or use 'lenient' strictness. More info: javadoc for UnnecessaryStubbingException class. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bejancsaba opened a new pull request, #6820: NIFI-11023 Address socket communication instability between minifi pr…
bejancsaba opened a new pull request, #6820: URL: https://github.com/apache/nifi/pull/6820 …ocess and bootstrap # Summary [NIFI-11023](https://issues.apache.org/jira/browse/NIFI-11023) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `mvn clean install -P contrib-check` - [x] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-5515) [CaptureChangeMysql] Cannot get real value of primitive type when column type is unsigned.
[ https://issues.apache.org/jira/browse/NIFI-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654083#comment-17654083 ] Matt Burgess commented on NIFI-5515: This information would have to be stored in the cache which would break compatibility. I recommend we ensure this capability works correctly in the CaptureChangeMySQLRecord processor under NIFI-4491. > [CaptureChangeMysql] Cannot get real value of primitive type when column type > is unsigned. > -- > > Key: NIFI-5515 > URL: https://issues.apache.org/jira/browse/NIFI-5515 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.6.0 >Reporter: ashwin konale >Priority: Minor > > *Processor: CaptureChangeMysql* > When mysql column type is of unsigned primitive type, CaptureChangeMysql is > not able to read the data properly when the value overflows. I investigated a > bit and came across this > ([https://github.com/shyiko/mysql-binlog-connector-java/issues/104|https://github.com/shyiko/mysql-binlog-connector-java/issues/104]). > The library which is internally being used in CaptureChangeMysql Processor > ([https://github.com/shyiko/mysql-binlog-connector-java|https://github.com/shyiko/mysql-binlog-connector-java]) > does not handle this case and its upto consumer to handle it. More > description can be found in the link. > *Issue summary Eg.* > Column type : unsigned tinyint > Actual value : 128 > Value from CaptureChangeMysql: -128 (Because its still 8bits storage and > mostsignificant bit is identified as sign from java.) > *Possible solution:* > Column types are read and stored Distributed Map Cache Client so we should be > able to do these conversions. > Nifi version : 1.6.0 > Mysql version : 5.7.22-log > Binlog type. : Row-based logging -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 commented on a diff in pull request #6689: [NIFI-10754] Initial check in of new getUri NIFI Expression Language …
dan-s1 commented on code in PR #6689:
URL: https://github.com/apache/nifi/pull/6689#discussion_r1060723358
##
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/GetUriEvaluator.java:
##
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import org.apache.nifi.attribute.expression.language.EvaluationContext;
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+import
org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageException;
+
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.List;
+import java.util.stream.Collectors;
+
+public class GetUriEvaluator extends StringEvaluator {
+
+private final List> uriArgs;
+
+public GetUriEvaluator(List> uriArgs) {
+this.uriArgs = uriArgs;
+}
+
+@Override
+public QueryResult evaluate(EvaluationContext evaluationContext) {
+List args = uriArgs.stream()
+.map(uriArg -> uriArg.evaluate(evaluationContext).getValue())
+.collect(Collectors.toList());
+
+try {
+if (args.size() == 7) {
+return new StringQueryResult(new URI(args.get(0), args.get(1),
args.get(2),
+Integer.parseInt(args.get(3)), args.get(4),
args.get(5), args.get(6)).toString());
Review Comment:
@exceptionfactory Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11023) MiNiFi - Delay in command response under heavy load causes command execution issues
Csaba Bejan created NIFI-11023: -- Summary: MiNiFi - Delay in command response under heavy load causes command execution issues Key: NIFI-11023 URL: https://issues.apache.org/jira/browse/NIFI-11023 Project: Apache NiFi Issue Type: Improvement Components: MiNiFi Reporter: Csaba Bejan Assignee: Csaba Bejan When MiNiFi is under heavy load the minifi service response to a command from bootstrap can be delayed as message written to the socket doesn't arrive at the bootstrap side. This is a problem for example when MiNiFi is under heavy load and receives a shutdown command. The shutdown response is delayed while the shutdown process is underway. After a while as the response doesn't arrive for bootstrap it considers the command execution as failed and stops. Meanwhile the long running bootstrap realises that the minifi process stopped for some reason and restarts it. So in this case the shutdown command is basically broken as minifi can't be shut down. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 closed pull request #5153: NIFI-6501 Added configurable buffer queue size and timeout in MySQL CDC processor
mattyb149 closed pull request #5153: NIFI-6501 Added configurable buffer queue size and timeout in MySQL CDC processor URL: https://github.com/apache/nifi/pull/5153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10846) GetSmbFile issue after upgrading to Nifi 1.18.0
[
https://issues.apache.org/jira/browse/NIFI-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17654052#comment-17654052
]
Florent commented on NIFI-10846:
[~gkulik]
Hello, can we expect a fix in a future release ?
> GetSmbFile issue after upgrading to Nifi 1.18.0
> ---
>
> Key: NIFI-10846
> URL: https://issues.apache.org/jira/browse/NIFI-10846
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.18.0
> Environment: Nifi Docker on RHEL7.9
> Samba Server : SVM on a metrocluster Netapp, model AFFA400, Ontap version 9.6
>Reporter: Florent
>Priority: Blocker
>
> After upgrading Nifi from 1.17.0 to 1.18.0, we saw some issue regarding
> Processors o.apache.nifi.processors.smb.*
>
> a Simple GetSmbFile works perfectly in 1.17.0 and after upgrading to 1.18.0
> we saw this error
> {code:java}
> 022-11-21 10:16:29,272 ERROR [Timer-Driven Process Thread-5]
> o.apache.nifi.processors.smb.GetSmbFile
> GetSmbFile[id=8b56acf9-0184-1000-ac23-874fb1140496] Could not establish smb
> connection because of error com.hierynomus.mssmb2.SMBApiException:
> STATUS_ACCESS_DENIED (0xc022): Create failed for
> \\MYSERVER\MySHARE\Directory
> at com.hierynomus.smbj.share.Share.receive(Share.java:380)
> at com.hierynomus.smbj.share.Share.sendReceive(Share.java:359)
> at com.hierynomus.smbj.share.Share.createFile(Share.java:156)
> at
> com.hierynomus.smbj.share.DiskShare.createFileAndResolve(DiskShare.java:75)
> at com.hierynomus.smbj.share.DiskShare.access$100(DiskShare.java:55)
> at com.hierynomus.smbj.share.DiskShare$2.apply(DiskShare.java:109)
> at com.hierynomus.smbj.share.DiskShare$2.apply(DiskShare.java:105)
> at
> com.hierynomus.smbj.paths.PathResolver$1.resolve(PathResolver.java:32)
> at
> com.hierynomus.smbj.paths.SymlinkPathResolver.resolve(SymlinkPathResolver.java:62)
> at
> com.hierynomus.smbj.share.DiskShare.resolveAndCreateFile(DiskShare.java:105)
> at com.hierynomus.smbj.share.DiskShare.open(DiskShare.java:65)
> at com.hierynomus.smbj.share.DiskShare.exists(DiskShare.java:214)
> at
> com.hierynomus.smbj.share.DiskShare.folderExists(DiskShare.java:210)
> at
> org.apache.nifi.processors.smb.GetSmbFile.performListing(GetSmbFile.java:334)
> at
> org.apache.nifi.processors.smb.GetSmbFile.onTrigger(GetSmbFile.java:404)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1354)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Suppressed: com.hierynomus.mssmb2.SMBApiException:
> STATUS_ACCESS_DENIED (0xc022): Error closing connection to
> \\MYSERVER\MySHARE
> at
> com.hierynomus.smbj.share.TreeConnect.close(TreeConnect.java:72)
> at com.hierynomus.smbj.share.Share.close(Share.java:116)
> at org.apache.nifi.processors.s {code}
> {{{}We have the same error "{}}}{{{}STATUS_ACCESS_DENIED
> (0xc022){}}}{{{}" with other processor like ListSmb{}}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11022) Add Decrypt Processors Compatible with EncryptContent Encoding
David Handermann created NIFI-11022:
---
Summary: Add Decrypt Processors Compatible with EncryptContent
Encoding
Key: NIFI-11022
URL: https://issues.apache.org/jira/browse/NIFI-11022
Project: Apache NiFi
Issue Type: New Feature
Components: Extensions, Security
Reporter: David Handermann
Assignee: David Handermann
The {{EncryptContent}} Processors supports a wide variety of configuration
options, enabling both encryption and decryption using various algorithms. Many
of these algorithms are not secure according to modern cryptographic analysis,
and existing secure options use a custom encoding format. New Processors should
be added that support decrypting content according to these legacy and custom
formats, which will enable deprecating {{EncryptContent}} for removal and
replacement with other approaches.
The majority allowable values for the {{Encryption Method}} property in
{{EncryptContent}} come from the PKCS #5 Password-Based Cryptography
Specification, described in [RFC 8018|https://www.rfc-editor.org/rfc/rfc8018].
These algorithm names start with {{PBE}} and incorporate a message digest
function along with a cipher algorithm. Although these methods include AES, the
key derivation process for all PBE algorithms follows the {{PBES1}}
specification from RFC 8018, which is not secure or suitable for modern
applications. The ability to decrypt older content is useful, but new content
should not be encrypted using these methods.
The PBE algorithms can be configured together with either the {{NiFi Legacy}}
or {{OpenSSL EVP BytesToKey}} option for key derivation. The {{NiFI Legacy}}
option derives from the [Jasypt|http://www.jasypt.org/] library, which provides
a standard wrapper for PBE algorithms that defaults to 1000 iterations of a
selected digest algorithm. The {{OpenSSL EVP}} option supports compatibility
with encryption operations implemented in the {{OpenSSL}} library and command.
Advanced Key Derivation Functions include Argon2, bcrypt, PBKDF2, and scrypt,
which can be used together with AES in Galois/Counter Mode (GCM) for
authenticated encryption. These options provide much better security than the
legacy PBE methods, but they rely on custom file encoding using byte delimiters
that are specific to Apache NiFi. In addition, these Key Derivation Functions
generate keys of 16 bytes, which supports AES with 128 bit keys, but not AES
with 256 bit keys. NiFi 0.5.0 added bcrypt, PBKDF2, and scrypt, and NiFi 1.12.0
added Argon2. Decrypting content according to the custom NiFi encoding should
be supported, but other options should be evaluated separately for encryption
in new flows. The salt parameter bytes associated with Argon2, bcrypt, and
scrypt allow for detection of file encoding, which can enable new decryption
processors to be configured without reference to a specific Key Derivation
Function.
Adding new decryption processors will enable clear separation of encryption and
decryption operations, providing a compatible transition path for historical
usage of {{EncryptContent}} without the need to continue supporting insecure
encryption methods.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-1887) Add default Connection size limits
[ https://issues.apache.org/jira/browse/MINIFICPP-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Zink reassigned MINIFICPP-1887: -- Assignee: Martin Zink > Add default Connection size limits > -- > > Key: MINIFICPP-1887 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1887 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Martin Zink >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11021) Upgrade Groovy to 3.0.14
[ https://issues.apache.org/jira/browse/NIFI-11021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11021: Status: Patch Available (was: Open) > Upgrade Groovy to 3.0.14 > > > Key: NIFI-11021 > URL: https://issues.apache.org/jira/browse/NIFI-11021 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 10m > Remaining Estimate: 0h > > Groovy [3.0.14|https://groovy-lang.org/changelogs/changelog-3.0.14.html] > includes a number of bug fixes and transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #6819: NIFI-11021 Upgrade Groovy from 3.0.9 to 3.0.14
exceptionfactory opened a new pull request, #6819: URL: https://github.com/apache/nifi/pull/6819 # Summary [NIFI-11021](https://issues.apache.org/jira/browse/NIFI-11021) Upgrades Groovy dependencies from 3.0.9 to to [3.0.14](https://groovy-lang.org/changelogs/changelog-3.0.14.html) to address a number of bugs and incorporate several transitive dependency upgrades. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11021) Upgrade Groovy to 3.0.14
David Handermann created NIFI-11021: --- Summary: Upgrade Groovy to 3.0.14 Key: NIFI-11021 URL: https://issues.apache.org/jira/browse/NIFI-11021 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: David Handermann Assignee: David Handermann Groovy [3.0.14|https://groovy-lang.org/changelogs/changelog-3.0.14.html] includes a number of bug fixes and transitive dependency upgrades. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10996) CSV output with header - but always, even for 0 record flowfiles
[
https://issues.apache.org/jira/browse/NIFI-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-10996:
Affects Version/s: (was: 1.19.1)
Status: Patch Available (was: In Progress)
> CSV output with header - but always, even for 0 record flowfiles
>
>
> Key: NIFI-10996
> URL: https://issues.apache.org/jira/browse/NIFI-10996
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Josef Zahner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: csv, writer
> Attachments: NiFi_CSV_header_true.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We use a “ConvertRecord” processor where we convert an AVRO to a CSV. For
> that *CSV* output we would like to have the {*}header enabled{*}, so we tried
> to set “{{{}Include Header Line – true{}}}” for the Controller Service of the
> CSVRecordSetWriter. The issue is, *if we have zero records, the header
> doesn’t show up* (but it was there of course in the AVRO file). We need to
> have it as the columns are important for us, even if we have 0 records.
> At the moment we solve it with an extra ExecuteScript processor just before
> the ConvertRecord, there we add always an extra record with the header lines
> as string. But it feels a bit hacky as the record.count attribute is 1 record
> too high (due to the fake header record).
> !NiFi_CSV_header_true.png!
> Comment from [~joewitt] from users mailinglist: _"Makes sense what you're
> looking for. Just not sure where this 'concern' would live whether it is in
> the processors themselves or the controller services for the writers."_
> It seems that I'm not alone with that requirement, at least one other person
> (Jens M. Kofoed) uses a similar workaround.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] mattyb149 opened a new pull request, #6818: NIFI-10996: Write out CSV header even if there are no records
mattyb149 opened a new pull request, #6818: URL: https://github.com/apache/nifi/pull/6818 # Summary [NIFI-10996](https://issues.apache.org/jira/browse/NIFI-10996) Adds code to write out the CSV header when configured to do so, even if there are no records written # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 8 - [x] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (MINIFICPP-2020) Minifi main cannot be run on Windows as a normal (non-Admin) user
[
https://issues.apache.org/jira/browse/MINIFICPP-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653873#comment-17653873
]
Ferenc Gerlits commented on MINIFICPP-2020:
---
{quote}Last time I tested, I could run it as Administrator directly from a
terminal.
{quote}
Thanks, I have fixed the title and the description.
> Minifi main cannot be run on Windows as a normal (non-Admin) user
> -
>
> Key: MINIFICPP-2020
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2020
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
> Fix For: 0.14.0
>
>
> If you start the {{minifi}} executable on Windows as a non-Admin user, it
> exits with code -1 immediately, without any error message.
> Also, document generation in {{MiNiFiMain.cpp}} should be moved outside of
> the complicated restarting mechanism, as it doesn't need it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2020) Minifi main cannot be run on Windows as a normal (non-Admin) user
[
https://issues.apache.org/jira/browse/MINIFICPP-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-2020:
--
Priority: Minor (was: Major)
> Minifi main cannot be run on Windows as a normal (non-Admin) user
> -
>
> Key: MINIFICPP-2020
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2020
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
> Fix For: 0.14.0
>
>
> If you start the {{minifi}} executable on Windows as a non-Admin user, it
> exits with code -1 immediately, without any error message.
> Also, document generation in {{MiNiFiMain.cpp}} should be moved outside of
> the complicated restarting mechanism, as it doesn't need it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2020) Minifi main cannot be run on Windows as a normal (non-Admin) user
[
https://issues.apache.org/jira/browse/MINIFICPP-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-2020:
--
Description:
If you start the {{minifi}} executable on Windows as a non-Admin user, it exits
with code -1 immediately, without any error message.
Also, document generation in {{MiNiFiMain.cpp}} should be moved outside of the
complicated restarting mechanism, as it doesn't need it.
was:
If you start the {{minifi}} executable on Windows as a non-system user, it
exits with code -1 immediately, without any error message.
Also, document generation in {{MiNiFiMain.cpp}} should be moved outside of the
complicated restarting mechanism, as it doesn't need it.
> Minifi main cannot be run on Windows as a normal (non-Admin) user
> -
>
> Key: MINIFICPP-2020
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2020
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Major
> Fix For: 0.14.0
>
>
> If you start the {{minifi}} executable on Windows as a non-Admin user, it
> exits with code -1 immediately, without any error message.
> Also, document generation in {{MiNiFiMain.cpp}} should be moved outside of
> the complicated restarting mechanism, as it doesn't need it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2020) Minifi main cannot be run on Windows as a normal (non-Admin) user
[
https://issues.apache.org/jira/browse/MINIFICPP-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits updated MINIFICPP-2020:
--
Summary: Minifi main cannot be run on Windows as a normal (non-Admin) user
(was: Minifi main cannot run on Windows as non-service)
> Minifi main cannot be run on Windows as a normal (non-Admin) user
> -
>
> Key: MINIFICPP-2020
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2020
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Major
> Fix For: 0.14.0
>
>
> If you start the {{minifi}} executable on Windows as a non-system user, it
> exits with code -1 immediately, without any error message.
> Also, document generation in {{MiNiFiMain.cpp}} should be moved outside of
> the complicated restarting mechanism, as it doesn't need it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2021) Enable auto-generation of everything in PROCESSORS.md
[
https://issues.apache.org/jira/browse/MINIFICPP-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits reassigned MINIFICPP-2021:
-
Assignee: Ferenc Gerlits
> Enable auto-generation of everything in PROCESSORS.md
> -
>
> Key: MINIFICPP-2021
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2021
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
>
> PROCESSORS.md can be auto generated by running
> {noformat}
> minifi docs /path/to/dir_containing_individual_processor_docs
> /path/to/new_processors_md
> {noformat}
> However, there are some sections of PROCESSORS.md which do not get
> auto-generated this way:
> * the {{Output Attributes}} sections
> * the {{Writes Attributes}} sections
> * the {{Dynamic Properties}} sections
> * and the {{Predefined Groups}} section in PerformanceDataMonitor.
> Also, the internal, test-only, processors {{KamikazeProcessor}} and
> {{LogOnDestructionProcessor}} should not be included in the generated docs,
> but currently they are.
> It would be good if all of PROCESSORS.md could be auto-generated by the
> command above. As a follow-up, we could then include these new sections in
> the manifest sent to C2, as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2020) Minifi main cannot run on Windows as non-service
[
https://issues.apache.org/jira/browse/MINIFICPP-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits reassigned MINIFICPP-2020:
-
Assignee: Ferenc Gerlits
> Minifi main cannot run on Windows as non-service
>
>
> Key: MINIFICPP-2020
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2020
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Major
> Fix For: 0.14.0
>
>
> If you start the {{minifi}} executable on Windows as a non-system user, it
> exits with code -1 immediately, without any error message.
> Also, document generation in {{MiNiFiMain.cpp}} should be moved outside of
> the complicated restarting mechanism, as it doesn't need it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
