[GitHub] [nifi] github-actions[bot] commented on pull request #5684: NIFI-9605: Docker Images for OpenShift
github-actions[bot] commented on PR #5684: URL: https://github.com/apache/nifi/pull/5684#issuecomment-1172794130 We're marking this PR as stale due to lack of updates in the past few months. If after another couple of weeks the stale label has not been removed this PR will be closed. This stale marker and eventual auto close does not indicate a judgement of the PR just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale tag. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on pull request #6172: NIFI-10143 - ListFTP/GetFTP charset issues
greyp9 commented on PR #6172: URL: https://github.com/apache/nifi/pull/6172#issuecomment-1172776789 > Thanks for digging into this issue and developing a new test @greyp9! The ClientProvider changes make sense. > > The new test is helpful, and covers a lot of ground. It does a lot of work, and it seems to stretch the boundary of a unit test versus an integration test. In light of having existing unit tests for FTP processors, it seems better to maintain a minimal unit test, and then provide this new test class as an integration test. As noted in the detailed comments, there are several ways the test could be optimized. Supporting an external FTP server is also useful, and could be parameterized through System properties or environment variables. > > What do you think about changing the test to an integration test, an updating existing unit tests with more minimal changes to incorporate evaluation of UTF-8 encoding? I've renamed to *IT to align with integration test naming convention. I'd like to defer modifications of the existing unit tests to be part of future efforts, as needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on a diff in pull request #6172: NIFI-10143 - ListFTP/GetFTP charset issues
greyp9 commented on code in PR #6172: URL: https://github.com/apache/nifi/pull/6172#discussion_r912281332 ## nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestFTPCharset.java: ## @@ -0,0 +1,295 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.standard; + +import org.apache.commons.io.FileUtils; +import org.apache.ftpserver.FtpServer; +import org.apache.ftpserver.FtpServerFactory; +import org.apache.ftpserver.ftplet.FileSystemFactory; +import org.apache.ftpserver.ftplet.FileSystemView; +import org.apache.ftpserver.ftplet.FtpException; +import org.apache.ftpserver.ftplet.FtpFile; +import org.apache.ftpserver.ftplet.UserManager; +import org.apache.ftpserver.listener.ListenerFactory; +import org.apache.ftpserver.usermanager.ClearTextPasswordEncryptor; +import org.apache.ftpserver.usermanager.PropertiesUserManagerFactory; +import org.apache.ftpserver.usermanager.impl.BaseUser; +import org.apache.nifi.processor.Processor; +import org.apache.nifi.processor.util.list.AbstractListProcessor; +import org.apache.nifi.processors.standard.util.FTPTransfer; +import org.apache.nifi.util.MockFlowFile; +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.MethodOrderer; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestMethodOrder; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import java.io.File; +import java.io.IOException; +import java.util.Arrays; +import java.util.Collections; +import java.util.Date; +import java.util.Iterator; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import static java.nio.charset.StandardCharsets.UTF_8; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.params.provider.Arguments.arguments; + +/** + * Seed a UTF-7 folder in FTP server with files using i18n known filenames. Iterate through a set of i18n folder + * names, copying the known content into each new target. Afterwards, verify that the last folder contains all + * files with the expected filenames. + * + * To test against a live FTP server: + * + * replace EMBED_FTP_SERVER value with false + * replace HOSTNAME, PORT, USER, PASSWORD as needed + * Review Comment: Added method `serverParametersProvider()`, which allows for override based on a system property. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] greyp9 commented on a diff in pull request #6172: NIFI-10143 - ListFTP/GetFTP charset issues
greyp9 commented on code in PR #6172:
URL: https://github.com/apache/nifi/pull/6172#discussion_r912276213
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestFTPCharset.java:
##
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.ftpserver.FtpServer;
+import org.apache.ftpserver.FtpServerFactory;
+import org.apache.ftpserver.ftplet.FileSystemFactory;
+import org.apache.ftpserver.ftplet.FileSystemView;
+import org.apache.ftpserver.ftplet.FtpException;
+import org.apache.ftpserver.ftplet.FtpFile;
+import org.apache.ftpserver.ftplet.UserManager;
+import org.apache.ftpserver.listener.ListenerFactory;
+import org.apache.ftpserver.usermanager.ClearTextPasswordEncryptor;
+import org.apache.ftpserver.usermanager.PropertiesUserManagerFactory;
+import org.apache.ftpserver.usermanager.impl.BaseUser;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processor.util.list.AbstractListProcessor;
+import org.apache.nifi.processors.standard.util.FTPTransfer;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.MethodOrderer;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestMethodOrder;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.params.provider.Arguments.arguments;
+
+/**
+ * Seed a UTF-7 folder in FTP server with files using i18n known filenames.
Iterate through a set of i18n folder
+ * names, copying the known content into each new target. Afterwards, verify
that the last folder contains all
+ * files with the expected filenames.
+ *
+ * To test against a live FTP server:
+ *
+ * replace EMBED_FTP_SERVER value with false
+ * replace HOSTNAME, PORT, USER, PASSWORD as needed
+ *
+ */
+@TestMethodOrder(MethodOrderer.MethodName.class)
+public class TestFTPCharset {
+private static final boolean EMBED_FTP_SERVER = true;
+private static FtpServer FTP_SERVER;
+
+private static final String USE_UTF8 = Boolean.TRUE.toString();
+private static final String HOSTNAME = "localhost";
+private static final int PORT = 2121;
+private static final String USER = "ftpuser";
+private static final String PASSWORD = "admin";
+private static final File FOLDER = new File("target",
TestFTPCharset.class.getSimpleName());
+
+public static Stream folderNamesProvider() {
+return Stream.of(
+arguments("folder1", "folder2"),
+arguments("folder2", "æøå"),
+arguments("æøå", "folder3"),
+arguments("folder3", "اختبار"),
+arguments("اختبار", "folder4"),
+arguments("folder4", "Госагїzатїой"),
+arguments("Госагїzатїой", "folder5"),
+arguments("folder5", "し回亡丹し工z丹卞工回几"),
+arguments("し回亡丹し工z丹卞工回几", "folder6")
+);
+}
+
+public static Stream filenamesProvider() {
+return Stream.of(
+"1.txt",
+"æøå.txt",
+"اختبار.txt",
+"Госагїzатїой.txt",
+"し回亡丹し工z丹卞工回几.txt");
+}
+
+/**
+ * Start test FTP server (if using embedded).
+ *
+ * @throws FtpException on server startup failure
+ */
+@BeforeAll
+static void beforeAll() throws FtpException {
+if
[GitHub] [nifi] greyp9 commented on a diff in pull request #6172: NIFI-10143 - ListFTP/GetFTP charset issues
greyp9 commented on code in PR #6172:
URL: https://github.com/apache/nifi/pull/6172#discussion_r912275991
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestFTPCharset.java:
##
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.ftpserver.FtpServer;
+import org.apache.ftpserver.FtpServerFactory;
+import org.apache.ftpserver.ftplet.FileSystemFactory;
+import org.apache.ftpserver.ftplet.FileSystemView;
+import org.apache.ftpserver.ftplet.FtpException;
+import org.apache.ftpserver.ftplet.FtpFile;
+import org.apache.ftpserver.ftplet.UserManager;
+import org.apache.ftpserver.listener.ListenerFactory;
+import org.apache.ftpserver.usermanager.ClearTextPasswordEncryptor;
+import org.apache.ftpserver.usermanager.PropertiesUserManagerFactory;
+import org.apache.ftpserver.usermanager.impl.BaseUser;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processor.util.list.AbstractListProcessor;
+import org.apache.nifi.processors.standard.util.FTPTransfer;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.MethodOrderer;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestMethodOrder;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.params.provider.Arguments.arguments;
+
+/**
+ * Seed a UTF-7 folder in FTP server with files using i18n known filenames.
Iterate through a set of i18n folder
+ * names, copying the known content into each new target. Afterwards, verify
that the last folder contains all
+ * files with the expected filenames.
+ *
+ * To test against a live FTP server:
+ *
+ * replace EMBED_FTP_SERVER value with false
+ * replace HOSTNAME, PORT, USER, PASSWORD as needed
+ *
+ */
+@TestMethodOrder(MethodOrderer.MethodName.class)
+public class TestFTPCharset {
+private static final boolean EMBED_FTP_SERVER = true;
+private static FtpServer FTP_SERVER;
+
+private static final String USE_UTF8 = Boolean.TRUE.toString();
+private static final String HOSTNAME = "localhost";
+private static final int PORT = 2121;
+private static final String USER = "ftpuser";
+private static final String PASSWORD = "admin";
+private static final File FOLDER = new File("target",
TestFTPCharset.class.getSimpleName());
+
+public static Stream folderNamesProvider() {
+return Stream.of(
+arguments("folder1", "folder2"),
+arguments("folder2", "æøå"),
+arguments("æøå", "folder3"),
+arguments("folder3", "اختبار"),
+arguments("اختبار", "folder4"),
+arguments("folder4", "Госагїzатїой"),
+arguments("Госагїzатїой", "folder5"),
+arguments("folder5", "し回亡丹し工z丹卞工回几"),
+arguments("し回亡丹し工z丹卞工回几", "folder6")
+);
+}
+
+public static Stream filenamesProvider() {
+return Stream.of(
+"1.txt",
+"æøå.txt",
+"اختبار.txt",
+"Госагїzатїой.txt",
+"し回亡丹し工z丹卞工回几.txt");
+}
+
+/**
+ * Start test FTP server (if using embedded).
+ *
+ * @throws FtpException on server startup failure
+ */
+@BeforeAll
+static void beforeAll() throws FtpException {
+if
[GitHub] [nifi] greyp9 commented on a diff in pull request #6172: NIFI-10143 - ListFTP/GetFTP charset issues
greyp9 commented on code in PR #6172:
URL: https://github.com/apache/nifi/pull/6172#discussion_r912263177
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ftp/StandardFTPClientProvider.java:
##
@@ -152,10 +153,10 @@ private void setClientProperties(final FTPClient client,
final PropertyContext c
client.setRemoteVerificationEnabled(false);
final boolean unicodeEnabled =
context.getProperty(UTF8_ENCODING).isSet() ?
context.getProperty(UTF8_ENCODING).asBoolean() : false;
-if (unicodeEnabled) {
-client.setControlEncoding(StandardCharsets.UTF_8.name());
-client.setAutodetectUTF8(true);
-}
+final Charset charset = unicodeEnabled ? StandardCharsets.UTF_8 :
Charset.defaultCharset();
+client.setCharset(charset);
Review Comment:
It was a speculative change. I have no evidence that it makes a difference.
Removed this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-10166) MiNiFi Bootstrap test coverage
[ https://issues.apache.org/jira/browse/NIFI-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-10166. - Fix Version/s: 1.17.0 Resolution: Fixed > MiNiFi Bootstrap test coverage > -- > > Key: NIFI-10166 > URL: https://issues.apache.org/jira/browse/NIFI-10166 > Project: Apache NiFi > Issue Type: Sub-task > Components: MiNiFi >Reporter: Ferenc Erdei >Assignee: Ferenc Erdei >Priority: Major > Fix For: 1.17.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Improve code coverage of MiNiFi bootstrap -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10166) MiNiFi Bootstrap test coverage
[ https://issues.apache.org/jira/browse/NIFI-10166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17561603#comment-17561603 ] ASF subversion and git services commented on NIFI-10166: Commit d78c667c197cf126e31d291e5f61e41629073d18 in nifi's branch refs/heads/main from Ferenc Erdei [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=d78c667c19 ] NIFI-10166 Improved MiNiFi bootstrap test coverage This closes #6160 Signed-off-by: David Handermann > MiNiFi Bootstrap test coverage > -- > > Key: NIFI-10166 > URL: https://issues.apache.org/jira/browse/NIFI-10166 > Project: Apache NiFi > Issue Type: Sub-task > Components: MiNiFi >Reporter: Ferenc Erdei >Assignee: Ferenc Erdei >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Improve code coverage of MiNiFi bootstrap -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #6160: NIFI-10166 improve MiNiFi bootstrap test coverage
exceptionfactory closed pull request #6160: NIFI-10166 improve MiNiFi bootstrap test coverage URL: https://github.com/apache/nifi/pull/6160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6134: NIFI-10137 Change nifi-toolkit-api to use Jersey 2 and Jackson
exceptionfactory commented on PR #6134: URL: https://github.com/apache/nifi/pull/6134#issuecomment-1172625430 Thanks for the review and feedback @NissimShiman! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6134: NIFI-10137 Change nifi-toolkit-api to use Jersey 2 and Jackson
exceptionfactory commented on code in PR #6134: URL: https://github.com/apache/nifi/pull/6134#discussion_r912190782 ## nifi-toolkit/nifi-toolkit-api/pom.xml: ## @@ -83,7 +84,7 @@ language governing permissions and limitations under the License. --> io.swagger swagger-codegen-maven-plugin -2.2.2 +2.4.27 Review Comment: Thanks for the note @NissimShiman! Moving to Swagger 3 is a good goal, but as you noted, it appears that the codegen plugin still depends on an old version of OkHttp. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6172: NIFI-10143 - ListFTP/GetFTP charset issues
exceptionfactory commented on code in PR #6172:
URL: https://github.com/apache/nifi/pull/6172#discussion_r912188451
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ftp/StandardFTPClientProvider.java:
##
@@ -152,10 +153,10 @@ private void setClientProperties(final FTPClient client,
final PropertyContext c
client.setRemoteVerificationEnabled(false);
final boolean unicodeEnabled =
context.getProperty(UTF8_ENCODING).isSet() ?
context.getProperty(UTF8_ENCODING).asBoolean() : false;
-if (unicodeEnabled) {
-client.setControlEncoding(StandardCharsets.UTF_8.name());
-client.setAutodetectUTF8(true);
-}
+final Charset charset = unicodeEnabled ? StandardCharsets.UTF_8 :
Charset.defaultCharset();
+client.setCharset(charset);
Review Comment:
After evaluating the Apache FTPClient class and parent classes, it appears
that the `Charset` property is not used, so it seems best to avoid setting the
value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] NissimShiman commented on a diff in pull request #6134: NIFI-10137 Change nifi-toolkit-api to use Jersey 2 and Jackson
NissimShiman commented on code in PR #6134: URL: https://github.com/apache/nifi/pull/6134#discussion_r912152508 ## nifi-toolkit/nifi-toolkit-api/pom.xml: ## @@ -83,7 +84,7 @@ language governing permissions and limitations under the License. --> io.swagger swagger-codegen-maven-plugin -2.2.2 +2.4.27 Review Comment: It is unclear if moving to swagger 3 would help this issue (i.e. using a more current version of okhttp) as it appears it still uses okhttp 2. See https://github.com/swagger-api/swagger-codegen/blob/3.0.0/samples/client/petstore/java/okhttp-gson/pom.xml#L215 but maybe there can be a follow up ticket at some point to go to swagger codegen 3 as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] NissimShiman commented on pull request #6134: NIFI-10137 Change nifi-toolkit-api to use Jersey 2 and Jackson
NissimShiman commented on PR #6134: URL: https://github.com/apache/nifi/pull/6134#issuecomment-1172605337 @exceptionfactory This looks good to me. LGTM +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] NissimShiman commented on a diff in pull request #6134: NIFI-10137 Change nifi-toolkit-api to use Jersey 2 and Jackson
NissimShiman commented on code in PR #6134: URL: https://github.com/apache/nifi/pull/6134#discussion_r912152508 ## nifi-toolkit/nifi-toolkit-api/pom.xml: ## @@ -83,7 +84,7 @@ language governing permissions and limitations under the License. --> io.swagger swagger-codegen-maven-plugin -2.2.2 +2.4.27 Review Comment: It is unclear if moving to swagger 3 would help this issue (i.e. using a more current version of okhttp) as it appears it still uses okhttp 2. See https://github.com/swagger-api/swagger-codegen/blob/3.0.0/samples/client/petstore/java/okhttp-gson/pom.xml#L215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6160: NIFI-10166 improve MiNiFi bootstrap test coverage
exceptionfactory commented on code in PR #6160:
URL: https://github.com/apache/nifi/pull/6160#discussion_r912128263
##
minifi/minifi-bootstrap/src/test/java/org/apache/nifi/minifi/bootstrap/command/CommandRunnerFactoryTest.java:
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.command;
+
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.DUMP;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.ENV;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.FLOWSTATUS;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.RESTART;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.RUN;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.START;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.STATUS;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.STOP;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.UNKNOWN;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.verifyNoInteractions;
+
+import java.io.File;
+import org.apache.nifi.minifi.bootstrap.MiNiFiParameters;
+import org.apache.nifi.minifi.bootstrap.RunMiNiFi;
+import org.apache.nifi.minifi.bootstrap.service.BootstrapFileProvider;
+import org.apache.nifi.minifi.bootstrap.service.CurrentPortProvider;
+import
org.apache.nifi.minifi.bootstrap.service.GracefulShutdownParameterProvider;
+import org.apache.nifi.minifi.bootstrap.service.MiNiFiCommandSender;
+import org.apache.nifi.minifi.bootstrap.service.MiNiFiExecCommandProvider;
+import org.apache.nifi.minifi.bootstrap.service.MiNiFiStatusProvider;
+import org.apache.nifi.minifi.bootstrap.service.MiNiFiStdLogHandler;
+import org.apache.nifi.minifi.bootstrap.service.PeriodicStatusReporterManager;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.junit.jupiter.MockitoExtension;
+
+@ExtendWith(MockitoExtension.class)
+class CommandRunnerFactoryTest {
+
+@Mock
+private MiNiFiCommandSender miNiFiCommandSender;
+@Mock
+private CurrentPortProvider currentPortProvider;
+@Mock
+private MiNiFiParameters miNiFiParameters;
+@Mock
+private MiNiFiStatusProvider miNiFiStatusProvider;
+@Mock
+private PeriodicStatusReporterManager periodicStatusReporterManager;
+@Mock
+private BootstrapFileProvider bootstrapFileProvider;
+@Mock
+private MiNiFiStdLogHandler miNiFiStdLogHandler;
+@Mock
+private File bootstrapConfigFile;
+@Mock
+private RunMiNiFi runMiNiFi;
+@Mock
+private GracefulShutdownParameterProvider
gracefulShutdownParameterProvider;
+@Mock
+private MiNiFiExecCommandProvider miNiFiExecCommandProvider;
+
+@InjectMocks
+private CommandRunnerFactory commandRunnerFactory;
+
+@Test
+void testRunCommandShouldStartCommandReturnStartRunner() {
+CommandRunner runner = commandRunnerFactory.getRunner(START);
+
+assertTrue(runner instanceof StartRunner);
+verifyNoInteractions(miNiFiCommandSender, currentPortProvider,
miNiFiParameters, miNiFiStatusProvider, periodicStatusReporterManager,
bootstrapFileProvider,
+miNiFiStdLogHandler, bootstrapConfigFile, runMiNiFi,
gracefulShutdownParameterProvider, miNiFiExecCommandProvider);
+}
+
+@Test
+void testRunCommandShouldRunCommandReturnStartRunner() {
+CommandRunner runner = commandRunnerFactory.getRunner(RUN);
+
+assertTrue(runner instanceof StartRunner);
+verifyNoInteractions(miNiFiCommandSender, currentPortProvider,
miNiFiParameters, miNiFiStatusProvider, periodicStatusReporterManager,
bootstrapFileProvider,
Review Comment:
Thanks for the response, that makes sense in this scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about
[GitHub] [nifi] markobean commented on pull request #6151: NIFI-10154 ReplaceText AdminYielding on long line
markobean commented on PR #6151: URL: https://github.com/apache/nifi/pull/6151#issuecomment-1172548765 Less critical, but in passing I noted an inconsistency in the documentation for AbstractTextDemarcator.fill(). It mentions throwing TokenTooLargeException in the description, and explicitly indicates an IOException. In reality, it only throws BufferOverflowException. Suggest updating the javadoc info to be consistent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markobean commented on pull request #6151: NIFI-10154 ReplaceText AdminYielding on long line
markobean commented on PR #6151: URL: https://github.com/apache/nifi/pull/6151#issuecomment-1172541808 I found some interesting things while testing this PR. When using ReplaceText in a flow, I found a buffer size < 8k never caused the BufferOverflowException. I believe this is due to a hard-coded default initial size of the buffer in StreamCallback (ReplaceText:747). While 8k seems a reasonable limit, I think it would be more accurate to use `Math.min(maxBufferSize, 8192)`. Otherwise, ReplaceText will not fail FlowFiles with lines larger than the buffer size (and less than 8 KB). I can imagine some users may want to use the failure relationship to identify lines longer than X bytes/characters. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #6174: NIFI-10189 Added WriteText processor to the standard bundle.
joewitt commented on PR #6174: URL: https://github.com/apache/nifi/pull/6174#issuecomment-1172540099 So I agree with the spirit of what Mike is trying to solve. Make it easier for users to find the right function with minimal additional complexity to do the thing. Especially for simple stuff. I agree with Pierre/David on what the solution is for this case. So how can we meet in the middle? Perhaps a tag or set of tags added to the processor would aid in finding the component faster. Perhaps add a processor tag that is literally 'WriteText' or even 'Write Text' though I'm not sure what we do with spaces. That simple step might be just enough. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markobean commented on pull request #6151: NIFI-10154 ReplaceText AdminYielding on long line
markobean commented on PR #6151: URL: https://github.com/apache/nifi/pull/6151#issuecomment-1172538089 There is a typo in a similar error message for exceeding buffer on line 319. It's not explicitly part of this PR, but please update. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6174: NIFI-10189 Added WriteText processor to the standard bundle.
exceptionfactory commented on PR #6174: URL: https://github.com/apache/nifi/pull/6174#issuecomment-1172536272 As @pvillard31 noted, the `Always Replace` strategy seems to meet the use case described, and would seem to obviate the need for a new processor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6172: NIFI-10143 - ListFTP/GetFTP charset issues
exceptionfactory commented on code in PR #6172:
URL: https://github.com/apache/nifi/pull/6172#discussion_r911999082
##
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestFTPCharset.java:
##
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.ftpserver.FtpServer;
+import org.apache.ftpserver.FtpServerFactory;
+import org.apache.ftpserver.ftplet.FileSystemFactory;
+import org.apache.ftpserver.ftplet.FileSystemView;
+import org.apache.ftpserver.ftplet.FtpException;
+import org.apache.ftpserver.ftplet.FtpFile;
+import org.apache.ftpserver.ftplet.UserManager;
+import org.apache.ftpserver.listener.ListenerFactory;
+import org.apache.ftpserver.usermanager.ClearTextPasswordEncryptor;
+import org.apache.ftpserver.usermanager.PropertiesUserManagerFactory;
+import org.apache.ftpserver.usermanager.impl.BaseUser;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processor.util.list.AbstractListProcessor;
+import org.apache.nifi.processors.standard.util.FTPTransfer;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.MethodOrderer;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.TestMethodOrder;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Date;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.params.provider.Arguments.arguments;
+
+/**
+ * Seed a UTF-7 folder in FTP server with files using i18n known filenames.
Iterate through a set of i18n folder
+ * names, copying the known content into each new target. Afterwards, verify
that the last folder contains all
+ * files with the expected filenames.
+ *
+ * To test against a live FTP server:
+ *
+ * replace EMBED_FTP_SERVER value with false
+ * replace HOSTNAME, PORT, USER, PASSWORD as needed
+ *
+ */
+@TestMethodOrder(MethodOrderer.MethodName.class)
+public class TestFTPCharset {
+private static final boolean EMBED_FTP_SERVER = true;
+private static FtpServer FTP_SERVER;
+
+private static final String USE_UTF8 = Boolean.TRUE.toString();
+private static final String HOSTNAME = "localhost";
+private static final int PORT = 2121;
+private static final String USER = "ftpuser";
+private static final String PASSWORD = "admin";
+private static final File FOLDER = new File("target",
TestFTPCharset.class.getSimpleName());
+
+public static Stream folderNamesProvider() {
+return Stream.of(
+arguments("folder1", "folder2"),
+arguments("folder2", "æøå"),
+arguments("æøå", "folder3"),
+arguments("folder3", "اختبار"),
+arguments("اختبار", "folder4"),
+arguments("folder4", "Госагїzатїой"),
+arguments("Госагїzатїой", "folder5"),
+arguments("folder5", "し回亡丹し工z丹卞工回几"),
+arguments("し回亡丹し工z丹卞工回几", "folder6")
+);
+}
+
+public static Stream filenamesProvider() {
+return Stream.of(
+"1.txt",
+"æøå.txt",
+"اختبار.txt",
+"Госагїzатїой.txt",
+"し回亡丹し工z丹卞工回几.txt");
+}
+
+/**
+ * Start test FTP server (if using embedded).
+ *
+ * @throws FtpException on server startup failure
+ */
+@BeforeAll
+static void beforeAll() throws FtpException {
+
[GitHub] [nifi] Lehel44 closed pull request #5654: NIFI-9558: ConnectWebSocket leaks connections and duplicates FlowFile
Lehel44 closed pull request #5654: NIFI-9558: ConnectWebSocket leaks connections and duplicates FlowFile URL: https://github.com/apache/nifi/pull/5654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] NissimShiman commented on pull request #6077: NIFI-10050: When copying a process group and 1 CS references a higher…
NissimShiman commented on PR #6077: URL: https://github.com/apache/nifi/pull/6077#issuecomment-1172402045 Good eye to catch this subtle issue. Very minor point, but maybe add step to jira example to set CSVReader's SchemaAccessStrategy to HWS Schema Reference Attributes (or some other other value where Schema Registry property is displayed) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10191) Method to stop NiFi on windows
Dennis Jaheruddin created NIFI-10191: Summary: Method to stop NiFi on windows Key: NIFI-10191 URL: https://issues.apache.org/jira/browse/NIFI-10191 Project: Apache NiFi Issue Type: Improvement Reporter: Dennis Jaheruddin I was unable to find an elegant method to stop NiFi on windows. After a search it was suggested to change run nifi.bat thus making it a stopping function. I succesfully tested this (Actually making a copy called stop nifi.bat), so I guess we would either want to add a file called stop nifi, or parameterize the run-nifi to accept a stopping parameter. The only reason I did not create anything to fix this is because there may already be a way to do this, so I will leave this idea open for a bit before picking it up. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 commented on a diff in pull request #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0
mr1716 commented on code in PR #6175: URL: https://github.com/apache/nifi/pull/6175#discussion_r911933610 ## nifi-nar-bundles/nifi-azure-bundle/pom.xml: ## @@ -31,7 +31,7 @@ 1.4.5 1.11.0 -4.26.0 +4.31.0 Review Comment: @exceptionfactory Thanks for the information. I'll come back to this in a bit. There are a few other updates that I want to test out first that go from >1 CVE to 0 CVE and see how that works. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-9809) Disable Kudu Tests on arm64 platforms
[
https://issues.apache.org/jira/browse/NIFI-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran reassigned NIFI-9809:
-
Assignee: Kevin Doran
> Disable Kudu Tests on arm64 platforms
> -
>
> Key: NIFI-9809
> URL: https://issues.apache.org/jira/browse/NIFI-9809
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Assignee: Kevin Doran
>Priority: Minor
>
> The kudu-binary artifact is not available for aarch64 (arm64) platforms:
> {noformat}
> Could not resolve dependencies for project
> org.apache.nifi:nifi-kudu-controller-service:jar:1.17.0-SNAPSHOT:
> Could not transfer artifact
> org.apache.kudu:kudu-binary:jar:osx-aarch_64:1.16.0
> {noformat}
> AFAICT, we only use this dependency in test code. so excluding those tests
> from compiling / running when on those platforms should fix the kudu bundle
> build when running on that chip architecture.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-9809) Disable Kudu Tests on arm64 platforms
[
https://issues.apache.org/jira/browse/NIFI-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran updated NIFI-9809:
--
Description:
The kudu-binary artifact is not available for aarch64 (arm64) platforms:
{noformat}
Could not resolve dependencies for project
org.apache.nifi:nifi-kudu-controller-service:jar:1.17.0-SNAPSHOT:
Could not transfer artifact org.apache.kudu:kudu-binary:jar:osx-aarch_64:1.16.0
{noformat}
AFAICT, we only use this dependency in test code. so excluding those tests from
compiling / running when on those platforms should fix the kudu bundle build
when running on that chip architecture.
was:
The kudu-binary artifact is not available for aarch64 (arm64) platforms:
{noformat}
We only use this dependency in test code. so excluding those tests from
compiling / running when on those platforms should fix the kudu bundle build
when running on that chip architecture.
{noformat}
Could not resolve dependencies for project
org.apache.nifi:nifi-kudu-controller-service:jar:1.17.0-SNAPSHOT: Could not
transfer artifact org.apache.kudu:kudu-binary:jar:osx-aarch_64:1.16.0
> Disable Kudu Tests on arm64 platforms
> -
>
> Key: NIFI-9809
> URL: https://issues.apache.org/jira/browse/NIFI-9809
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Priority: Minor
>
> The kudu-binary artifact is not available for aarch64 (arm64) platforms:
> {noformat}
> Could not resolve dependencies for project
> org.apache.nifi:nifi-kudu-controller-service:jar:1.17.0-SNAPSHOT:
> Could not transfer artifact
> org.apache.kudu:kudu-binary:jar:osx-aarch_64:1.16.0
> {noformat}
> AFAICT, we only use this dependency in test code. so excluding those tests
> from compiling / running when on those platforms should fix the kudu bundle
> build when running on that chip architecture.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-9809) Disable Kudu Tests on arm64 platforms
[
https://issues.apache.org/jira/browse/NIFI-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran updated NIFI-9809:
--
Description:
The kudu-binary artifact is not available for aarch64 (arm64) platforms:
{noformat}
We only use this dependency in test code. so excluding those tests from
compiling / running when on those platforms should fix the kudu bundle build
when running on that chip architecture.
{noformat}
Could not resolve dependencies for project
org.apache.nifi:nifi-kudu-controller-service:jar:1.17.0-SNAPSHOT: Could not
transfer artifact org.apache.kudu:kudu-binary:jar:osx-aarch_64:1.16.0
was:For now, the nifi-kudu-bundle is excluded when building for arm because
the kudu dependencies are not available for arm platforms. This ticket is to
investigate what it would take to support kudu on NiFi on arm.
> Disable Kudu Tests on arm64 platforms
> -
>
> Key: NIFI-9809
> URL: https://issues.apache.org/jira/browse/NIFI-9809
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Kevin Doran
>Priority: Minor
>
> The kudu-binary artifact is not available for aarch64 (arm64) platforms:
> {noformat}
> We only use this dependency in test code. so excluding those tests from
> compiling / running when on those platforms should fix the kudu bundle build
> when running on that chip architecture.
> {noformat}
> Could not resolve dependencies for project
> org.apache.nifi:nifi-kudu-controller-service:jar:1.17.0-SNAPSHOT: Could not
> transfer artifact org.apache.kudu:kudu-binary:jar:osx-aarch_64:1.16.0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-9809) Disable Kudu Tests on arm64 platforms
[ https://issues.apache.org/jira/browse/NIFI-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-9809: -- Summary: Disable Kudu Tests on arm64 platforms (was: Support Kudu NAR on arm64 platforms) > Disable Kudu Tests on arm64 platforms > - > > Key: NIFI-9809 > URL: https://issues.apache.org/jira/browse/NIFI-9809 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Kevin Doran >Priority: Minor > > For now, the nifi-kudu-bundle is excluded when building for arm because the > kudu dependencies are not available for arm platforms. This ticket is to > investigate what it would take to support kudu on NiFi on arm. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0
exceptionfactory commented on code in PR #6175: URL: https://github.com/apache/nifi/pull/6175#discussion_r911926375 ## nifi-nar-bundles/nifi-azure-bundle/pom.xml: ## @@ -31,7 +31,7 @@ 1.4.5 1.11.0 -4.26.0 +4.31.0 Review Comment: @mr1716 NiFi uses the Jackson Bill-of-Materials dependency in the root Maven configuration to force the latest version across all modules. Based on that approach, NiFi is not subject to the vulnerability. However, it is still worth upgrading this dependency. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (NIFI-10190) Update azure-cosmos from 4.26.0 to 4.31.0
[ https://issues.apache.org/jira/browse/NIFI-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R resolved NIFI-10190. --- Resolution: Won't Fix > Update azure-cosmos from 4.26.0 to 4.31.0 > - > > Key: NIFI-10190 > URL: https://issues.apache.org/jira/browse/NIFI-10190 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.16.1, 1.16.2, 1.16.3 >Reporter: Mike R >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Update azure-cosmos component from 4.26.0 to 4.31.0 to remediate > CVE-2020-36518 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10190) Update azure-cosmos from 4.26.0 to 4.31.0
[ https://issues.apache.org/jira/browse/NIFI-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17561441#comment-17561441 ] Mike R commented on NIFI-10190: --- Will wait until the CVE is updated before updating this. > Update azure-cosmos from 4.26.0 to 4.31.0 > - > > Key: NIFI-10190 > URL: https://issues.apache.org/jira/browse/NIFI-10190 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.16.1, 1.16.2, 1.16.3 >Reporter: Mike R >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Update azure-cosmos component from 4.26.0 to 4.31.0 to remediate > CVE-2020-36518 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 closed pull request #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0
mr1716 closed pull request #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0 URL: https://github.com/apache/nifi/pull/6175 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-10190) Update azure-cosmos from 4.26.0 to 4.31.0
[ https://issues.apache.org/jira/browse/NIFI-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17561440#comment-17561440 ] Mike R commented on NIFI-10190: --- Even with the update, it is still vulnerable to [CVE-2020-8908|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908]. So maybe wait until that is resolved in the Azure links, then go from there. > Update azure-cosmos from 4.26.0 to 4.31.0 > - > > Key: NIFI-10190 > URL: https://issues.apache.org/jira/browse/NIFI-10190 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.16.1, 1.16.2, 1.16.3 >Reporter: Mike R >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Update azure-cosmos component from 4.26.0 to 4.31.0 to remediate > CVE-2020-36518 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 commented on a diff in pull request #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0
mr1716 commented on code in PR #6175: URL: https://github.com/apache/nifi/pull/6175#discussion_r911921943 ## nifi-nar-bundles/nifi-azure-bundle/pom.xml: ## @@ -31,7 +31,7 @@ 1.4.5 1.11.0 -4.26.0 +4.31.0 Review Comment: If so, we can close this issue and wait for the update. I believe that might be the best option here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mr1716 commented on a diff in pull request #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0
mr1716 commented on code in PR #6175: URL: https://github.com/apache/nifi/pull/6175#discussion_r911921267 ## nifi-nar-bundles/nifi-azure-bundle/pom.xml: ## @@ -31,7 +31,7 @@ 1.4.5 1.11.0 -4.26.0 +4.31.0 Review Comment: @exceptionfactory will do. It looks like 4.32.1 is still vulnerable to [CVE-2020-8908](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908) So maybe the best thing to do is hold off on this until they fix that issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #6174: NIFI-10189 Added WriteText processor to the standard bundle.
pvillard31 commented on PR #6174: URL: https://github.com/apache/nifi/pull/6174#issuecomment-1172295029 ReplaceText with "Always Replace" as the strategy won't evaluate the flowfile's content. We're also checking the specified regex to see if Always Replace should be applied by default. In that case, if the user does not change the replacement strategy we would skip the regex evaluation anyway. https://github.com/apache/nifi/blob/main/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java#L277-L281 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0
exceptionfactory commented on code in PR #6175:
URL: https://github.com/apache/nifi/pull/6175#discussion_r911916049
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/pom.xml:
##
@@ -97,7 +97,7 @@
com.azure
azure-cosmos
-${azure-cosmos.version}
+4.31.0
Review Comment:
This change should be reverted.
##
nifi-nar-bundles/nifi-azure-bundle/pom.xml:
##
@@ -31,7 +31,7 @@
1.4.5
1.11.0
-4.26.0
+4.31.0
Review Comment:
The latest version appears to be 4.32.0, released on 2022-06-28. Can you
upgrade to that version?
```suggestion
4.32.0
```
##
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-services-api/pom.xml:
##
@@ -42,7 +42,7 @@
com.azure
azure-cosmos
-${azure-cosmos.version}
+4.31.0
Review Comment:
This change should be reverted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10187) Upgrade Apache FtpServer to 1.2.0
[ https://issues.apache.org/jira/browse/NIFI-10187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-10187: -- Fix Version/s: 1.17.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Apache FtpServer to 1.2.0 > - > > Key: NIFI-10187 > URL: https://issues.apache.org/jira/browse/NIFI-10187 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 1.17.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Apache FtpServer 1.1.1 depends on a version of Apache MINA vulnerable to > CVE-2019-0231, related to socket close handling for TLS communication. The > FtpServer dependency should be upgraded to 1.2.0, which incorporates a new > version of Apache MINA. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10187) Upgrade Apache FtpServer to 1.2.0
[ https://issues.apache.org/jira/browse/NIFI-10187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17561436#comment-17561436 ] ASF subversion and git services commented on NIFI-10187: Commit ea0e1b446ef83021f9e685975b3eb643a207af1f in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ea0e1b446e ] NIFI-10187 Upgraded FtpServer from 1.1.1 to 1.2.0 Signed-off-by: Pierre Villard This closes #6171. > Upgrade Apache FtpServer to 1.2.0 > - > > Key: NIFI-10187 > URL: https://issues.apache.org/jira/browse/NIFI-10187 > Project: Apache NiFi > Issue Type: Improvement >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Time Spent: 10m > Remaining Estimate: 0h > > Apache FtpServer 1.1.1 depends on a version of Apache MINA vulnerable to > CVE-2019-0231, related to socket close handling for TLS communication. The > FtpServer dependency should be upgraded to 1.2.0, which incorporates a new > version of Apache MINA. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #6171: NIFI-10187 Upgrade FtpServer from 1.1.1 to 1.2.0
asfgit closed pull request #6171: NIFI-10187 Upgrade FtpServer from 1.1.1 to 1.2.0 URL: https://github.com/apache/nifi/pull/6171 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1362: MINIFICPP-1830 Verify regex properties in processors in case it is set to empty
lordgamez commented on code in PR #1362:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1362#discussion_r911906233
##
libminifi/include/utils/HTTPUtils.h:
##
@@ -31,14 +31,14 @@ so we convert localhost to our local hostname.
inline bool parse_http_components(const std::string , std::string ,
std::string , std::string ) {
#ifdef WIN32
auto hostname =
(url.find(org::apache::nifi::minifi::io::Socket::getMyHostName()) !=
std::string::npos ? org::apache::nifi::minifi::io::Socket::getMyHostName() :
"localhost");
- std::string regex_str = "^(http|https)://(" + hostname + ":)([0-9]+)?(/.*)$";
+ std::string regex_str = "(http|https)://(" + hostname + ":)([0-9]+)?(/.*)$";
#else
- std::string regex_str = "^(http|https)://(localhost:)([0-9]+)?(/.*)$";
+ std::string regex_str = "(http|https)://(localhost:)([0-9]+)?(/.*)$";
Review Comment:
Good catch, removed in 4d1784c2a28e7e9982f71ba3c77d0fac4f88c0c4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1362: MINIFICPP-1830 Verify regex properties in processors in case it is set to empty
szaszm commented on code in PR #1362:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1362#discussion_r911902756
##
libminifi/include/utils/HTTPUtils.h:
##
@@ -31,14 +31,14 @@ so we convert localhost to our local hostname.
inline bool parse_http_components(const std::string , std::string ,
std::string , std::string ) {
#ifdef WIN32
auto hostname =
(url.find(org::apache::nifi::minifi::io::Socket::getMyHostName()) !=
std::string::npos ? org::apache::nifi::minifi::io::Socket::getMyHostName() :
"localhost");
- std::string regex_str = "^(http|https)://(" + hostname + ":)([0-9]+)?(/.*)$";
+ std::string regex_str = "(http|https)://(" + hostname + ":)([0-9]+)?(/.*)$";
#else
- std::string regex_str = "^(http|https)://(localhost:)([0-9]+)?(/.*)$";
+ std::string regex_str = "(http|https)://(localhost:)([0-9]+)?(/.*)$";
Review Comment:
Please remove the trailing `$` sign / end of line match as well. It should
no longer be necessary, just like `^`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1362: MINIFICPP-1830 Verify regex properties in processors in case it is set to empty
lordgamez commented on code in PR #1362:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1362#discussion_r911883288
##
extensions/librdkafka/PublishKafka.cpp:
##
@@ -184,12 +184,12 @@ class ReadCallback {
});
}
- static rd_kafka_headers_unique_ptr make_headers(const core::FlowFile&
flow_file, utils::Regex& attribute_name_regex) {
+ static rd_kafka_headers_unique_ptr make_headers(const core::FlowFile&
flow_file, std::optional& attribute_name_regex) {
const gsl::owner result{ rd_kafka_headers_new(8) };
if (!result) { throw std::bad_alloc{}; }
for (const auto& kv : flow_file.getAttributes()) {
- if (utils::regexSearch(kv.first, attribute_name_regex)) {
+ if (attribute_name_regex && utils::regexSearch(kv.first,
*attribute_name_regex)) {
Review Comment:
Good idea, updated in 2c70c9d7f88a3b7568408c2b45c44516b199ae37
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-10190) Update azure-cosmos from 4.26.0 to 4.31.0
[ https://issues.apache.org/jira/browse/NIFI-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike R updated NIFI-10190: -- Description: Update azure-cosmos component from 4.26.0 to 4.31.0 to remediate CVE-2020-36518 (was: Update azure-cosmos component from 4.26.0 to 4.31.0) > Update azure-cosmos from 4.26.0 to 4.31.0 > - > > Key: NIFI-10190 > URL: https://issues.apache.org/jira/browse/NIFI-10190 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.16.1, 1.16.2, 1.16.3 >Reporter: Mike R >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Update azure-cosmos component from 4.26.0 to 4.31.0 to remediate > CVE-2020-36518 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mr1716 opened a new pull request, #6175: NIFI-10190 Update azure-cosmos from 4.26.0 to 4.31.0
mr1716 opened a new pull request, #6175: URL: https://github.com/apache/nifi/pull/6175 # Summary [NiFi 10190](https://issues.apache.org/jira/browse/NIFI-10190) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [NiFi 10190](https://issues.apache.org/jira/browse/NIFI-10190) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [X] JDK 8 - [X] JDK 11 - [X] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-10190) Update azure-cosmos from 4.26.0 to 4.31.0
Mike R created NIFI-10190: - Summary: Update azure-cosmos from 4.26.0 to 4.31.0 Key: NIFI-10190 URL: https://issues.apache.org/jira/browse/NIFI-10190 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.16.3, 1.16.2, 1.16.1 Reporter: Mike R Update azure-cosmos component from 4.26.0 to 4.31.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] ferencerdei commented on a diff in pull request #6160: NIFI-10166 improve MiNiFi bootstrap test coverage
ferencerdei commented on code in PR #6160:
URL: https://github.com/apache/nifi/pull/6160#discussion_r911639888
##
minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java:
##
@@ -750,6 +756,72 @@ protected static void addTextElement(final Element
element, final String name, f
element.appendChild(toAdd);
}
+public static ByteBuffer overrideNonFlowSectionsFromOriginalSchema(byte[]
newSchema, ByteBuffer currentConfigScheme, Properties bootstrapProperties)
+throws IOException, InvalidConfigurationException,
SchemaLoaderException {
+try {
+boolean overrideCoreProperties =
ConfigTransformer.overrideCoreProperties(bootstrapProperties);
+boolean overrideSecurityProperties =
ConfigTransformer.overrideSecurityProperties(bootstrapProperties);
+if (overrideCoreProperties && overrideSecurityProperties) {
+return ByteBuffer.wrap(newSchema);
+} else {
+ConvertableSchema schemaNew = ConfigTransformer
+
.throwIfInvalid(SchemaLoader.loadConvertableSchemaFromYaml(new
ByteArrayInputStream(newSchema)));
+ConfigSchema configSchemaNew =
ConfigTransformer.throwIfInvalid(schemaNew.convert());
+ConvertableSchema schemaOld = ConfigTransformer
+
.throwIfInvalid(SchemaLoader.loadConvertableSchemaFromYaml(new
ByteBufferInputStream(currentConfigScheme)));
+ConfigSchema configSchemaOld =
ConfigTransformer.throwIfInvalid(schemaOld.convert());
+
+
configSchemaNew.setNifiPropertiesOverrides(configSchemaOld.getNifiPropertiesOverrides());
+
+if (!overrideCoreProperties) {
+logger.debug("Preserving previous core properties...");
+
configSchemaNew.setCoreProperties(configSchemaOld.getCoreProperties());
+}
+
+if (!overrideSecurityProperties) {
+logger.debug("Preserving previous security properties...");
+
configSchemaNew.setSecurityProperties(configSchemaOld.getSecurityProperties());
+}
+
+StringWriter writer = new StringWriter();
+SchemaLoader.toYaml(configSchemaNew, writer);
+return
ByteBuffer.wrap(writer.toString().getBytes()).asReadOnlyBuffer();
+}
+} catch (Exception e) {
+logger.error("Loading the old and the new schema for merging was
not successful", e);
+throw e;
+}
+}
+
+private static boolean overrideSecurityProperties(Properties properties) {
+String overrideSecurityProperties = (String)
properties.getOrDefault(OVERRIDE_SECURITY, "false");
+boolean overrideSecurity;
+if ("true".equalsIgnoreCase(overrideSecurityProperties) ||
"false".equalsIgnoreCase(overrideSecurityProperties)) {
+overrideSecurity =
Boolean.parseBoolean(overrideSecurityProperties);
+} else {
+throw new IllegalArgumentException(
+"Property, " + OVERRIDE_SECURITY + ", to specify whether to
override security properties must either be a value boolean value (\"true\" or
\"false\")" +
+" or left to the default value of \"false\". It is set to
\"" + overrideSecurityProperties + "\".");
+}
Review Comment:
no reason, it was a leftover from the existing code. Changing both to simple
parseBoolean
##
minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java:
##
@@ -750,6 +756,72 @@ protected static void addTextElement(final Element
element, final String name, f
element.appendChild(toAdd);
}
+public static ByteBuffer overrideNonFlowSectionsFromOriginalSchema(byte[]
newSchema, ByteBuffer currentConfigScheme, Properties bootstrapProperties)
+throws IOException, InvalidConfigurationException,
SchemaLoaderException {
+try {
+boolean overrideCoreProperties =
ConfigTransformer.overrideCoreProperties(bootstrapProperties);
+boolean overrideSecurityProperties =
ConfigTransformer.overrideSecurityProperties(bootstrapProperties);
+if (overrideCoreProperties && overrideSecurityProperties) {
+return ByteBuffer.wrap(newSchema);
+} else {
+ConvertableSchema schemaNew = ConfigTransformer
+
.throwIfInvalid(SchemaLoader.loadConvertableSchemaFromYaml(new
ByteArrayInputStream(newSchema)));
+ConfigSchema configSchemaNew =
ConfigTransformer.throwIfInvalid(schemaNew.convert());
+ConvertableSchema schemaOld = ConfigTransformer
+
.throwIfInvalid(SchemaLoader.loadConvertableSchemaFromYaml(new
ByteBufferInputStream(currentConfigScheme)));
+ConfigSchema configSchemaOld =
[GitHub] [nifi] ferencerdei commented on a diff in pull request #6160: NIFI-10166 improve MiNiFi bootstrap test coverage
ferencerdei commented on code in PR #6160:
URL: https://github.com/apache/nifi/pull/6160#discussion_r911653320
##
minifi/minifi-bootstrap/src/test/java/org/apache/nifi/minifi/bootstrap/command/DumpRunnerTest.java:
##
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.command;
+
+import static org.apache.nifi.minifi.bootstrap.RunMiNiFi.CMD_LOGGER;
+import static org.apache.nifi.minifi.bootstrap.Status.ERROR;
+import static org.apache.nifi.minifi.bootstrap.Status.MINIFI_NOT_RUNNING;
+import static org.apache.nifi.minifi.bootstrap.Status.OK;
+import static org.apache.nifi.minifi.bootstrap.command.DumpRunner.DUMP_CMD;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.Mockito.verifyNoInteractions;
+import static org.mockito.Mockito.verifyNoMoreInteractions;
+import static org.mockito.Mockito.when;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.util.Optional;
+import org.apache.nifi.minifi.bootstrap.TestLogAppender;
+import org.apache.nifi.minifi.bootstrap.service.CurrentPortProvider;
+import org.apache.nifi.minifi.bootstrap.service.MiNiFiCommandSender;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.junit.jupiter.MockitoExtension;
+
+@ExtendWith(MockitoExtension.class)
+class DumpRunnerTest {
+
+private static final int MINIFI_PORT = 1337;
+private static final String DUMP_CONTENT = "dump_content";
+private static final TestLogAppender LOG_APPENDER = new TestLogAppender();
+
+@Mock
+private MiNiFiCommandSender miNiFiCommandSender;
+@Mock
+private CurrentPortProvider currentPortProvider;
+
+@InjectMocks
+private DumpRunner dumpRunner;
+
+@BeforeAll
+static void setupAll() {
+((ch.qos.logback.classic.Logger) CMD_LOGGER).addAppender(LOG_APPENDER);
+LOG_APPENDER.start();
+}
+
+@BeforeEach
+void setup() {
+LOG_APPENDER.reset();
+}
+
+@Test
+void testRunCommandShouldDumpToConsoleIfNoFileDefined() throws IOException
{
+when(currentPortProvider.getCurrentPort()).thenReturn(MINIFI_PORT);
+when(miNiFiCommandSender.sendCommand(DUMP_CMD,
MINIFI_PORT)).thenReturn(Optional.of(DUMP_CONTENT));
+
+int statusCode = dumpRunner.runCommand(new String[0]);
+
+assertEquals(OK.getStatusCode(), statusCode);
+assertEquals(DUMP_CONTENT,
LOG_APPENDER.getLastLoggedEvent().getMessage());
+verifyNoMoreInteractions(currentPortProvider, miNiFiCommandSender);
+}
+
+@Test
+void testRunCommandShouldDumpToFileIfItIsDefined() throws IOException {
+when(currentPortProvider.getCurrentPort()).thenReturn(MINIFI_PORT);
+when(miNiFiCommandSender.sendCommand(DUMP_CMD,
MINIFI_PORT)).thenReturn(Optional.of(DUMP_CONTENT));
+File file = Files.createTempFile(null, null).toFile();
+file.deleteOnExit();
+String tmpFilePath = file.getAbsolutePath();
+
+int statusCode = dumpRunner.runCommand(new String[] {DUMP_CMD,
tmpFilePath});
+
+assertEquals(OK.getStatusCode(), statusCode);
+assertEquals(DUMP_CONTENT, getDumpContent(file));
+assertEquals("Successfully wrote thread dump to " + tmpFilePath,
LOG_APPENDER.getLastLoggedEvent().getFormattedMessage());
Review Comment:
done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] ferencerdei commented on a diff in pull request #6160: NIFI-10166 improve MiNiFi bootstrap test coverage
ferencerdei commented on code in PR #6160:
URL: https://github.com/apache/nifi/pull/6160#discussion_r911779208
##
minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/configuration/ingestors/PullHttpChangeIngestor.java:
##
@@ -234,8 +218,9 @@ public void run() {
.build();
final Request.Builder requestBuilder = new Request.Builder()
-.get()
-.url(url);
+.get()
+.cacheControl(CacheControl.FORCE_NETWORK)
Review Comment:
no, removed it.
##
minifi/minifi-bootstrap/src/main/java/org/apache/nifi/minifi/bootstrap/util/ConfigTransformer.java:
##
@@ -750,6 +756,72 @@ protected static void addTextElement(final Element
element, final String name, f
element.appendChild(toAdd);
}
+public static ByteBuffer overrideNonFlowSectionsFromOriginalSchema(byte[]
newSchema, ByteBuffer currentConfigScheme, Properties bootstrapProperties)
+throws IOException, InvalidConfigurationException,
SchemaLoaderException {
+try {
+boolean overrideCoreProperties =
ConfigTransformer.overrideCoreProperties(bootstrapProperties);
+boolean overrideSecurityProperties =
ConfigTransformer.overrideSecurityProperties(bootstrapProperties);
+if (overrideCoreProperties && overrideSecurityProperties) {
+return ByteBuffer.wrap(newSchema);
+} else {
+ConvertableSchema schemaNew = ConfigTransformer
+
.throwIfInvalid(SchemaLoader.loadConvertableSchemaFromYaml(new
ByteArrayInputStream(newSchema)));
+ConfigSchema configSchemaNew =
ConfigTransformer.throwIfInvalid(schemaNew.convert());
+ConvertableSchema schemaOld = ConfigTransformer
+
.throwIfInvalid(SchemaLoader.loadConvertableSchemaFromYaml(new
ByteBufferInputStream(currentConfigScheme)));
+ConfigSchema configSchemaOld =
ConfigTransformer.throwIfInvalid(schemaOld.convert());
+
+
configSchemaNew.setNifiPropertiesOverrides(configSchemaOld.getNifiPropertiesOverrides());
+
+if (!overrideCoreProperties) {
+logger.debug("Preserving previous core properties...");
+
configSchemaNew.setCoreProperties(configSchemaOld.getCoreProperties());
+}
+
+if (!overrideSecurityProperties) {
+logger.debug("Preserving previous security properties...");
+
configSchemaNew.setSecurityProperties(configSchemaOld.getSecurityProperties());
+}
+
+StringWriter writer = new StringWriter();
+SchemaLoader.toYaml(configSchemaNew, writer);
+return
ByteBuffer.wrap(writer.toString().getBytes()).asReadOnlyBuffer();
+}
+} catch (Exception e) {
+logger.error("Loading the old and the new schema for merging was
not successful", e);
+throw e;
Review Comment:
Wrapped it now.
##
minifi/minifi-bootstrap/src/test/java/org/apache/nifi/minifi/bootstrap/command/CommandRunnerFactoryTest.java:
##
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.command;
+
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.DUMP;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.ENV;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.FLOWSTATUS;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.RESTART;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.RUN;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.START;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.STATUS;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.STOP;
+import static org.apache.nifi.minifi.bootstrap.BootstrapCommand.UNKNOWN;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.verifyNoInteractions;
+
+import java.io.File;
+import
[GitHub] [nifi] ferencerdei commented on a diff in pull request #6160: NIFI-10166 improve MiNiFi bootstrap test coverage
ferencerdei commented on code in PR #6160:
URL: https://github.com/apache/nifi/pull/6160#discussion_r911657359
##
minifi/minifi-bootstrap/src/test/java/org/apache/nifi/minifi/bootstrap/service/BootstrapCodecTest.java:
##
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.minifi.bootstrap.service;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.verifyNoInteractions;
+import static org.mockito.Mockito.when;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.stream.Stream;
+import org.apache.nifi.minifi.bootstrap.RunMiNiFi;
+import
org.apache.nifi.minifi.bootstrap.configuration.ConfigurationChangeCoordinator;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+class BootstrapCodecTest {
+
+private static final int VALID_PORT = 1;
+private static final String SECRET = "secret";
+private static final String OK = "OK";
+private static final String EMPTY_STRING = "";
+private RunMiNiFi runner;
+
+@BeforeEach
+void setup() {
+runner = mock(RunMiNiFi.class);
+}
+
+@Test
+void testCommunicateShouldThrowIOExceptionIfThereIsNoCommand() {
+InputStream inputStream = new ByteArrayInputStream(new byte[0]);
+ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
+BootstrapCodec bootstrapCodec = new BootstrapCodec(runner,
inputStream, outputStream);
+
+IOException expectedException = assertThrows(IOException.class,
bootstrapCodec::communicate);
+assertEquals("Received invalid command from MiNiFi: null",
expectedException.getMessage());
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
