[jira] [Commented] (MINIFI-521) minifi-toolkit-0.5.0: "Connection with id ... has invalid destination id" confused by targetId?
[
https://issues.apache.org/jira/browse/MINIFI-521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17077308#comment-17077308
]
Javi Roman commented on MINIFI-521:
---
[~aldrin] I've just tested your PR
[https://github.com/apache/nifi-minifi/pull/185] and everything works fine for
me against NiFi 1.11.4 release.
{noformat}
[vagrant@edge ~]$ minifi-toolkit-0.6.0-SNAPSHOT/bin/config.sh transform
Edge_Node_Flow_Tail_File.xml config.yml
Java home: /usr/lib/jvm/java
MiNiFi Toolkit home: /home/vagrant/minifi-toolkit-0.6.0-SNAPSHOT
No validation errors found in converted configuration.
{noformat}
The [https://nifi.apache.org/minifi/getting-started.html] guide is working fine
with your minifi-tookit patch. With the current toolkit (v0.5.0) I'm getting
the error explained in this issue.
> minifi-toolkit-0.5.0: "Connection with id ... has invalid destination id"
> confused by targetId?
> ---
>
> Key: MINIFI-521
> URL: https://issues.apache.org/jira/browse/MINIFI-521
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Components: Agent Configuration/Installation
>Affects Versions: 0.5.0
> Environment: version: '3'
> services:
> nifi:
> image: apache/nifi:1:10.0
> minifi:
> image: apache/nifi-minifi:0.5.0
> minifi-toolkit-0.5.0, Windows
>Reporter: Sebastian Napiorkowski
>Assignee: Aldrin Piri
>Priority: Minor
> Attachments: MiNiFi-buggy.xml, MiNiFi-expected.xml, config-buggy.yml,
> config-expected.yml, config.yml, minifi-flow.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I followed this tutorial: [Getting started with
> MiNiFi|https://nifi.apache.org/minifi/getting-started.html]
> And the problem was that, while transforming the Template to the yml-file, it
> failed with: "Connection with id 10de7342-41ef-37a5-- has
> invalid destination id 9b02eb45-3c2d-3fb4-91cf-880d5038c0cd"
> {code:java}
> minifi-toolkit-0.5.0\bin\config.bat transform MiNiFi-buggy.xml
> config-buggy.yml
> There are validation errors with the template, still outputting YAML but it
> will need to be edited.
> Connection with id 10de7342-41ef-37a5-- has invalid
> destination id 9b02eb45-3c2d-3fb4-91cf-880d5038c0cd
> {code}
> By the help of brosander@github (Thank you, mate!) we discovered that the
> tool is confused by the targetId property:
> {code:xml}
>
>
>
>
>
> 1
> true
> true
>
> db4d2e20-016e-1000-d3c7-73a67e2b1d67
> 9b02eb45-3c2d-3fb4-91cf-880d5038c0cd
> input_minifi
>
> dadd7f29-016e-1000-3c91-89495c813674
> false
> false
> false
>
>
> {code}
> just by removing the {{targetId}}, the toolkit runs through:
> {code:java}
> minifi-toolkit-0.5.0\bin\config.bat transform MiNiFi-expected.xml
> config-expected.yml
> No validation errors found in converted configuration.
> {code}
> The produced files differ:
> {code:java}
> diff config-buggy.yml config-expected.yml
> 101c101
> < - id: dadd7f29-016e-1000-3c91-89495c813674
> ---
> > - id: 9b02eb45-3c2d-3fb4-91cf-880d5038c0cd
> {code}
> -I tested it and NiFi discovers MiNiFi, everything seems fine.-
> EDIT:
> I celebrated to early. Removing the {{targetId}} is *not* a working
> workaround. The easiest way to get around this issue is to downgrade NiFi
> from 1.10 to 1.9.2.
> Originally I thought this is a MiNiFi issue, but now I'm thinking that
> something changed in NiFi from 1.9.2 to 1.10.0 when exporting Templates. The
> ids are mixed up. This could be a much more critical bug,
svn commit: r1876243 - /nifi/site/trunk/registry.html
Author: aboda Date: Tue Apr 7 13:52:54 2020 New Revision: 1876243 URL: http://svn.apache.org/viewvc?rev=1876243=rev Log: Added NiFi-Registry 0.6.0 download links to site Modified: nifi/site/trunk/registry.html Modified: nifi/site/trunk/registry.html URL: http://svn.apache.org/viewvc/nifi/site/trunk/registry.html?rev=1876243=1876242=1876243=diff == --- nifi/site/trunk/registry.html (original) +++ nifi/site/trunk/registry.html Tue Apr 7 13:52:54 2020 @@ -170,6 +170,35 @@ + 0.6.0 + + + Sources + + https://www.apache.org/dyn/closer.lua?path=/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-source-release.zip;>nifi-registry-0.6.0-source-release.zip ( + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-source-release.zip.asc;>asc, + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-source-release.zip.sha256;>sha256, + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-source-release.zip.sha512;>sha512 ) + + + + Binaries + + https://www.apache.org/dyn/closer.lua?path=/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.tar.gz;>nifi-registry-0.6.0-bin.tar.gz ( + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.tar.gz.asc;>asc, + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.tar.gz.sha256;>sha256, + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.tar.gz.sha512;>sha512 ) + + https://www.apache.org/dyn/closer.lua?path=/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.zip;>nifi-registry-0.6.0-bin.zip ( + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.zip.asc;>asc, + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.zip.sha256;>sha256, + https://downloads.apache.org/nifi/nifi-registry/nifi-registry-0.6.0/nifi-registry-0.6.0-bin.zip.sha512;>sha512 ) + + + https://cwiki.apache.org/confluence/display/NIFIREG/Release+Notes#ReleaseNotes-NiFiRegistry0.6.0;>Release Notes + + + 0.5.0
[nifi-registry] annotated tag rel/nifi-registry-0.6.0 updated (4ddfde5 -> e1b0617)
This is an automated email from the ASF dual-hosted git repository. aboda pushed a change to annotated tag rel/nifi-registry-0.6.0 in repository https://gitbox.apache.org/repos/asf/nifi-registry.git. *** WARNING: tag rel/nifi-registry-0.6.0 was modified! *** from 4ddfde5 (commit) to e1b0617 (tag) tagging 4ddfde57f0163baa08eb782fc8be9c51d8f58af7 (commit) replaces rel/nifi-registry-0.5.0 by Arpad Boda on Tue Apr 7 15:09:34 2020 +0200 - Log - NIFIREG-371 Signed release tag for approved release of NiFi-Registry 0.6.0 -BEGIN PGP SIGNATURE- iQFFBAABCgAvFiEE7ni7ioKnhRR3qq0KOQwbWt6XiDUFAl6Me44RHGFib2RhQGFw YWNoZS5vcmcACgkQOQwbWt6XiDVZTggAoY2mlC28my2ev3zb1aMJKiQrpEdHrP73 u3lzLzIHDApYLGId7a0rHsZ92IEWnDbfCqe4Zbe+/ugDjfDM0qEZPDNuQzT/EClj VNeT6Yn3RrImdpDOO3jHNCiLpc4I9l4R9+/+gYzM99rmM/1tVM12F8aDjoiN/VOH q8x8YSG8qTcKlbxx6kniFyndqr+gcHf/9m0HhTOuz4mI5kRY4/jII2wmdxYa3OXx uaTjykgxjNGXGoOdO0coi+JpM70yUrWObUeIze2sLMHutET+EdHc8WvaHdrxRyUN DHrWrjiZCsn/QBy5YwqcCPl/caADYcR/h+rH3fi96rO6hy4fMa07+g== =j79a -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[nifi-minifi-cpp] branch master updated: MINIFICPP-1189 win/debug: expect correct yaml lib
This is an automated email from the ASF dual-hosted git repository.
aboda pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
The following commit(s) were added to refs/heads/master by this push:
new 90e6cda MINIFICPP-1189 win/debug: expect correct yaml lib
90e6cda is described below
commit 90e6cda97b66944cb2abf722c443e58dd237c208
Author: Marton Szasz
AuthorDate: Tue Apr 7 03:58:50 2020 -0400
MINIFICPP-1189 win/debug: expect correct yaml lib
Signed-off-by: Arpad Boda
This closes #751
---
cmake/BundledYamlCpp.cmake | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/cmake/BundledYamlCpp.cmake b/cmake/BundledYamlCpp.cmake
index 0c9a3f4..f486681 100644
--- a/cmake/BundledYamlCpp.cmake
+++ b/cmake/BundledYamlCpp.cmake
@@ -18,7 +18,11 @@
function(use_bundled_yamlcpp SOURCE_DIR BINARY_DIR)
# Define byproducts
if (WIN32)
-set(BYPRODUCT "lib/libyaml-cppmd.lib")
+if ("${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
+set(BYPRODUCT "lib/libyaml-cppmdd.lib")
+else()
+set(BYPRODUCT "lib/libyaml-cppmd.lib")
+endif()
else()
set(BYPRODUCT "lib/libyaml-cpp.a")
endif()
[nifi] branch master updated: NIFI-7314 HandleHttpRequest stops Jetty in OnUnscheduled instead of OnStopped. Also reject pending request and clean their queue when shutting down.
This is an automated email from the ASF dual-hosted git repository.
turcsanyi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/master by this push:
new 09cece8 NIFI-7314 HandleHttpRequest stops Jetty in OnUnscheduled
instead of OnStopped. Also reject pending request and clean their queue when
shutting down.
09cece8 is described below
commit 09cece8e9936f7d8e6662e6f887444e85a36f79f
Author: Tamas Palfy
AuthorDate: Tue Apr 7 15:35:20 2020 +0200
NIFI-7314 HandleHttpRequest stops Jetty in OnUnscheduled instead of
OnStopped. Also reject pending request and clean their queue when shutting down.
NIFI-7314 In HandleHttpRequest returning 503 when rejecting pending
requests before shutdown.
NIFI-7314 In HandleHttpRequest add logs and better response message during
cleanup.
This closes #4191.
Signed-off-by: Peter Turcsanyi
---
.../processors/standard/HandleHttpRequest.java | 46 +-
.../standard/ITestHandleHttpRequest.java | 567 +
2 files changed, 400 insertions(+), 213 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java
index 3717a33..66b940a 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java
@@ -25,7 +25,7 @@ import
org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
-import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.annotation.lifecycle.OnUnscheduled;
import org.apache.nifi.annotation.notification.OnPrimaryNodeStateChange;
import org.apache.nifi.annotation.notification.PrimaryNodeState;
import org.apache.nifi.components.AllowableValue;
@@ -304,6 +304,7 @@ public class HandleHttpRequest extends AbstractProcessor {
}
private volatile Server server;
+private volatile boolean ready;
private AtomicBoolean initialized = new AtomicBoolean(false);
private volatile BlockingQueue containerQueue;
private AtomicBoolean runOnPrimary = new AtomicBoolean(false);
@@ -323,7 +324,7 @@ public class HandleHttpRequest extends AbstractProcessor {
initialized.set(false);
}
-private synchronized void initializeServer(final ProcessContext context)
throws Exception {
+synchronized void initializeServer(final ProcessContext context) throws
Exception {
if(initialized.get()){
return;
}
@@ -461,6 +462,12 @@ public class HandleHttpRequest extends AbstractProcessor {
response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE, "Processor queue
is full");
return;
+} else if (!ready) {
+getLogger().warn("Request from {} cannot be processed,
processor is being shut down; responding with SERVICE_UNAVAILABLE",
+new Object[]{request.getRemoteAddr()});
+
+
response.sendError(HttpServletResponse.SC_SERVICE_UNAVAILABLE, "Processor is
shutting down");
+return;
}
// Right now, that information, though, is only in the
ProcessSession, not the ProcessContext,
@@ -491,6 +498,7 @@ public class HandleHttpRequest extends AbstractProcessor {
getLogger().info("Server started and listening on port " + getPort());
initialized.set(true);
+ready = true;
}
protected int getPort() {
@@ -535,10 +543,13 @@ public class HandleHttpRequest extends AbstractProcessor {
return sslFactory;
}
-@OnStopped
+@OnUnscheduled
public void shutdown() throws Exception {
+ready = false;
+
if (server != null) {
getLogger().debug("Shutting down server");
+rejectPendingRequests();
server.stop();
server.destroy();
server.join();
@@ -547,6 +558,35 @@ public class HandleHttpRequest extends AbstractProcessor {
}
}
+void rejectPendingRequests() {
+HttpRequestContainer container;
+while ((container = getNextContainer()) != null) {
+try {
+getLogger().warn("Rejecting request from {} during cleanup
after processor shutdown; responding with SERVICE_UNAVAILABLE",
+new Object[]{container.getRequest().getRemoteAddr()});
+
+
[nifi-registry] branch NIFIREG-380 updated (eae3a86 -> e176bf0)
This is an automated email from the ASF dual-hosted git repository. thenatog pushed a change to branch NIFIREG-380 in repository https://gitbox.apache.org/repos/asf/nifi-registry.git. discard eae3a86 NIFIREG-380 - Allow NiFi Registry admins to configure whether Jetty will send the Jetty server version in responses. add 6d4c119 NIFIREG-381 skipping integration tests NIFIREG-381 adding build info for Github Actions to Readme add e176bf0 NIFIREG-380 - Allow NiFi Registry admins to configure whether Jetty will send the Jetty server version in responses. This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (eae3a86) \ N -- N -- N refs/heads/NIFIREG-380 (e176bf0) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .github/workflows/ci-workflow.yml | 4 ++-- README.md | 4 +++- 2 files changed, 5 insertions(+), 3 deletions(-)
[nifi-registry] branch master updated: NIFIREG-380 - Allow NiFi Registry admins to configure whether Jetty will send the Jetty server version in responses. NIFIREG-380 Added new property to default ni
This is an automated email from the ASF dual-hosted git repository.
alopresto pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nifi-registry.git
The following commit(s) were added to refs/heads/master by this push:
new 56a44c8 NIFIREG-380 - Allow NiFi Registry admins to configure whether
Jetty will send the Jetty server version in responses. NIFIREG-380 Added new
property to default nifi-registry.properties.
56a44c8 is described below
commit 56a44c8528ec6f420b4b9d1c7b402da8417f518d
Author: Nathan Gough
AuthorDate: Mon Apr 6 21:10:25 2020 -0400
NIFIREG-380 - Allow NiFi Registry admins to configure whether Jetty will
send the Jetty server version in responses.
NIFIREG-380 Added new property to default nifi-registry.properties.
This closes #273.
Signed-off-by: Andy LoPresto
---
nifi-registry-assembly/pom.xml | 1 +
.../src/main/java/org/apache/nifi/registry/jetty/JettyServer.java | 1 +
.../org/apache/nifi/registry/properties/NiFiRegistryProperties.java | 6 ++
.../src/main/resources/conf/nifi-registry.properties| 1 +
4 files changed, 9 insertions(+)
diff --git a/nifi-registry-assembly/pom.xml b/nifi-registry-assembly/pom.xml
index 655964c..b4c7acd 100644
--- a/nifi-registry-assembly/pom.xml
+++ b/nifi-registry-assembly/pom.xml
@@ -152,6 +152,7 @@
./work/jetty
200
+
true
diff --git
a/nifi-registry-core/nifi-registry-jetty/src/main/java/org/apache/nifi/registry/jetty/JettyServer.java
b/nifi-registry-core/nifi-registry-jetty/src/main/java/org/apache/nifi/registry/jetty/JettyServer.java
index 387857f..d20fce4 100644
---
a/nifi-registry-core/nifi-registry-jetty/src/main/java/org/apache/nifi/registry/jetty/JettyServer.java
+++
b/nifi-registry-core/nifi-registry-jetty/src/main/java/org/apache/nifi/registry/jetty/JettyServer.java
@@ -148,6 +148,7 @@ public class JettyServer {
final HttpConfiguration httpConfiguration = new HttpConfiguration();
httpConfiguration.setRequestHeaderSize(HEADER_BUFFER_SIZE);
httpConfiguration.setResponseHeaderSize(HEADER_BUFFER_SIZE);
+
httpConfiguration.setSendServerVersion(properties.shouldSendServerVersion());
if (properties.getPort() != null) {
final Integer port = properties.getPort();
diff --git
a/nifi-registry-core/nifi-registry-properties/src/main/java/org/apache/nifi/registry/properties/NiFiRegistryProperties.java
b/nifi-registry-core/nifi-registry-properties/src/main/java/org/apache/nifi/registry/properties/NiFiRegistryProperties.java
index e1e9a39..d3b4a25 100644
---
a/nifi-registry-core/nifi-registry-properties/src/main/java/org/apache/nifi/registry/properties/NiFiRegistryProperties.java
+++
b/nifi-registry-core/nifi-registry-properties/src/main/java/org/apache/nifi/registry/properties/NiFiRegistryProperties.java
@@ -38,6 +38,7 @@ public class NiFiRegistryProperties extends Properties {
public static final String WEB_HTTPS_HOST = "nifi.registry.web.https.host";
public static final String WEB_WORKING_DIR =
"nifi.registry.web.jetty.working.directory";
public static final String WEB_THREADS = "nifi.registry.web.jetty.threads";
+public static final String WEB_SHOULD_SEND_SERVER_VERSION =
"nifi.registry.web.should.send.server.version";
public static final String SECURITY_KEYSTORE =
"nifi.registry.security.keystore";
public static final String SECURITY_KEYSTORE_TYPE =
"nifi.registry.security.keystoreType";
@@ -95,6 +96,7 @@ public class NiFiRegistryProperties extends Properties {
public static final String
DEFAULT_SECURITY_IDENTITY_PROVIDER_CONFIGURATION_FILE =
"./conf/identity-providers.xml";
public static final String DEFAULT_AUTHENTICATION_EXPIRATION = "12 hours";
public static final String DEFAULT_EXTENSIONS_WORKING_DIR =
"./work/extensions";
+public static final String DEFAULT_WEB_SHOULD_SEND_SERVER_VERSION = "true";
public int getWebThreads() {
int webThreads = 200;
@@ -122,6 +124,10 @@ public class NiFiRegistryProperties extends Properties {
return getSslPort() != null;
}
+public boolean shouldSendServerVersion() {
+return
Boolean.parseBoolean(getProperty(WEB_SHOULD_SEND_SERVER_VERSION,
DEFAULT_WEB_SHOULD_SEND_SERVER_VERSION));
+}
+
public String getHttpsHost() {
return getProperty(WEB_HTTPS_HOST);
}
diff --git
a/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/nifi-registry.properties
b/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/nifi-registry.properties
index 2341f38..1b62023 100644
---
a/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/nifi-registry.properties
+++
b/nifi-registry-core/nifi-registry-resources/src/main/resources/conf/nifi-registry.properties
@@ -21,6 +21,7 @@
