This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 056ef41c088a CAMEL-27211: camel-tahu - each tahu-host consumer builds
its own host application (#30467)
056ef41c088a is described below
commit 056ef41c088ac5b17648858225eb77b7c8104f89
Author: allthingssecurity <[email protected]>
AuthorDate: Fri Nov 20 14:07:15 2026 +0530
CAMEL-27211: camel-tahu - each tahu-host consumer builds its own host
application (#30467)
TahuHostConsumer took its TahuHostApplication from a static map keyed by
the host id, built for the first consumer of that host id in the JVM and
never removed. That host application gives the Sparkplug messages to the
consumer that built it and connects with that consumer's servers and
credentials. A later consumer of the same host id (a route reloaded or
removed and added again, or a route of another CamelContext in the same
JVM) started the old host application again, and every message went to
the old, stopped consumer: the new route received nothing.
- Each consumer builds its own host application; the static map is gone.
- TahuHostApplication gives out its event handler, and the consumer its
host application (package-private), for the test.
- TahuHostConsumerHostApplicationTest: a consumer created again for the
same host id, and consumers of two CamelContexts, each get their own
messages (no MQTT server: the consumers are only initialized).
camel-tahu - one SequenceMonitorTimer thread for all host applications, and
one consumer per JVM documented
Review follow-up (davsclaus):
- HostApplicationBuilder.build() stops the timer of Tahu's
SequenceReorderManager singleton before it builds the host application:
the HostApplication constructor starts a new SequenceMonitorTimer thread
without cancelling the one before, so with a host application per
consumer every consumer created left one more thread running. Builds are
serialized on a lock, as they share that timer.
- Two consumers that run at the same time in one JVM are not supported:
Tahu gives the Sparkplug messages of all host applications to the event
handler of the one built last. The consumer comment says so, and the
tahu-host docs (and their catalog copy) document it.
- TahuHostConsumerHostApplicationTest: the test that called the event
handler directly is replaced by one through Tahu's real dispatch with
FakeMqttBroker (route removed and added again, an NBIRTH reaches the new
route), and one that checks a single live SequenceMonitorTimer thread
after five consumers are created. JUnit assertions, no Hamcrest.
- The test accessors getEventHandler() and getTahuHostApplication() are
removed.
- TahuHostConsumerMessageTest: the comment about the static map is removed.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../camel/catalog/docs/tahu-host-component.adoc | 10 ++
.../src/main/docs/tahu-host-component.adoc | 10 ++
.../camel/component/tahu/TahuHostConsumer.java | 30 +++--
.../tahu/handlers/TahuHostApplication.java | 16 ++-
.../tahu/TahuHostConsumerHostApplicationTest.java | 124 +++++++++++++++++++++
.../tahu/TahuHostConsumerMessageTest.java | 1 -
6 files changed, 170 insertions(+), 21 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/tahu-host-component.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/tahu-host-component.adoc
index e4cfcc9bb781..9faa99ca1570 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/tahu-host-component.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/tahu-host-component.adoc
@@ -63,3 +63,13 @@ for each data message:
- log:
message: "Temperature:
${header[CamelTahuMetric.temperature].value}"
----
+
+=== One Host Application consumer per JVM
+
+Define only one `tahu-host` route in a JVM, also across CamelContexts, and
including routes that are not started
+(`autoStartup=false`): their consumer is created when the route is set up.
Eclipse Tahu hands the Sparkplug messages of
+every Host Application in the JVM to one shared `SequenceReorderManager`,
which gives them to the Host Application
+created last. With two consumers, all the Sparkplug messages the two receive
go to the consumer created last, and the
+other one gets none of them.
+
+A route that is restarted, reloaded, or removed and added again is fine: its
new consumer is the one created last.
diff --git a/components/camel-tahu/src/main/docs/tahu-host-component.adoc
b/components/camel-tahu/src/main/docs/tahu-host-component.adoc
index e4cfcc9bb781..9faa99ca1570 100644
--- a/components/camel-tahu/src/main/docs/tahu-host-component.adoc
+++ b/components/camel-tahu/src/main/docs/tahu-host-component.adoc
@@ -63,3 +63,13 @@ for each data message:
- log:
message: "Temperature:
${header[CamelTahuMetric.temperature].value}"
----
+
+=== One Host Application consumer per JVM
+
+Define only one `tahu-host` route in a JVM, also across CamelContexts, and
including routes that are not started
+(`autoStartup=false`): their consumer is created when the route is set up.
Eclipse Tahu hands the Sparkplug messages of
+every Host Application in the JVM to one shared `SequenceReorderManager`,
which gives them to the Host Application
+created last. With two consumers, all the Sparkplug messages the two receive
go to the consumer created last, and the
+other one gets none of them.
+
+A route that is restarted, reloaded, or removed and added again is fine: its
new consumer is the one created last.
diff --git
a/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/TahuHostConsumer.java
b/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/TahuHostConsumer.java
index 74c64d041cf0..81fd7ab4e729 100644
---
a/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/TahuHostConsumer.java
+++
b/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/TahuHostConsumer.java
@@ -18,8 +18,6 @@ package org.apache.camel.component.tahu;
import java.util.List;
import java.util.UUID;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ConcurrentMap;
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
@@ -44,8 +42,6 @@ public class TahuHostConsumer extends DefaultConsumer {
private static final Logger LOG =
LoggerFactory.getLogger(TahuHostConsumer.class);
- private static final ConcurrentMap<String, TahuHostApplication>
hostHandlers = new ConcurrentHashMap<>();
-
@SuppressWarnings("unused")
private final TahuDefaultEndpoint endpoint;
@@ -62,19 +58,19 @@ public class TahuHostConsumer extends DefaultConsumer {
TahuConfiguration configuration = endpoint.getConfiguration();
- tahuHostApplication = hostHandlers.computeIfAbsent(hostId, hId -> {
- List<MqttServerDefinition> serverDefinitions =
configuration.getServerDefinitionList();
-
- TahuHostApplication thah = new
TahuHostApplication.HostApplicationBuilder()
- .hostId(hId)
- .serverDefinitions(serverDefinitions)
- .onMessageConsumer(this::onMessageConsumer)
- .onMetricConsumer(this::onMetricConsumer)
- .build();
-
- return thah;
- });
-
+ // the host application of this consumer: it gives the messages to
this consumer, and connects with the servers
+ // and credentials of this endpoint. It is not shared with the
consumer created again for the same host id when
+ // the route is restarted, reloaded or removed and added again, as it
would keep giving the messages to the
+ // consumer that created it, which is stopped by then. Two consumers
in one JVM are still not supported, even
+ // when one of them is not started: Tahu gives the Sparkplug messages
of all host applications to the event
+ // handler of the host application built last (its
SequenceReorderManager is a singleton)
+ List<MqttServerDefinition> serverDefinitions =
configuration.getServerDefinitionList();
+ tahuHostApplication = new TahuHostApplication.HostApplicationBuilder()
+ .hostId(hostId)
+ .serverDefinitions(serverDefinitions)
+ .onMessageConsumer(this::onMessageConsumer)
+ .onMetricConsumer(this::onMetricConsumer)
+ .build();
}
@Override
diff --git
a/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/handlers/TahuHostApplication.java
b/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/handlers/TahuHostApplication.java
index ef7b2af3f441..c92739e1a8e6 100644
---
a/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/handlers/TahuHostApplication.java
+++
b/components/camel-tahu/src/main/java/org/apache/camel/component/tahu/handlers/TahuHostApplication.java
@@ -21,6 +21,7 @@ import java.util.function.BiConsumer;
import org.eclipse.tahu.host.HostApplication;
import org.eclipse.tahu.host.api.MultiHostApplicationEventHandler;
+import org.eclipse.tahu.host.seq.SequenceReorderManager;
import org.eclipse.tahu.message.PayloadDecoder;
import org.eclipse.tahu.message.SparkplugBPayloadDecoder;
import org.eclipse.tahu.message.model.EdgeNodeDescriptor;
@@ -45,6 +46,9 @@ public class TahuHostApplication extends HostApplication {
public static final class HostApplicationBuilder {
+ // the host applications are built one at a time, as they share the
timer of Tahu's SequenceReorderManager
+ private static final Object BUILD_LOCK = new Object();
+
private String hostId;
private List<String> sparkplugSubscriptions =
List.of(SparkplugMeta.SPARKPLUG_B_TOPIC_PREFIX + "/#");
private List<MqttServerDefinition> serverDefinitions;
@@ -110,9 +114,15 @@ public class TahuHostApplication extends HostApplication {
MultiHostApplicationEventHandler eventHandler
= new
MultiTahuHostApplicationEventHandler(onMessageConsumer, onMetricConsumer);
- cachedTahuHostApplication = tahuHostApplication = new
TahuHostApplication(
- eventHandler, hostId, sparkplugSubscriptions,
serverDefinitions, randomStartupDelay, payloadDecoder,
- onlineState);
+ synchronized (BUILD_LOCK) {
+ // Tahu's HostApplication constructor starts a new
SequenceMonitorTimer thread on the
+ // SequenceReorderManager singleton without cancelling the
one started before: stop it first, so
+ // that only one runs however many host applications are
built (each consumer builds its own)
+ SequenceReorderManager.getInstance().stop();
+ cachedTahuHostApplication = tahuHostApplication = new
TahuHostApplication(
+ eventHandler, hostId, sparkplugSubscriptions,
serverDefinitions, randomStartupDelay,
+ payloadDecoder, onlineState);
+ }
}
return cachedTahuHostApplication;
diff --git
a/components/camel-tahu/src/test/java/org/apache/camel/component/tahu/TahuHostConsumerHostApplicationTest.java
b/components/camel-tahu/src/test/java/org/apache/camel/component/tahu/TahuHostConsumerHostApplicationTest.java
new file mode 100644
index 000000000000..7a29606c4aeb
--- /dev/null
+++
b/components/camel-tahu/src/test/java/org/apache/camel/component/tahu/TahuHostConsumerHostApplicationTest.java
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.tahu;
+
+import java.util.Date;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.Consumer;
+import org.apache.camel.Endpoint;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.impl.DefaultCamelContext;
+import org.apache.camel.support.service.ServiceHelper;
+import org.eclipse.tahu.message.SparkplugBPayloadEncoder;
+import org.eclipse.tahu.message.model.Metric.MetricBuilder;
+import org.eclipse.tahu.message.model.MetricDataType;
+import org.eclipse.tahu.message.model.SparkplugBPayload;
+import org.eclipse.tahu.message.model.SparkplugMeta;
+import org.junit.jupiter.api.Test;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * A tahu-host consumer created again for the same host id (the route is
restarted, reloaded or removed and added again)
+ * gets the Sparkplug messages through its own host application, and building
a host application for each consumer
+ * leaves one SequenceMonitorTimer thread of Tahu running.
+ */
+class TahuHostConsumerHostApplicationTest {
+
+ private static final String SPARKPLUG_SUBSCRIPTION = "spBv1.0/#";
+
+ @Test
+ void routeAddedAgainGetsTheMessages() throws Exception {
+ try (FakeMqttBroker broker = new FakeMqttBroker(); CamelContext
context = new DefaultCamelContext()) {
+ String from = "tahu-host:AgainHost" + broker.getPort() +
"?clientId=AgainHostClient&servers=Fake:tcp://localhost:"
+ + broker.getPort();
+ context.addRoutes(route(from, "mock:a"));
+ context.start();
+ await().atMost(20, TimeUnit.SECONDS).until(() ->
sparkplugSubscriptions(broker) == 1);
+
+ // the route is removed and added again with the same endpoint: a
new consumer is created
+ context.getRouteController().stopRoute("host");
+ context.removeRoute("host");
+ context.addRoutes(route(from, "mock:b"));
+ await().atMost(20, TimeUnit.SECONDS).until(() ->
sparkplugSubscriptions(broker) >= 2);
+
+ MockEndpoint b = context.getEndpoint("mock:b", MockEndpoint.class);
+ b.expectedMessageCount(1);
+ b.expectedHeaderReceived(TahuConstants.MESSAGE_TYPE, "NBIRTH");
+ // wait a while after the message for one to the old route too
+ b.setAssertPeriod(500);
+ broker.publish("spBv1.0/AgainGroup/NBIRTH/AgainNode", nbirth());
+
+ // without the fix the new consumer took the host application of
the old one from a static map: Tahu gave
+ // the message to the old consumer (or rejected it, as that host
application was shut down)
+ MockEndpoint.assertIsSatisfied(context, 20, TimeUnit.SECONDS);
+ assertEquals(0, context.getEndpoint("mock:a",
MockEndpoint.class).getReceivedCounter());
+ }
+ }
+
+ @Test
+ void oneSequenceMonitorTimerThread() throws Exception {
+ try (CamelContext context = new DefaultCamelContext()) {
+ context.start();
+ Endpoint endpoint
+ =
context.getEndpoint("tahu-host:TimerHost?clientId=TimerHostClient&servers=Mqtt:tcp://localhost:1883");
+ for (int i = 0; i < 5; i++) {
+ // a consumer builds its host application when it is created
(initialized only: starting would connect)
+ Consumer consumer = endpoint.createConsumer(exchange -> {
+ });
+ ServiceHelper.initService(consumer);
+ }
+
+ // without the fix every host application started a timer thread
of its own, and none of them was stopped
+ // (a timer that is cancelled ends its thread right after)
+ await().atMost(20, TimeUnit.SECONDS).until(() ->
sequenceMonitorTimerThreads() == 1);
+ }
+ }
+
+ private static RouteBuilder route(String from, String to) {
+ return new RouteBuilder() {
+ @Override
+ public void configure() {
+ from(from).routeId("host")
+ // the exchanges of the messages, not those of each
metric
+ .filter(header(TahuConstants.MESSAGE_TYPE).isNotNull())
+ .to(to);
+ }
+ };
+ }
+
+ private static long sparkplugSubscriptions(FakeMqttBroker broker) {
+ return
broker.getSubscriptions().stream().filter(SPARKPLUG_SUBSCRIPTION::equals).count();
+ }
+
+ private static long sequenceMonitorTimerThreads() {
+ return Thread.getAllStackTraces().keySet().stream()
+ .filter(t -> t.isAlive() &&
"SequenceMonitorTimer".equals(t.getName())).count();
+ }
+
+ private static byte[] nbirth() throws Exception {
+ SparkplugBPayload payload = new
SparkplugBPayload.SparkplugBPayloadBuilder().setTimestamp(new Date()).setSeq(0L)
+ .addMetric(new
MetricBuilder(SparkplugMeta.SPARKPLUG_BD_SEQUENCE_NUMBER_KEY,
MetricDataType.Int64, 0L)
+ .createMetric())
+ .createPayload();
+ return new SparkplugBPayloadEncoder().getBytes(payload, false);
+ }
+}
diff --git
a/components/camel-tahu/src/test/java/org/apache/camel/component/tahu/TahuHostConsumerMessageTest.java
b/components/camel-tahu/src/test/java/org/apache/camel/component/tahu/TahuHostConsumerMessageTest.java
index 474ce4cf8aad..03d19bad4b8e 100644
---
a/components/camel-tahu/src/test/java/org/apache/camel/component/tahu/TahuHostConsumerMessageTest.java
+++
b/components/camel-tahu/src/test/java/org/apache/camel/component/tahu/TahuHostConsumerMessageTest.java
@@ -50,7 +50,6 @@ class TahuHostConsumerMessageTest {
context.addRoutes(new RouteBuilder() {
@Override
public void configure() {
- // a host id of its own, as a consumer keeps its host
application in a static map
from("tahu-host:MessageHost" + broker.getPort()
+
"?clientId=MessageHostClient&servers=Fake:tcp://localhost:" + broker.getPort())
// the exchanges of the messages, not those of
each metric