This is an automated email from the ASF dual-hosted git repository.

davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git


The following commit(s) were added to refs/heads/main by this push:
     new dc3e6f74ba20 CAMEL-27210: camel-box - the events consumer listens 
again after the event stream failed, and routes only events (#30466)
dc3e6f74ba20 is described below

commit dc3e6f74ba2043bcd309ae300cca09c3c237103e
Author: allthingssecurity <[email protected]>
AuthorDate: Fri Nov 20 14:07:30 2026 +0530

    CAMEL-27210: camel-box - the events consumer listens again after the event 
stream failed, and routes only events (#30466)
    
    When a request of the poller thread of the Box SDK EventStream fails (a
    network error, Box not reachable for longer than the SDK retries), the
    SDK stops the stream before it calls onException of the listeners, and
    nothing starts it again. BoxConsumer only handed the exception to the
    exception handler: the consumer received no event any more while the
    route stayed started. onNextPosition ignored the stream position.
    
    BoxConsumer is also an AbstractApiConsumer, a scheduled poll consumer,
    and did not override poll(): every poll (500 ms by default) invoked the
    listen method again without a listener, which does nothing, and routed
    an exchange with a null body for its void result.
    
    - onNextPosition keeps the next stream position.
    - onException schedules a restart (1 s, doubled up to 60 s while it keeps
      failing, back to 1 s once events are received) that listens with a new
      event stream from the position after the last events received, or from
      the startingPosition of the endpoint before any.
    - The restart runs on a scheduled executor of the consumer, created in
      doStart and shut down in doStop; a lock orders a restart and a stop.
    - poll() routes nothing and marks the consumer ready.
    - BoxEventsConsumerTest: a Box connection with a request interceptor
      answers like the service and fails requests with the exception of a
      network error.
    
    camel-box - upgrade guide entry for the events consumer
    
    Review follow-up. box:events/listen no longer routes an exchange with a null
    body at every poll, and the consumer listens again after the event stream
    fails (after 1 s, doubled after each failure up to 60 s).
    
    Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
 .../apache/camel/component/box/BoxConsumer.java    | 107 +++++++++-
 .../camel/component/box/BoxEventsConsumerTest.java | 233 +++++++++++++++++++++
 .../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc    |  13 ++
 3 files changed, 349 insertions(+), 4 deletions(-)

diff --git 
a/components/camel-box/camel-box-component/src/main/java/org/apache/camel/component/box/BoxConsumer.java
 
b/components/camel-box/camel-box-component/src/main/java/org/apache/camel/component/box/BoxConsumer.java
index 31c4a50d73eb..844d4bd4d9b1 100644
--- 
a/components/camel-box/camel-box-component/src/main/java/org/apache/camel/component/box/BoxConsumer.java
+++ 
b/components/camel-box/camel-box-component/src/main/java/org/apache/camel/component/box/BoxConsumer.java
@@ -19,6 +19,10 @@ package org.apache.camel.component.box;
 import java.util.HashMap;
 import java.util.Map;
 import java.util.Set;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
 
 import com.box.sdk.BoxAPIConnection;
 import com.box.sdk.BoxEvent;
@@ -42,11 +46,24 @@ public class BoxConsumer extends 
AbstractApiConsumer<BoxApiName, BoxConfiguratio
     private static final Logger LOG = 
LoggerFactory.getLogger(BoxConsumer.class);
 
     private static final String LISTENER_PROPERTY = "listener";
+    private static final String STARTING_POSITION_PROPERTY = 
"startingPosition";
+    // the delay before the events are listened to again after the event 
stream failed, doubled up to the maximum while
+    // it keeps failing
+    private static final long RESTART_DELAY = 1000;
+    private static final long MAX_RESTART_DELAY = 60000;
 
     private BoxAPIConnection boxConnection;
     private BoxEventsManager apiProxy;
     private final ApiMethod apiMethod;
     private final Map<String, Object> properties;
+    // guards the event stream between a restart and a stop
+    private final Lock lock = new ReentrantLock();
+    private ScheduledExecutorService restartExecutor;
+    // the position after the last events received: a restart continues from 
there, so the events of the outage are
+    // not skipped
+    private volatile Long nextPosition;
+    // guarded by the lock
+    private long restartDelay = RESTART_DELAY;
 
     public BoxConsumer(BoxEndpoint endpoint, Processor processor) {
         super(endpoint, processor);
@@ -64,6 +81,18 @@ public class BoxConsumer extends 
AbstractApiConsumer<BoxApiName, BoxConfiguratio
         propertyNames.add(LISTENER_PROPERTY);
     }
 
+    /**
+     * The events are received from the event stream that {@link #doStart()} 
listens to, not polled: a poll of the API
+     * consumer would call the listen method again without this consumer as 
listener (which does nothing and returns no
+     * result), and route an exchange with no body for that result, on every 
poll.
+     */
+    @Override
+    protected int poll() throws Exception {
+        // the event stream is listened to, so the consumer is ready
+        forceConsumerAsReady();
+        return 0;
+    }
+
     @Override
     public void onEvent(BoxEvent event) {
         try {
@@ -75,15 +104,67 @@ public class BoxConsumer extends 
AbstractApiConsumer<BoxApiName, BoxConfiguratio
 
     @Override
     public void onNextPosition(long position) {
-        // noop
+        nextPosition = position;
+        lock.lock();
+        try {
+            // the event stream works again
+            restartDelay = RESTART_DELAY;
+        } finally {
+            lock.unlock();
+        }
     }
 
+    /**
+     * Called by the event stream when it fails (for example when Box cannot 
be reached for a while). The event stream
+     * has stopped itself by then and is never started again by the Box SDK: 
listen again after a delay, else the
+     * consumer stays started without receiving any event.
+     */
     @Override
     public boolean onException(Throwable e) {
         
getExceptionHandler().handleException(RuntimeCamelException.wrapRuntimeCamelException(e));
+        scheduleRestart();
         return true;
     }
 
+    private void scheduleRestart() {
+        lock.lock();
+        try {
+            if (!isRunAllowed() || restartExecutor == null) {
+                return;
+            }
+            long delay = restartDelay;
+            restartDelay = Math.min(restartDelay * 2, MAX_RESTART_DELAY);
+            LOG.info("The Box event stream stopped, listening for events again 
in {} ms", delay);
+            restartExecutor.schedule(this::restart, delay, 
TimeUnit.MILLISECONDS);
+        } finally {
+            lock.unlock();
+        }
+    }
+
+    private void restart() {
+        lock.lock();
+        try {
+            if (!isRunAllowed()) {
+                return;
+            }
+            Long position = nextPosition != null ? nextPosition : 
startingPosition();
+            apiProxy.stopListening();
+            apiProxy.listen(this, position);
+            LOG.info("Listening for Box events again from position {}", 
position != null ? position : "now");
+        } catch (Exception e) {
+            // such as Box still not reachable when the event stream starts
+            LOG.debug("Cannot listen for Box events again", e);
+            scheduleRestart();
+        } finally {
+            lock.unlock();
+        }
+    }
+
+    private Long startingPosition() {
+        Object value = properties.get(STARTING_POSITION_PROPERTY);
+        return value != null ? 
getEndpoint().getCamelContext().getTypeConverter().convertTo(Long.class, value) 
: null;
+    }
+
     @Override
     public BoxEndpoint getEndpoint() {
         return (BoxEndpoint) super.getEndpoint();
@@ -94,15 +175,33 @@ public class BoxConsumer extends 
AbstractApiConsumer<BoxApiName, BoxConfiguratio
         super.doStart();
 
         boxConnection = getEndpoint().getBoxConnection();
+        nextPosition = null;
+        restartDelay = RESTART_DELAY;
+        restartExecutor = 
getEndpoint().getCamelContext().getExecutorServiceManager()
+                .newSingleThreadScheduledExecutor(this, "BoxEventsRestart");
         apiProxy = new BoxEventsManager(boxConnection);
         // invoke the API method to start listening
-        ApiMethodHelper.invokeMethod(apiProxy, apiMethod, properties);
+        lock.lock();
+        try {
+            ApiMethodHelper.invokeMethod(apiProxy, apiMethod, properties);
+        } finally {
+            lock.unlock();
+        }
     }
 
     @Override
     protected void doStop() throws Exception {
-        if (apiProxy != null) {
-            apiProxy.stopListening();
+        lock.lock();
+        try {
+            if (restartExecutor != null) {
+                
getEndpoint().getCamelContext().getExecutorServiceManager().shutdownNow(restartExecutor);
+                restartExecutor = null;
+            }
+            if (apiProxy != null) {
+                apiProxy.stopListening();
+            }
+        } finally {
+            lock.unlock();
         }
         super.doStop();
     }
diff --git 
a/components/camel-box/camel-box-component/src/test/java/org/apache/camel/component/box/BoxEventsConsumerTest.java
 
b/components/camel-box/camel-box-component/src/test/java/org/apache/camel/component/box/BoxEventsConsumerTest.java
new file mode 100644
index 000000000000..73998cffa6ef
--- /dev/null
+++ 
b/components/camel-box/camel-box-component/src/test/java/org/apache/camel/component/box/BoxEventsConsumerTest.java
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.box;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import com.box.sdk.BoxAPIConnection;
+import com.box.sdk.BoxAPIException;
+import com.box.sdk.BoxAPIRequest;
+import com.box.sdk.BoxAPIResponse;
+import com.box.sdk.BoxEvent;
+import com.box.sdk.BoxJSONResponse;
+import com.eclipsesource.json.JsonArray;
+import com.eclipsesource.json.JsonObject;
+import org.apache.camel.CamelContext;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.box.internal.BoxConnectionHelper;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.impl.DefaultCamelContext;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * The events consumer routes the events of the Box event stream and nothing 
else (the polls of the API consumer do not
+ * route exchanges of their own), and listens again when the event stream of 
the Box SDK fails (Box cannot be reached
+ * for a while): the SDK stops the stream for good on such a failure.
+ * <p>
+ * The Box connection answers from memory, as the service does: the long poll 
URL from OPTIONS /events, new_change on
+ * the long poll while there are events after the stream position, and the 
events after a stream position with the next
+ * position. A request that fails throws the exception the SDK throws for a 
network error.
+ */
+class BoxEventsConsumerTest {
+
+    private static final String REALTIME_URL = 
"https://realtime.box.test/subscribe?channel=1";;
+
+    /** The events of the account, in order: the stream position is the index 
in this list. */
+    private final List<String> events = Collections.synchronizedList(new 
ArrayList<>());
+    /** The number of requests for events that fail before the next one is 
answered. */
+    private final AtomicInteger failingEventRequests = new AtomicInteger();
+    private final List<String> eventRequestPositions = 
Collections.synchronizedList(new ArrayList<>());
+    private final CountDownLatch firstEventRouted = new CountDownLatch(1);
+
+    private CamelContext context;
+    private MockedStatic<BoxConnectionHelper> helper;
+
+    @BeforeEach
+    void setUp() {
+        BoxAPIConnection connection = new BoxAPIConnection("test-token");
+        connection.setRequestInterceptor(this::respond);
+        helper = Mockito.mockStatic(BoxConnectionHelper.class);
+        helper.when(() -> 
BoxConnectionHelper.createConnection(Mockito.any())).thenReturn(connection);
+    }
+
+    @AfterEach
+    void tearDown() throws Exception {
+        try {
+            if (context != null) {
+                context.stop();
+            }
+        } finally {
+            helper.close();
+        }
+    }
+
+    @Test
+    void routesNothingWithoutEvents() throws Exception {
+        MockEndpoint mock = createRoute();
+        mock.expectedMessageCount(0);
+        // several polls of the consumer (delay 500 ms)
+        mock.setAssertPeriod(2000);
+        context.start();
+
+        mock.assertIsSatisfied();
+    }
+
+    @Test
+    void routesEachEventOnce() throws Exception {
+        events.add("event-0");
+        MockEndpoint mock = createRoute();
+        mock.expectedBodiesReceived("event-0");
+        mock.setResultWaitTime(10000);
+        // several polls of the consumer (delay 500 ms)
+        mock.setAssertPeriod(2000);
+        context.start();
+
+        mock.assertIsSatisfied();
+    }
+
+    @Test
+    void listensAgainWhenTheEventStreamFails() throws Exception {
+        events.add("event-0");
+        // Box cannot be reached when the consumer asks for the first events
+        failingEventRequests.set(1);
+
+        MockEndpoint mock = createRoute();
+        mock.expectedBodiesReceived("event-0");
+        mock.setResultWaitTime(10000);
+        context.start();
+
+        mock.assertIsSatisfied();
+    }
+
+    @Test
+    void continuesAfterTheLastEventsReceived() throws Exception {
+        events.add("event-0");
+        MockEndpoint mock = createRoute();
+        mock.expectedBodiesReceived("event-0", "event-1");
+        mock.setResultWaitTime(10000);
+        context.start();
+
+        assertTrue(firstEventRouted.await(10, TimeUnit.SECONDS));
+        // Box cannot be reached when the next event comes
+        failingEventRequests.set(2);
+        events.add("event-1");
+
+        mock.assertIsSatisfied();
+        // the stream is opened again at the position after the events 
received, not at the start
+        assertEquals("1", 
eventRequestPositions.get(eventRequestPositions.size() - 1));
+    }
+
+    /**
+     * Creates the CamelContext with the route of the consumer; the test sets 
the expectations of the mock endpoint it
+     * returns, then starts the context.
+     */
+    private MockEndpoint createRoute() throws Exception {
+        BoxConfiguration configuration = new BoxConfiguration();
+        configuration.setUserName("[email protected]");
+        configuration.setUserPassword("p4ssw0rd");
+        configuration.setClientId("camel-client-id");
+        configuration.setClientSecret("camel-client-secret");
+        configuration.setAuthenticationType("STANDARD_AUTHENTICATION");
+        BoxComponent component = new BoxComponent();
+        component.setConfiguration(configuration);
+
+        context = new DefaultCamelContext();
+        context.addComponent("box", component);
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                from("box:events/listen?startingPosition=0")
+                        // the id of the event as body, and any other body as 
it is
+                        .process(e -> {
+                            BoxEvent event = 
e.getMessage().getBody(BoxEvent.class);
+                            if (event != null) {
+                                e.getMessage().setBody(event.getID());
+                            }
+                        })
+                        .to("mock:events")
+                        .process(e -> firstEventRouted.countDown());
+            }
+        });
+        return context.getEndpoint("mock:events", MockEndpoint.class);
+    }
+
+    private BoxAPIResponse respond(BoxAPIRequest request) {
+        String url = request.getUrl().toString();
+        if ("OPTIONS".equals(request.getMethod())) {
+            JsonObject entry = new JsonObject().add("type", 
"realtime_server").add("url", REALTIME_URL)
+                    .add("ttl", "10").add("max_retries", 
"10").add("retry_timeout", 610);
+            return json(request, new JsonObject().add("chunk_size", 
1).add("entries", new JsonArray().add(entry)));
+        }
+        long position = Long.parseLong(parameter(url, "stream_position"));
+        if (url.startsWith(REALTIME_URL)) {
+            if (position < events.size()) {
+                return json(request, new JsonObject().add("message", 
"new_change"));
+            }
+            // a long poll waits for a change: answer after a while that there 
was none
+            try {
+                new CountDownLatch(1).await(50, TimeUnit.MILLISECONDS);
+            } catch (InterruptedException e) {
+                Thread.currentThread().interrupt();
+            }
+            return json(request, new JsonObject().add("message", "reconnect"));
+        }
+        eventRequestPositions.add(String.valueOf(position));
+        if (failingEventRequests.getAndUpdate(n -> Math.max(0, n - 1)) > 0) {
+            throw new BoxAPIException(
+                    "Couldn't connect to the Box API due to a network error. 
Request\n" + url,
+                    new IOException("Connection refused"));
+        }
+        JsonArray entries = new JsonArray();
+        int next;
+        synchronized (events) {
+            next = events.size();
+            for (int i = (int) position; i < next; i++) {
+                entries.add(new JsonObject().add("type", 
"event").add("event_id", events.get(i))
+                        .add("event_type", "ITEM_UPLOAD").add("created_at", 
"2026-01-01T00:00:00-00:00"));
+            }
+        }
+        return json(request, new JsonObject().add("chunk_size", entries.size())
+                .add("next_stream_position", next).add("entries", entries));
+    }
+
+    private static BoxJSONResponse json(BoxAPIRequest request, JsonObject 
body) {
+        return new BoxJSONResponse(200, request.getMethod(), 
request.getUrl().toString(), Map.of(), body);
+    }
+
+    private static String parameter(String url, String name) {
+        for (String parameter : url.substring(url.indexOf('?') + 
1).split("&")) {
+            if (parameter.startsWith(name + "=")) {
+                return parameter.substring(name.length() + 1);
+            }
+        }
+        throw new IllegalArgumentException("No " + name + " in " + url);
+    }
+}
diff --git 
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc 
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index fdd37e2227b3..8e5d21dfb688 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -6902,6 +6902,19 @@ Previously it returned only the first chunk of the event 
log (at most 500 events
 any sign. A wide `after`/`before` window now returns every event in one list: 
that is one request per 500 events plus
 one that finds the end of the log, and more memory. Narrow the window when the 
event log is large.
 
+=== camel-box - the events consumer routes only events, and listens again 
after the event stream fails
+
+A route that reads from `box:events/listen` no longer gets an exchange with a 
`null` body at every poll of the consumer
+(every 500 ms by default), besides the events. A route that counted or 
filtered those exchanges, or used them as a
+heartbeat, gets only the exchanges of the Box events now.
+
+When the event stream fails (such as when Box cannot be reached for a while), 
the Box SDK stops it for good, and the
+consumer stayed started without receiving any event. The consumer now listens 
again after 1 second, doubled after each
+failure up to 60 seconds, and back to 1 second once the stream receives events 
again. It continues from the position
+after the last events it received, or from the `startingPosition` of the 
endpoint (from now when it has none) when it
+received no event yet. Each failure of the stream still goes to the exception 
handler of the consumer (a WARN log by
+default).
+
 === camel-aws2-eventbridge - the consumer keeps one target for a queue given 
with queueUrl
 
 A consumer that uses its own queue (`queueUrl`) added a target with a new 
random id to the EventBridge rule at every

Reply via email to