[
https://issues.apache.org/jira/browse/CAMEL-26893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parker Mauney updated CAMEL-26893:
----------------------------------
Issue Type: Improvement (was: Bug)
> camel-core - adding routes with explicit ids slows quadratically
> ----------------------------------------------------------------
>
> Key: CAMEL-26893
> URL: https://issues.apache.org/jira/browse/CAMEL-26893
> Project: Camel
> Issue Type: Improvement
> Components: camel-core
> Affects Versions: 4.21.0, 4.23.0
> Reporter: Parker Mauney
> Priority: Major
>
> When a route definition with an id is added,
> {{DefaultModel.removeRouteDefinition}} first calls {{stopRoute(id)}} and then
> {{removeRoute(id)}} on the context, so that an existing route with the same
> id gets replaced.
> {{AbstractCamelContext.removeRoute(String, LoggingLevel)}} builds the
> {{endpointsInUse}} map by calling {{gatherEndpoints()}} on every
> {{RouteService}} in the context. Only after that does it look up the route
> being removed:
> {code:java}
> lock.lock();
> try {
> DefaultRouteError.reset(this, routeId);
> Map<String, Set<Endpoint>> endpointsInUse = new HashMap<>();
> for (Map.Entry<String, RouteService> entry : routeServices.entrySet()) {
> endpointsInUse.put(entry.getKey(),
> entry.getValue().gatherEndpoints());
> }
> RouteService routeService = routeServices.get(routeId);
> if (routeService != null) {
> ...
> }
> return false;
> } finally {
> lock.unlock();
> }
> {code}
> When the id is new, the map is built and then discarded. The same happens
> when the route exists but is still started. {{gatherEndpoints()}} walks the
> child services of each route, so every added route pays a full walk of every
> route already in the context. Adding n routes with explicit ids is therefore
> O(n²) in that walk alone.
> Routes without an id don't hit this, because {{removeRouteDefinition}} only
> calls {{removeRoute}} when the id is set. Applications that add many routes
> with explicit ids after the context has started pay that walk for each route.
> *Reproducer:* the manual test below adds n small routes, one at a time, to a
> started {{DefaultCamelContext}} with JMX disabled, using a fresh context for
> each n.
> {code:java}
> @Disabled("Manual test")
> public class AddRoutesWithIdsManualTest {
> private static final Logger LOG =
> LoggerFactory.getLogger(AddRoutesWithIdsManualTest.class);
> @Test
> public void testAddRoutesWithIds() throws Exception {
> // warm up
> addRoutes(1000);
> for (int count = 1000; count <= 5000; count += 1000) {
> long millis = addRoutes(count);
> LOG.info("Added {} routes in {} ms ({} us per route)", count,
> millis, millis * 1000 / count);
> }
> }
> private static long addRoutes(int count) throws Exception {
> try (DefaultCamelContext context = new DefaultCamelContext()) {
> context.disableJMX();
> context.start();
> long start = System.nanoTime();
> for (int i = 0; i < count; i++) {
> final int n = i;
> context.addRoutes(new RouteBuilder() {
> @Override
> public void configure() {
> from("direct:in" + n).routeId("route" + n)
> .filter(header("foo").isNotNull())
> .to("mock:out" + n);
> }
> });
> }
> return TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);
> }
> }
> }
> {code}
> To see how much of the cost is that walk, I tried a local build of main where
> {{removeRoute}} looks the route up before gathering the endpoints, so the
> gather only runs when the route is going to be removed. Results on main
> (4d3b74b1cd0b), JDK 25, one run each, without and with that change:
> ||Routes||main||with the change||
> |1,000|1,087 ms (1,087 us/route)|245 ms (245 us/route)|
> |2,000|4,394 ms (2,197 us/route)|564 ms (282 us/route)|
> |3,000|9,971 ms (3,323 us/route)|885 ms (295 us/route)|
> |4,000|18,998 ms (4,749 us/route)|1,675 ms (418 us/route)|
> |5,000|30,537 ms (6,107 us/route)|2,425 ms (485 us/route)|
> On main, the cost per route grows in proportion to the number of routes
> already loaded.
> The closest existing issue I found is CAMEL-19456, which is about a different
> cost in the same method (the endpoint key mismatch for {{RAW()}} URIs on
> removal) and does not touch this gather.
> Happy to open a PR with the change and the test above if it helps to see it
> in context.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
