[
https://issues.apache.org/jira/browse/CAMEL-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118117#comment-18118117
]
Claus Ibsen commented on CAMEL-24911:
-------------------------------------
Design note, after reading DefaultErrorRegistry (Claus's question: the registry
is per exchange, so every entry is unique).
Right, and the entries must stay per exchange: each one carries its own
exchange id, body, headers and message history, which is the evidence a person
opens the console for. "Collapse" therefore means two things, neither of which
merges exchanges.
*1. A count per signature, kept in the registry*
On capture, derive a signature from what makes two failures "the same": route
id, failure node id, exception class and exception message. Keep a bounded map
signature -> {count, first timestamp, last timestamp}, incremented on every
capture, evicted like the entries (say 100 signatures, and the same time to
live). Cheap, no extra allocation per exchange, and it survives the eviction of
the entries themselves: after the storm the registry still knows "132
occurrences between 16:09:28 and 16:10:43" although only the newest few
exchanges are kept.
The HTTP storm that prompted this had two signatures, 132 for one URL and 88
for another (the message carries the URL), which is the granularity we want:
same route, same node, same error text = one group.
*2. Folding on the reading side, not in the store*
{{camel get errors}}, {{camel_get_errors}} and the error dev console answer one
row per signature by default: the newest entry in full, plus count, first and
last time; a flag ({{--all}}, {{group=false}}) returns every entry as today.
Nothing is lost, the default answer is short enough for a model's context and
for a person's screen. This part needs no new state at all when the counters of
(1) exist.
*3. The log is a separate reader*
{{camel_get_log}} and {{camel get log}} fold identical consecutive records into
one line with a count and a time span ("... (x132, 16:09:28 to 16:10:43)").
Purely in the reader, the log file is untouched.
So: counters in the registry (bounded, per signature), entries untouched,
folding in the three readers. Happy to implement 1 and 2 first, 3 after.
> camel-core, camel-jbang - collapse identical repeated errors in the error
> registry and in camel_get_log
> -------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-24911
> URL: https://issues.apache.org/jira/browse/CAMEL-24911
> Project: Camel
> Issue Type: Improvement
> Components: camel-core, camel-jbang
> Reporter: Claus Ibsen
> Priority: Major
> Fix For: 4.23.0
>
>
> When a route fails the same way for every message, the log and the error
> registry fill with identical entries: in one step of the camel-jbang-mcp
> stepwise benchmark a file consumer replayed three orders on every reload and
> each line failed the same HTTP call, giving 264 entries in the error
> registry, 132 of them for one URL, each with a full stack trace and message
> history. A model then reads 60 log records and sees nothing but repeats of
> one failure; a person tailing a dev log has the same problem.
> Collapse them: the error registry keeps one entry per (route, node,
> exception, message) with a count, first and last time, instead of one per
> exchange, and {{camel_get_log}} / {{camel get log}} folds identical
> consecutive records into one line with "(x132, 16:09:28 to 16:10:43)". The
> individual exchanges stay available (the console's detail view,
> {{camel_get_errors}} with a limit), only the default answer is folded.
> Related: CAMEL-24863 made the registry mark a copy handled when the original
> recovers; this is the volume side of the same data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
