shashank created CAMEL-25525:
--------------------------------
Summary: camel-keycloak - the consumer routes the latest events of
a poll again in the next poll, although the docs promise fingerprint-based
deduplication
Key: CAMEL-25525
URL: https://issues.apache.org/jira/browse/CAMEL-25525
Project: Camel
Issue Type: Bug
Components: camel-keycloak
Reporter: shashank
{{KeycloakConsumer.pollEvents()}} (lines 111-183 at main af49586fea02) queries
the events of the realm without a lower time bound ({{dateFrom}} is an optional
date), skips the events older than its checkpoint {{lastEventTime}} and the
events whose fingerprint it has already routed, and then moves the checkpoint:
{code:java}
if (highestEventTime > (lastEventTime != null ? lastEventTime : 0)) {
// Clear fingerprints when moving to a new timestamp to free memory
processedEventFingerprints.clear();
lastEventTime = highestEventTime;
}
{code}
The clear also forgets the fingerprints of the events *at* the new checkpoint.
The next poll gets those events again from Keycloak; they are not older than
the checkpoint and their fingerprints are gone, so they are routed a second
time. Every poll that sees a newer event therefore delivers the latest event(s)
twice, in the default configuration. {{pollAdminEvents()}} (lines 204-271) has
the same code. The docs say "The consumer uses a polling mechanism with
fingerprint-based deduplication to ensure events are not processed multiple
times."
h3. Reproduction
New {{KeycloakConsumerLatestEventsTest}} (mocked {{Keycloak}} client as in
{{KeycloakConsumerTest}}, {{startScheduler=false}}, the test calls {{poll()}},
so it is deterministic): the realm has LOGOUT at 2000 and LOGIN at 1000; the
first poll routes both, the second poll (same events) must route none; then an
event at 3000 and a second one at 2000 are added and polled twice. The same for
admin events. On main both tests fail, twice:
{noformat}
the events of the first poll must not be routed again ==> expected: <0> but
was: <1>
the admin events of the first poll must not be routed again ==> expected: <0>
but was: <1>
{noformat}
(The existing {{KeycloakConsumerTest}} uses {{expectedMinimumMessageCount(2)}},
so the third delivery passes it.)
h3. Proposed fix
When the checkpoint moves, forget only the fingerprints of the events older
than the new checkpoint and keep those at it (a fingerprint starts with the
event time), for user events and admin events. The fingerprint set still holds
at most the events of the latest timestamp after each poll. Each poll routes
the same events as before except the repeated ones, so no upgrade note.
camel-keycloak tests with the fix: 141 unit tests, 0 failures (the integration
tests need Docker and were not run).
Found with a Lean 4 model of the poll (events as time and id, the checkpoint
and the fingerprint set as state): "a poll of the same events routes nothing
new" fails for two events, and main is proved to route again every event at the
timestamp a poll moves the checkpoint to. The fix is proved to never route
again an event the previous poll routed, for every state and every list of
distinct events. Confirmed with the real consumer.
Not changed: Keycloak returns at most {{maxResults}} (default 100) events per
query, ordered by descending time, so a poll with more new events than that
misses the older ones; a fix needs paging with {{first}} and is left for a
separate issue.
Affected: main, camel-4.22.x, camel-4.18.x (same code, GitHub contents API);
the consumer was added in 4.15 (CAMEL-22487), camel-4.14.x does not have it.
Duplicate check (2026-10-10): JIRA text "keycloak" with "consumer" or "events"
(12, only CAMEL-22487 is about the consumer); GitHub pull requests
"KeycloakConsumer" (none), "keycloak consumer duplicate" (none). No open pull
request changes {{KeycloakConsumer}}.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
