[
https://issues.apache.org/jira/browse/CAMEL-25525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25525 started by shashank.
----------------------------------------
> camel-keycloak - the consumer routes the latest events of a poll again in the
> next poll, although the docs promise fingerprint-based deduplication
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25525
> URL: https://issues.apache.org/jira/browse/CAMEL-25525
> Project: Camel
> Issue Type: Bug
> Components: camel-keycloak
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> {{KeycloakConsumer.pollEvents()}} (lines 111-183 at main af49586fea02)
> queries the events of the realm without a lower time bound ({{dateFrom}} is
> an optional date), skips the events older than its checkpoint
> {{lastEventTime}} and the events whose fingerprint it has already routed, and
> then moves the checkpoint:
> {code:java}
> if (highestEventTime > (lastEventTime != null ? lastEventTime : 0)) {
> // Clear fingerprints when moving to a new timestamp to free memory
> processedEventFingerprints.clear();
> lastEventTime = highestEventTime;
> }
> {code}
> The clear also forgets the fingerprints of the events *at* the new
> checkpoint. The next poll gets those events again from Keycloak; they are not
> older than the checkpoint and their fingerprints are gone, so they are routed
> a second time. Every poll that sees a newer event therefore delivers the
> latest event(s) twice, in the default configuration. {{pollAdminEvents()}}
> (lines 204-271) has the same code. The docs say "The consumer uses a polling
> mechanism with fingerprint-based deduplication to ensure events are not
> processed multiple times."
> h3. Reproduction
> New {{KeycloakConsumerLatestEventsTest}} (mocked {{Keycloak}} client as in
> {{KeycloakConsumerTest}}, {{startScheduler=false}}, the test calls
> {{poll()}}, so it is deterministic): the realm has LOGOUT at 2000 and LOGIN
> at 1000; the first poll routes both, the second poll (same events) must route
> none; then an event at 3000 and a second one at 2000 are added and polled
> twice. The same for admin events. On main both tests fail, twice:
> {noformat}
> the events of the first poll must not be routed again ==> expected: <0> but
> was: <1>
> the admin events of the first poll must not be routed again ==> expected: <0>
> but was: <1>
> {noformat}
> (The existing {{KeycloakConsumerTest}} uses
> {{expectedMinimumMessageCount(2)}}, so the third delivery passes it.)
> h3. Proposed fix
> When the checkpoint moves, forget only the fingerprints of the events older
> than the new checkpoint and keep those at it (a fingerprint starts with the
> event time), for user events and admin events. The fingerprint set still
> holds at most the events of the latest timestamp after each poll. Each poll
> routes the same events as before except the repeated ones, so no upgrade
> note. camel-keycloak tests with the fix: 141 unit tests, 0 failures (the
> integration tests need Docker and were not run).
> Found with a Lean 4 model of the poll (events as time and id, the checkpoint
> and the fingerprint set as state): "a poll of the same events routes nothing
> new" fails for two events, and main is proved to route again every event at
> the timestamp a poll moves the checkpoint to. The fix is proved to never
> route again an event the previous poll routed, for every state and every list
> of distinct events. Confirmed with the real consumer.
> Not changed: Keycloak returns at most {{maxResults}} (default 100) events per
> query, ordered by descending time, so a poll with more new events than that
> misses the older ones; a fix needs paging with {{first}} and is left for a
> separate issue.
> Affected: main, camel-4.22.x, camel-4.18.x (same code, GitHub contents API);
> the consumer was added in 4.15 (CAMEL-22487), camel-4.14.x does not have it.
> Duplicate check (2026-10-10): JIRA text "keycloak" with "consumer" or
> "events" (12, only CAMEL-22487 is about the consumer); GitHub pull requests
> "KeycloakConsumer" (none), "keycloak consumer duplicate" (none). No open pull
> request changes {{KeycloakConsumer}}.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
