shashank created CAMEL-25213:
--------------------------------
Summary: camel-support - FileStateRepository can lose the stored
offsets: stop rewrites the file without the lock, the rewrite truncates the
file first, and an incomplete line prevents a restart
Key: CAMEL-25213
URL: https://issues.apache.org/jira/browse/CAMEL-25213
Project: Camel
Issue Type: Bug
Components: camel-support
Reporter: shashank
{{FileStateRepository}} is the file based {{StateRepository}} documented for
the camel-kafka {{offsetRepository}} option (and usable as the MongoDB change
stream token repository). It keeps the state in a {{HashMap}} and appends a
{{key=value}} line to the file on every {{setState}}; when the file reaches
{{maxFileStoreSize}}, and on every {{doStop}}, it rewrites the whole file from
the map ({{trunkStore}}). Three defects can lose the stored state:
# *{{doStop}} does not hold the lock.* {{setState}}/{{getState}} hold
{{cacheAndStoreLock}}, {{doStop}} calls {{trunkStore()}} and {{cache.clear()}}
without it. A {{setState}} while the repository stops (another route sharing
the repository, a Kafka consumer thread still committing after
{{shutdownTimeout}}, or the MongoDB change stream thread, which
{{MongoDbChangeStreamsConsumer.doStop}} does not wait for) modifies the map
while {{trunkStore}} iterates it: {{doStop}} fails with
{{ConcurrentModificationException}} and the file keeps only the lines written
until then.
# *The rewrite truncates the file first.* {{trunkStore}} opens {{new
FileOutputStream(fileStore)}} (truncate) and then writes the entries, so any
failure or crash while rewriting (on every stop, and every time the file
reaches the size limit) leaves the file empty or partial.
# *An incomplete line prevents a restart.* {{appendToStore}} writes key, {{=}},
value and newline as four separate writes; if the process dies in between, the
last line has no {{=}}, and {{loadStore}} fails with
{{StringIndexOutOfBoundsException}} ({{line.substring(0, -1)}}), so the
repository, and the consumer that starts it, cannot start.
For a Kafka consumer, a lost offset means the partition restarts from
{{auto.offset.reset}} ({{latest}} by default, so the records in between are
skipped; {{earliest}} replays the partition), because the component disables
Kafka's own commits when an offset repository is set.
The {{doStop}} path has not held the lock since the class was added
(CAMEL-20199 only turned the {{synchronized}} blocks into the
{{ReentrantLock}}).
h3. Reproduction
Deterministic tests against the class (three runs each):
* a map as 1st level cache whose iteration pauses while {{stop()}} rewrites the
file; another thread calls {{setState}} meanwhile: {{stop()}} fails with
{{ConcurrentModificationException}}, and after a restart three of the five
stored keys are gone (the file holds only the first two lines);
* a map whose iteration fails half way while {{stop()}} rewrites the file: the
file loses the entries after the failure;
* a store file {{"key1=value1\nkey2=value2\nkey3"}}: {{start()}} fails with
{{StringIndexOutOfBoundsException: Range [0, -1) out of bounds for length 4}}.
h3. Proposed fix
* {{doStop}} holds {{cacheAndStoreLock}}, as {{reset()}} does.
* {{trunkStore}} writes into {{<file>.tmp}}, syncs it and moves it over the
file store ({{ATOMIC_MOVE}}, with a plain replace where the file system does
not support it); on failure the temporary file is deleted and the store is
unchanged.
* {{appendToStore}} writes the line with a single write.
* {{loadStore}} skips a line without {{=}} with a WARN.
No API or file format change. Tests: three new tests in
{{FileStateRepositoryTest}}, which fail without the fix.
Affected: all versions (the same code at camel-3.x and 4.x, now in
camel-support).
Duplicate check (2026-09-30): JIRA text "FileStateRepository",
"StateRepository", "offsetRepository": only CAMEL-20212 (move to
camel-support), CAMEL-12732 and CAMEL-13710 (Kafka manual commit and examples),
CAMEL-23994 (Kafka offset handling, not the repository). GitHub pull requests
"FileStateRepository": none for this.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
