shashank created CAMEL-25960:
--------------------------------
Summary: camel-file - a done file with a fixed name can be deleted
by the last file of the previous poll when that file completes asynchronously
while the next poll decides which done files to keep
Key: CAMEL-25960
URL: https://issues.apache.org/jira/browse/CAMEL-25960
Project: Camel
Issue Type: Bug
Components: camel-file
Reporter: shashank
Assignee: shashank
With a done file with a fixed name ({{doneFileName=ready}}), a poll that leaves
files of a directory for the next poll keeps the done file, so the last file of
the poll does not delete it ({{GenericFileOnCompletion.handleDoneFile}}, line
199: {{keptDoneFiles.isKept(doneFileName)}}).
When the next poll lists the directory completely,
{{GenericFileKeptDoneFiles.startBatch(listed, true)}} clears the whole set
(line 61, {{kept.clear()}}). The done files of the files this poll leaves are
kept again only later in the same poll: in {{GenericFileConsumer.poll()}} for
the non-eager {{maxMessagesPerPoll}} cut (line 249, after the exchanges are
created and sorted), and in {{processBatch}} for a file that is not started
(line 315) or that is left when the batch stops (line 322).
If the last file of the previous poll completes in between, on another thread
(the route continues with {{threads()}}, {{seda}}, or an asynchronous producer,
so the consumer's next poll can start before that exchange is done),
{{handleDoneFile}} finds the done file not kept and deletes it. The files the
new poll then leaves for the next one have no done file, so they are not
consumed until a new done file is written.
CAMEL-25772 removed the same kind of window on the cut-listing path; this one
is on the complete-listing path and is older (CAMEL-25728).
h3. Reproduction
Not reproduced by a test yet. Outline: {{doneFileName=ready}},
{{maxMessagesPerPoll=2}}, {{eagerMaxMessagesPerPoll=false}}, 3 files and
{{ready}} in the directory; the route hands off with {{threads()}} and its
processor for the second file blocks on a latch; release it after the next poll
has called {{startBatch}} (complete listing) and before it re-keeps the done
file. Expected on main: {{ready}} is deleted and the third file stays in the
directory.
h3. Proposed fix
Decide the kept set of a poll before clearing the old one, or keep the old
entries until the poll has re-kept its own (for example: compute the done files
to keep, then replace the set in one step), so a completing exchange of the
previous poll never sees a done file that both polls keep as not kept.
Affected: main (e6885df7894d). The kept done files exist since CAMEL-25728
(4.24 only).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
