shashank created CAMEL-25808:
--------------------------------
Summary: camel-mail - after the IMAP folder connection dropped
during a poll, the consumer marks (or deletes) another mail than the one it
routed
Key: CAMEL-25808
URL: https://issues.apache.org/jira/browse/CAMEL-25808
Project: Camel
Issue Type: Bug
Components: camel-mail
Reporter: shashank
Assignee: shashank
{{MailConsumer.processCommit}} (main 7aae27315946, lines 520-589) marks a mail
once its exchange completed: it opens the folder again if it is closed, then
sets SEEN or DELETED (and copies / moves the mail for {{copyTo}} / {{moveTo}})
on the mail object of the poll:
{code:java}
// ensure folder is open
if (!folder.isOpen()) {
folder.open(Folder.READ_WRITE);
}
...
mail.setFlag(Flags.Flag.SEEN, true); // or DELETED, after
folder.copyMessages(new Message[] { mail }, dest)
{code}
The folder is closed at that point when its IMAP connection dropped while the
mail was routed (an idle timeout of the server during a long exchange, a
network failure): angus-mail cleans the folder up when it notices that, and
{{isOpen()}} returns false. The mail object belongs to the earlier open. An
angus-mail {{IMAPMessage}} sends its commands with {{getSequenceNumber()}},
which looks its message number up in the message cache of the *current* open of
its folder ({{folder.messageCache.seqnumOf(msgnum)}}); the new open builds that
cache from the mailbox as it is now. When mails were expunged between the two
opens (another mail client, a second Camel consumer on the same mailbox with a
shared {{idempotentRepository}} closing its folder, or a server that expunges a
deleted mail at once), the number designates another mail:
* the mail that was routed is not marked: it stays unread and is routed again
(a duplicate);
* another mail is marked instead. With the default {{unseen=true}} a mail
marked SEEN is never polled again, so if it was not routed yet (for example a
mail after the {{maxMessagesPerPoll}} limit) it is never consumed. With
{{delete=true}} or {{moveTo}} it is deleted (moved) without being routed.
Nothing is logged.
*Reproduction* ({{MailConsumerFolderReopenedTest}}, GreenMail and the real
angus-mail client, as the other tests of the module): three mails,
{{maxMessagesPerPoll=2}}, the consumer connects through a small TCP proxy.
While the second mail is routed, another client removes the first one (already
consumed) and the proxy drops the consumer's connections. On main the commit of
the second mail marks the third mail SEEN: the mock receives "Message 0",
"Message 1", "Message 1" and never "Message 2" ({{Message with body Message 2
was expected but not found in [Message 1]}}, 2 of 2 runs).
*Fix*: {{MailConsumer}} fetches the UIDs of the mails of a poll when it
retrieves them (one FETCH), and counts the opens of the folder. When the folder
was opened again after the mails of the batch were retrieved, the mail to mark
is looked up by its UID (same UIDVALIDITY); when it is no longer there, nothing
is marked (WARN). The batch is not routed further once the folder was opened
again (the other mail objects are stale too, and with {{mapMailMessage=false}}
their content would be read from other mails); those mails are given back as
for a batch cut short by a stop (CAMEL-25710) and consumed by the next poll.
POP3 is not changed: {{processCommit}} already finds a POP3 mail again by its
key (see CAMEL-24434 for that key).
*Tests*: the new test fails on main and passes with the fix. camel-mail unit
tests with the fix: 246 run, 0 failures, 0 errors, 4 skipped.
*How it was found*: deterministic simulation testing with a fake IMAP world
(camel-test-dst, CAMEL-25738): the real {{MailConsumer}}, a fake IMAP server
and client that follows RFC 3501 and the angus-mail message cache behaviour
above, seeded connection drops, a second consumer, other clients that expunge
and a server that expunges at once. Invariant: no mail is marked SEEN or
DELETED, or copied, before its exchange completed. 384 of 5000 seeds failed on
main, first failing seed 4 (a failing seed replays with the same trace hash); 0
of 5000 with the fix. The world re-finds CAMEL-25710 when that fix is reverted.
Then the JUnit test above with the real client.
*Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x (the same
{{processCommit}}; read, not run). Not run against a real IMAP server other
than GreenMail.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
