[
https://issues.apache.org/jira/browse/CAMEL-25807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25807 started by shashank.
----------------------------------------
> camel-github2 - the commit consumer silently skips the older commits when
> more than 100 commits are pushed between two polls
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25807
> URL: https://issues.apache.org/jira/browse/CAMEL-25807
> Project: Camel
> Issue Type: Bug
> Components: camel-github
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> {{CommitConsumer.poll}} (main 7aae27315946, lines 104-165) lists the commits
> of the branch, newest first, and stops after 100 ("Stop after 100 commits to
> avoid fetching too many", lines 113-117). It then clips the list at the last
> routed commit ({{lastSha}}) and routes the commits before it, oldest first;
> {{onRouting}} moves {{lastSha}} to each routed commit, so it ends at the
> newest one.
> When more than 100 commits were pushed to the branch between two polls,
> {{lastSha}} is not among the 100 newest: the poll routes those 100 (the ones
> not in the remembered set), {{lastSha}} becomes the newest commit, and the
> next poll stops at it. The older new commits are never routed, and nothing is
> logged. This happens with a large merge or push, and at the first poll after
> GitHub was unreachable (or the polls failed, for example on the rate limit)
> while more than 100 commits were pushed.
> *Fix*: when there is a last routed commit, the poll reads the listing up to
> it, at most 1000 commits (the number of commits the consumer already
> remembers in its set of routed shas). When the last commit is not found (it
> is no longer on the branch, after a force push) or there is none
> ({{startingSha=beginning}}), only the 100 newest commits are considered, as
> before. Upgrade note in the 4.24 guide.
> *Tests*: new {{GitHub2CommitConsumerManyNewCommitsTest}} (mocked repository
> listing, as in {{GitHub2ConsumerPositionTest}}). {{everyNewCommitIsRouted}}
> (150 commits pushed after {{startingSha=c0}}) fails on main: the routed list
> is {{[c51, c52, ..., c150]}} instead of {{[c1, ..., c150]}}; passes with the
> fix. {{withoutTheLastCommitOnTheBranchOnlyTheNewest100AreRouted}} passes on
> main and with the fix (the force push case keeps its behaviour).
> camel-github2 unit tests: 16 run, 0 failures.
> *How it was found*: the same Z3 model as the camel-jira newIssues consumer (a
> cursor that moves to the newest item of a poll whose answer is cut to the
> newest M): an item is lost as soon as M+1 items arrive between two polls,
> none when the poll reads up to the cursor; then the JUnit test above. A
> variant of CAMEL-25806 (camel-jira newIssues).
> Related: CAMEL-21256 ({{startingSha=last}} with a large history). When there
> is no {{lastSha}}, or it is not in the listing (a force push), the poll still
> reads at most the newest 100 commits as before, so that case is unchanged.
> *Affected*: main, camel-4.23.x, camel-4.22.x and camel-4.18.x (the same cut
> at 100; read, not run). camel-4.14.x has no camel-github2.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
