[
https://issues.apache.org/jira/browse/CAMEL-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marc Carter updated CAMEL-8010:
-------------------------------
Description:
CAMEL-6097 Patched a pretty clear race condition between the completion thread
(CT) and recovery thread (RT) but leaves several holes when exercised with a
Jdbc repository and a separate aggregation thread (AT).
#1 is relevant to all repository backends.
#2 only affects fully transactional backends
I'm currently taking a look into this bug as its a show-stopper that
_persistent_ repositories actually *decreases* reliability. (Untested)
workaround is to add an in-memory idemptotentconsumer immediately after the
aggregation.
Here AT starts and completes an aggregation between defensive copy and when RT
repo scanning starts. CT then confirms it (in memory (*)) before repo scanning
ends.
|| AT || RT || CT ||
| | inProg COPY to inProgCopy | |
| inProg ADD x | | |
| repo START x | | |
| | | repo REMOVE x |
| | | <commit> |
| | repo SCAN (sees x) | |
| | | {color:red}process x{color} |
| | | repo CONFIRM x |
| | | inProg REMOVE x |
| | | <commit> |
| | x not inProg or inProgCopy | |
| | {color:red}process x{color} | |
| | repo CONFIRM x | | Fails silently as this is doInTransactionWithoutResult
| | <commit> | |
{noformat}SQLWarning ignored: SQL state '02000', error code '10000', message
[No row was found for FETCH, UPDATE or DELETE; or the result of a query is an
empty table.]{noformat}
(*) Side note: inProgressExchanges is updated by a {{Synchronisation}} inside
the UOW so is immediately visible although any DB change may not be visible for
ages (in threading terms) as the entire transaction must commit first.
was:
CAMEL-6097 Patched a pretty clear race condition between the completion thread
(CT) and recovery thread (RT) but leaves several holes when exercised with a
Jdbc repository and a separate aggregation thread (AT).
#1 is relevant to all repository backends.
#2 only affects fully transactional backends
I'm currently taking a look into this bug as its a show-stopper that
_persistent_ repositories actually *decreases* reliability. (Untested)
workaround is to add an in-memory idemptotentconsumer immediately after the
aggregation.
Here AT starts and completes an aggregation between defensive copy and when RT
repo scanning starts. CT then confirms it (in memory (*)) before repo scanning
ends.
|| AT || RT || CT ||
| | inProg COPY to inProgCopy | |
| inProg ADD x | | |
| repo START x | | |
| repo REMOVE x | | |
| <commit> | | |
| | repo SCAN (sees x) | |
| | | {color:red}process x{color} |
| | | repo CONFIRM x |
| | | inProg REMOVE x |
| | | <commit> |
| | x not inProg or inProgCopy | |
| | {color:red}process x{color} | |
| | <commit> | |
(*) Side note: inProgressExchanges is updated by a {{Synchronisation}} inside
the UOW so is immediately visible although any DB change may not be visible for
ages (in threading terms) as the entire transaction must commit first.
> Race condition in AggregatorProcessor recovery sometimes causes duplicates
> (still)
> ----------------------------------------------------------------------------------
>
> Key: CAMEL-8010
> URL: https://issues.apache.org/jira/browse/CAMEL-8010
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.14.0
> Reporter: Marc Carter
>
> CAMEL-6097 Patched a pretty clear race condition between the completion
> thread (CT) and recovery thread (RT) but leaves several holes when exercised
> with a Jdbc repository and a separate aggregation thread (AT).
> #1 is relevant to all repository backends.
> #2 only affects fully transactional backends
> I'm currently taking a look into this bug as its a show-stopper that
> _persistent_ repositories actually *decreases* reliability. (Untested)
> workaround is to add an in-memory idemptotentconsumer immediately after the
> aggregation.
> Here AT starts and completes an aggregation between defensive copy and when
> RT repo scanning starts. CT then confirms it (in memory (*)) before repo
> scanning ends.
> || AT || RT || CT ||
> | | inProg COPY to inProgCopy | |
> | inProg ADD x | | |
> | repo START x | | |
> | | | repo REMOVE x |
> | | | <commit> |
> | | repo SCAN (sees x) | |
> | | | {color:red}process x{color} |
> | | | repo CONFIRM x |
> | | | inProg REMOVE x |
> | | | <commit> |
> | | x not inProg or inProgCopy | |
> | | {color:red}process x{color} | |
> | | repo CONFIRM x | | Fails silently as this is doInTransactionWithoutResult
> | | <commit> | |
> {noformat}SQLWarning ignored: SQL state '02000', error code '10000', message
> [No row was found for FETCH, UPDATE or DELETE; or the result of a query is an
> empty table.]{noformat}
> (*) Side note: inProgressExchanges is updated by a {{Synchronisation}} inside
> the UOW so is immediately visible although any DB change may not be visible
> for ages (in threading terms) as the entire transaction must commit first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
