[
https://issues.apache.org/jira/browse/CAMEL-25758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-25758:
--------------------------------
Fix Version/s: 4.24.0
> camel-azure-storage-blob - consumer with moveAfterRead deletes the source
> blob although the copy to the destination container failed, was aborted or
> its status could not be read: the blob is lost
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25758
> URL: https://issues.apache.org/jira/browse/CAMEL-25758
> Project: Camel
> Issue Type: Bug
> Components: camel-azure
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> With {{moveAfterRead=true}} the blob consumer moves a consumed blob in
> {{BlobConsumer.processCommit}} (main e13303ef1ef7, lines 283-286) as copy +
> delete:
> {code:java}
> copyBlobToDestination(containerName, blobName, destinationContainer,
> destinationBlobName);
> deleteBlob(containerName, blobName);
> {code}
> and {{copyBlobToDestination}} (lines 362-363) runs:
> {code:java}
> destBlobClient.beginCopy(sourceBlobClient.getBlobUrl(), Duration.ofSeconds(2))
> .waitForCompletion(timeout);
> {code}
> The result of {{waitForCompletion}} is not looked at. A Copy Blob is an
> asynchronous operation of the service: its status is {{pending}} until it
> ends as {{success}}, {{aborted}} or {{failed}} (Get Blob Properties,
> {{x-ms-copy-status}}). The {{SyncPoller}} of {{BlobClientBase.beginCopy}}
> (azure-storage-blob 12.32, read from the jar) does not throw when the copy
> does not succeed: it returns the last poll response once its status is
> complete, that is {{FAILED}} for a failed copy, {{USER_CANCELLED}} for an
> aborted copy, and the custom complete status {{POLLING_FAILED}} when reading
> the status of a pending copy failed (an exception of {{getProperties()}} is
> caught in {{onPoll}}). Apart from a copy request that the service refuses
> ({{BlobStorageException}} when the copy starts), only a timeout throws
> ({{RuntimeException}} with a {{TimeoutException}}). In all three cases the
> consumer deletes the source blob next. The destination of a failed or aborted
> copy has no content (Copy Blob remarks: an aborted copy leaves a blob of 0
> bytes, a copy that times out on the service leaves an empty blob), and with
> {{POLLING_FAILED}} the copy can still be pending when the source is deleted
> ("any changes to the source while the copy operation is in progress will
> cause that operation to fail", Copy Blob remarks). The blob is gone from both
> containers and Camel logs nothing.
> How likely: the move is always within one storage account (one
> {{BlobServiceClient}}), and Microsoft Learn says such a copy "can complete
> synchronously". A copy that completes synchronously is read back as
> {{success}} and is not affected. The loss needs a copy that the service runs
> asynchronously and that then fails or is aborted, or a failed status read
> while it is pending. How often same-account copies are asynchronous is not
> documented.
> The copy was a synchronous {{copyFromUrl}} up to camel-4.18.x (it throws on
> failure); CAMEL-24158 switched to {{beginCopy}} for the authorization of the
> copy source.
> *Fix* (one class): {{processCommit}} deletes the source blob only when the
> copy ended with {{SUCCESSFULLY_COMPLETED}}. Otherwise it reports the copy
> status to the consumer's exception handler and keeps the blob, which is
> consumed again on the next poll, the same as after a copy request that fails
> with a {{BlobStorageException}} today. A copy that keeps failing is therefore
> retried at every poll with one exception handler call each time, as a copy
> that is refused already is. A copy reported as {{POLLING_FAILED}} that had in
> fact succeeded is copied and routed again on the next poll (at least once
> instead of a possible loss); if the status can never be read (credentials
> that may write but not read the destination container), every move is
> reported and retried this way, where main deleted the source without knowing
> the outcome. A copy that times out is unchanged: the exception leaves the on
> completion, Camel logs it and the source blob is kept.
> *Tests*: new {{BlobConsumerMoveAfterReadCopyStatusTest}} (4 tests, Mockito
> clients as in {{BlobConsumerInProgressTest}}; the poller returns the given
> status): the source blob is kept and the exception handler is called when the
> copy ends {{FAILED}}, {{USER_CANCELLED}} or {{POLLING_FAILED}}; the source
> blob is deleted after a successful copy. The three "kept" tests fail on main
> ({{NeverWantedButInvoked: blobClient.delete()}}), the fourth passes on main
> and guards the success path of the fix. The camel-azure-storage-blob unit
> tests pass with the fix (68 run, 0 failures).
> *How it was found*: deterministic simulation testing with a fake Azure
> storage world (the consumer, its scheduled polls, its on completion and
> Camel's routing real; the sync {{BlobServiceClient}} /
> {{BlobContainerClient}} / {{BlobClient}} faked from the Blob service REST
> reference, with asynchronous copies whose final status the seed picks). The
> seed picked {{moveAfterRead}} or {{deleteAfterRead}}, {{maxResultsPerPage}},
> the poll delay, the {{timeout}}, an asynchronous route step, fault rates, and
> how often a copy stays pending and ends as failed, aborted or with a failed
> status read. The invariant "nothing lost: every blob is still in the source
> container or was moved with its content" failed in 99 of 1000 seeds on main,
> all with {{moveAfterRead=true}} and a copy that did not succeed (first
> failing seed 11: a pending copy ended as failed after about 4 s and the
> source blob was deleted); with the fix, 0 of the same 1000 seeds fail. The
> world's self-check: with CAMEL-25551 reverted, 733 of the 1000 seeds fail its
> invariant "a blob is not routed again while its exchange is in flight".
> *Affected*: main, camel-4.23.x and camel-4.22.x (the same
> {{beginCopy(...).waitForCompletion(timeout)}} followed by the delete; read,
> not run). camel-4.18.x uses {{copyFromUrl}} and is not affected.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
