This is a curious problem in computer science and especially in distributed
networks.
API Idempotency is a major issue for software like Fineract, unless the
API handles drop off and recognize the second call or any n call as either
duplicate or important in any given case, like how stripe does this by
specifying a key from the client end ensures that even in case of failure
the charge does not happen twice!
This can lead to financial loss in case of real payment taking place!
I think, as of right now there is no such mechanism in Fineract 1.x that
eliminates such risk. Except, on the validation side, but that’s not
enough.
To have this implemented we have done some work on this last year , maybe
we can present some code also !
I have used some cryptography libs to have the client-side send a unique
key that’s generated by the private key infrastructure and the client-side
has to send these as headers and on the server-side, the signatures are
validated thus making the idempotency in POST and PUT request.
The code on the client side
var keys = {
alice: '38f93bdda21a5c4a7bae4eb75bb7811cbc3eb627176805c1009ff2099263c6ad',
bob: '09880c962437080d72f72c8c63a69efd65d086c9e7851a87b76373eb6ce9aab5'};
// GET
for(k in keys) {
var url = 'APIURL';
var dataToSign = url;
var options = {
url: url,
headers: {
'x-identity': auth.getPublicKeyFromPrivateKey(keys[k]),
'x-signature': auth.sign(dataToSign, keys[k])
}
};
The above code generates these Keys are now passed on each request to the
server, and the server using the middleware in case of Fineract have to do
two things
1. VerifySignature
2. GetSinFromThePublicKey
I started writing some code for this and quickly realized that it's
actually doable, it requires around 20 hours of work, There are some
dependency that needs to be added on the server side and some quick changes
on the middleware and its done.
We presented a paper on this last year on Apache Con ! Remember something
about using zero knowledge of proof and password less auth .
Let me know , I would like to make sure that this gets added in this
upcoming release.
On Friday, October 22, 2021, James Dailey <[email protected]> wrote:
> Devs -
>
> Through a conversation with a computer science person, I've learned that
> Fineract should have a clearly articulated approach with regard to
> idempotency. This is particularly true when thinking about how fineract
> interacts with payment systems.
>
> "When building a system to move money, it is paramount that operations
> that move money are idempotent. Failure to do this might result in
> accidentally double charging your customer, or paying a vendor multiple
> times. The risk of this happening is elevated based on the way software is
> typically built today, since developers take advantage of scalable systems
> that process multiple items in parallel. "
> (
> https://www.moderntreasury.com/journal/why-idempotency-matters-in-payments
> )
>
> Or this article on Stripe engineering calling for liberal use of
> Idempotency:
>
> "The easiest way to address inconsistencies in distributed state caused by
> failures is to implement server endpoints so that they’re idempotent, which
> means that they can be called any number of times while guaranteeing that
> side effects only occur once." (https://stripe.com/blog/idempotency)
>
> and this article in Apache Camel
>
> https://camel.apache.org/camel-kafka-connector/latest/user-guide/idempotency.html
>
> So, my understanding may be wrong and we've got this covered, or maybe we
> just consider this overkill. Perhaps there is something in the "magic"
> that's developed in the CQRS plus in-memory database queuing and our clever
> transaction state management, but... I don't know.
>
> Or perhaps we haven't really had to think about this because we haven't
> had that many large scale deployments and the scalability issues associated
> with multiple parallel calls haven't come to the fore, or haven't come back
> to the list. So, that's the intent of this email.
>
> Do we have a need now for layering on Idempotency checks? What is the
> right approach? Has anyone already implemented this, or decided not to for
> very good reasons? What would be the right approach to this? Implement
> where? Time limited to 3 hrs? Etc...
>
> So, this is a call to discuss. I'm far from an expert on this.
>
>
>
