Re: Integration - Retry/Repeated/concurrent calls

[Sergio Junior]

Vishwas,

Thank your very much for your clarification, im starting with Fineract and
still absorbing information.





Em qui., 26 de mar. de 2020 às 09:09, Vishwas Babu A J <
vishwasbab...@gmail.com> escreveu:

> Sergio,
>
> A quick clarification:
>
> https://github.com/apache/fineract/pull/515 allows setting a globally
> unique client generated identifier for a command. This allows the client to
> later query for the status of the execution of the command associated with
> this guid if the response is lost in transit etc. At the server side, the
> guid is assumed to be unique.
>
> By adding a unique constraint to the command source table, we can enforce
> the uniqueness of the client generated identifier at the server side,
> thereby ensuring duplicate requests are not served. However, this approach
> would likely result in a write penalty for very large installations.
>
> Alternatives could be to check the presence of this key in an in memory
> key value store, something along the lines of setnx in Redis (assuming
> duplicates requests are sent in a fixed time interval). We could maybe
> handle this from an infrastructure level too, by carrying out this check at
> the API gateway,
>
> Regards,
> Vishwas
>
>
>
> On Mar 25, 2020, at 6:54 PM, Vishwas Babu  wrote:
>
> >>Question, before this PR, no one have this kind of problem before?
> I am sure people would have faced this issue and figured out solutions
> like this PR.
>
> >>A simple retry or double click at any of the apps would cause issues,
> right?
> It might not have been so obvious as clients like the Mifos community-app
> (most common UI used with Fineract) handle double clicks at their end.
> However, your point is valid
>
> Regards,
> Vishwas
>
>
> On Wed, 25 Mar 2020 at 07:19, Sergio Junior  wrote:
>
>> Vishwas,
>>
>> I really appreciate your response, missed that PR because it was closed,
>> i will take a look.
>>
>> Question, before this PR, no one have this kind of problem before? A
>> simple retry or double click at any of the apps would cause issues, right?
>>
>> Thanks!
>>
>> Em qua., 25 de mar. de 2020 às 01:24, Vishwas Babu A J <
>> vishwasbab...@gmail.com> escreveu:
>>
>>> Hello Sergio,
>>>
>>> Here is a pr https://github.com/apache/fineract/pull/515 which aimed to
>>> address this problem by relying on the client to optionally supply a unique
>>> id with each request. Maybe you can build on the same (it was missing test
>>> cases and api doc updates) and get it merged ?
>>>
>>> Regards,
>>> Vishwas
>>>
>>>
>>>
>>> On Mar 24, 2020, at 7:39 PM, Sergio Junior  wrote:
>>>
>>> Hello everyone,
>>>
>>> Im integrating a system with Fineract, and doing some tests with the
>>> Savings API (Deposit/Withdrawal functions), nothing is considered “Unique”,
>>> so if for any reason my system interpret a call as failed (maybe a reverse
>>> proxy fault, tcp issues etc) and perform a retry, Fineract will Credit (or
>>> Debit) a transaction multiple times, no field is considered an Identifier
>>> and i couldnt find any resource to notify the API that a transaction is a
>>> retry or a new one.
>>>
>>> Whats the idea for such scenario, to maintain consistency and avoid
>>> duplicates?
>>>
>>> Thanks!
>>>
>>>
>>>
>>
>

Re: Integration - Retry/Repeated/concurrent calls

[Vishwas Babu A J]

Sergio,

A quick clarification:

https://github.com/apache/fineract/pull/515 
 allows setting a globally unique 
client generated identifier for a command. This allows the client to later 
query for the status of the execution of the command associated with this guid 
if the response is lost in transit etc. At the server side, the guid is assumed 
to be unique.

By adding a unique constraint to the command source table, we can enforce the 
uniqueness of the client generated identifier at the server side, thereby 
ensuring duplicate requests are not served. However, this approach would likely 
result in a write penalty for very large installations.

Alternatives could be to check the presence of this key in an in memory key 
value store, something along the lines of setnx in Redis (assuming duplicates 
requests are sent in a fixed time interval). We could maybe handle this from an 
infrastructure level too, by carrying out this check at the API gateway,

Regards,
Vishwas



> On Mar 25, 2020, at 6:54 PM, Vishwas Babu  wrote:
> 
> >>Question, before this PR, no one have this kind of problem before?
> I am sure people would have faced this issue and figured out solutions like 
> this PR.
> 
> >>A simple retry or double click at any of the apps would cause issues, right?
> It might not have been so obvious as clients like the Mifos community-app 
> (most common UI used with Fineract) handle double clicks at their end. 
> However, your point is valid
> 
> Regards,
> Vishwas
> 
> 
> On Wed, 25 Mar 2020 at 07:19, Sergio Junior  > wrote:
> Vishwas,
> 
> I really appreciate your response, missed that PR because it was closed, i 
> will take a look.
> 
> Question, before this PR, no one have this kind of problem before? A simple 
> retry or double click at any of the apps would cause issues, right?
> 
> Thanks!
> 
> Em qua., 25 de mar. de 2020 às 01:24, Vishwas Babu A J 
> mailto:vishwasbab...@gmail.com>> escreveu:
> Hello Sergio,
> 
> Here is a pr https://github.com/apache/fineract/pull/515 
>  which aimed to address this 
> problem by relying on the client to optionally supply a unique id with each 
> request. Maybe you can build on the same (it was missing test cases and api 
> doc updates) and get it merged ?
> 
> Regards,
> Vishwas
> 
> 
> 
>> On Mar 24, 2020, at 7:39 PM, Sergio Junior > > wrote:
>> 
>> Hello everyone,
>> 
>> Im integrating a system with Fineract, and doing some tests with the Savings 
>> API (Deposit/Withdrawal functions), nothing is considered “Unique”, so if 
>> for any reason my system interpret a call as failed (maybe a reverse proxy 
>> fault, tcp issues etc) and perform a retry, Fineract will Credit (or Debit) 
>> a transaction multiple times, no field is considered an Identifier and i 
>> couldnt find any resource to notify the API that a transaction is a retry or 
>> a new one.
>> 
>> Whats the idea for such scenario, to maintain consistency and avoid 
>> duplicates?
>> 
>> Thanks!
> 
> 
> 
> -- 
> Atenciosamente,
> Sergio Luiz Miziara Junior
> Tel.: (011) 8727.6447

Re: Integration - Retry/Repeated/concurrent calls

[Sergio Junior]

Vishwas,

I really appreciate your response, missed that PR because it was closed, i
will take a look.

Question, before this PR, no one have this kind of problem before? A simple
retry or double click at any of the apps would cause issues, right?

Thanks!

Em qua., 25 de mar. de 2020 às 01:24, Vishwas Babu A J <
vishwasbab...@gmail.com> escreveu:

> Hello Sergio,
>
> Here is a pr https://github.com/apache/fineract/pull/515 which aimed to
> address this problem by relying on the client to optionally supply a unique
> id with each request. Maybe you can build on the same (it was missing test
> cases and api doc updates) and get it merged ?
>
> Regards,
> Vishwas
>
>
>
> On Mar 24, 2020, at 7:39 PM, Sergio Junior  wrote:
>
> Hello everyone,
>
> Im integrating a system with Fineract, and doing some tests with the
> Savings API (Deposit/Withdrawal functions), nothing is considered “Unique”,
> so if for any reason my system interpret a call as failed (maybe a reverse
> proxy fault, tcp issues etc) and perform a retry, Fineract will Credit (or
> Debit) a transaction multiple times, no field is considered an Identifier
> and i couldnt find any resource to notify the API that a transaction is a
> retry or a new one.
>
> Whats the idea for such scenario, to maintain consistency and avoid
> duplicates?
>
> Thanks!
>
>
>

-- 
Atenciosamente,
Sergio Luiz Miziara Junior
Tel.: (011) 8727.6447

Integration - Retry/Repeated/concurrent calls

[Sergio Junior]

Hello everyone,

Im integrating a system with Fineract, and doing some tests with the
Savings API (Deposit/Withdrawal functions), nothing is considered “Unique”,
so if for any reason my system interpret a call as failed (maybe a reverse
proxy fault, tcp issues etc) and perform a retry, Fineract will Credit (or
Debit) a transaction multiple times.

Whats the idea for such scenario, to maintain consistency and avoid
duplicates?

Thanks!

Re: Integration - Retry/Repeated/concurrent calls

[Vishwas Babu A J]

Hello Sergio,

Here is a pr https://github.com/apache/fineract/pull/515 
 which aimed to address this 
problem by relying on the client to optionally supply a unique id with each 
request. Maybe you can build on the same (it was missing test cases and api doc 
updates) and get it merged ?

Regards,
Vishwas



> On Mar 24, 2020, at 7:39 PM, Sergio Junior  wrote:
> 
> Hello everyone,
> 
> Im integrating a system with Fineract, and doing some tests with the Savings 
> API (Deposit/Withdrawal functions), nothing is considered “Unique”, so if for 
> any reason my system interpret a call as failed (maybe a reverse proxy fault, 
> tcp issues etc) and perform a retry, Fineract will Credit (or Debit) a 
> transaction multiple times, no field is considered an Identifier and i 
> couldnt find any resource to notify the API that a transaction is a retry or 
> a new one.
> 
> Whats the idea for such scenario, to maintain consistency and avoid 
> duplicates?
> 
> Thanks!

Integration - Retry/Repeated/concurrent calls

[Sergio Junior]

Hello everyone,

Im integrating a system with Fineract, and doing some tests with the Savings 
API (Deposit/Withdrawal functions), nothing is considered “Unique”, so if for 
any reason my system interpret a call as failed (maybe a reverse proxy fault, 
tcp issues etc) and perform a retry, Fineract will Credit (or Debit) a 
transaction multiple times, no field is considered an Identifier and i couldnt 
find any resource to notify the API that a transaction is a retry or a new one.

Whats the idea for such scenario, to maintain consistency and avoid duplicates?

Thanks!