To partially answer my own question, some of the thought behind the use of RabbitMQ is captured by a class project described here: https://github.com/airavata-courses/spring17-microservice-data-management/wiki. We can also dig up the Airavata dev list postings by Gourav Shenoy and Ajinkya Dhamnaskar about the Airavata implementation.
Marlon On 6/10/19, 12:47 PM, "Pierce, Marlon" <[email protected]> wrote: Hi Aarushi, Thanks for the nice problem description. It may also be helpful to understand why the sharing and profile services communicate over RabbitMQ while the others are direct Thrift calls. Marlon On 6/10/19, 10:26 AM, "Bisht, Aarushi" <[email protected]> wrote: This message was sent from a non-IU address. Please exercise caution when clicking links or opening attachments from external sources. ------- ?Hi All, For those of you who don't know me, I am a GA in IU Bloomington, working on the Custos project. Since the past week I have been working on migrating Sharing Service as a standalone Custos service. I have been trying to remove all dependencies of airavata from sharing registry service. So far I am able to start it as an independent service but there are few code dependencies on airavata's utility modules. On investigating these dependencies I found out few things and would like to get advice on some decisions. The sharing registry service is being used by the following services in airavata: 1. Group Manager service 2. Services-security 3. Airavata-api-Server 4.Tenant Profile service 5. User Profile service 5. Registry server All the above services are communicating with the sharing service through a thrift client except for user/tenant profile service and registry service. The communication between profile service and sharing service is done asynchronously through RabbitMQ . When a new user or tenant is registered, a message is added to the PROFILE / TENANT queue which is then consumed by the sharing service and corresponding database entries are done in sharing database. Similarly registry service consumes the message from PROFILE queue and sends a message to PROJECT queue to create a default project for the new user account. This message is consumed by the sharing registry service. I want to seek advice on how the communication with the standalone sharing registry service should be handled in future so that it aligns with custos project goals. There are couple of approaches that I can think of: 1. Thrift API calls Any communication with sharing service will be done only through a thrift client. This will involve making changes to registry and profile services. This makes the call synchronous, and if required, the consuming services will need to handle asynchronous calls and error handling. 2. Using RabbitMQ ?The advantage of using this approach is that communication will be asynchronous but the services consuming sharing service will have to manage RabbitMQ connection. 3. Expose Thrift API call which internally use RabbitMQ for asynchronous communication This will involve adding API call in sharing service which can be used by other services to add a message to the sharing service queue. The advantage of this approach is that other services consuming sharing service will not have to manage RabbitMQ connection and the communication will still be asynchronous (other than the call to push request in RabbitMQ) The decision mainly boils down to choosing between synchronous and asynchronous approach. Asynchronous will mean faster response time for APIs, with each service responsible for handling and resolving its own errors. But this also means that there might be few seconds delay in processing of requests as these are asynchronous. On other hand, synchronous approach will ensure that all requests are processed immediately, which increases the API response time but also makes it tricky to handle failure edge cases as if any request fails, the corresponding DB entries needs to be rolled back which may include another thrift call (if entries were made via thrift). If rollback fails due to some reason, there is no easy way to ensure data consistency, though this is a edge case scenario. Thanks & Regards, Aarushi Bisht
smime.p7s
Description: S/MIME cryptographic signature
