Hi Dinuka, I will reply inline
On Fri, Apr 17, 2020 at 12:37 PM Dinuka Desilva <l.dinukadesi...@gmail.com> wrote: > Hi, > > I'm currently looking in to convert the existing java implementation of > agent module to golang considering better concurrency, performance and make > it more platform enabled. Before just starting up, I'm trying to > discuss which kind of changes has to be done on which end. > > 1) I believe we are considering to accommodate multiple agents available > per controller and each agent to support multiple transports. Which means, > it's a collection of agents connected to a one controller. I think this is > not done yet. > > There should be only one active controller for entire system. Controller is the master and agents are the workers. This follows the master slave architecture. Controller sends messages to agents when a transfer needs to be performed. And this is already in place. 2) Currently my understanding according to the implementation is that the > agent module is kind of coupled with the api and controller. I feel that > agent has to be an independent module. Or am I missing something? > > Agent is not coupled with the controller. But it has a standard messaging API built on top of Consul. If controller needs to get some work done by the agent, it has to communicate in that particular messaging API. But there is no code level dependency between the controller and agents. However Agent has dependencies with Resource and Secret APIs to fetch metadata of the resources. This is also not with the Service but with the APIs. > 3) The agent being independent is required for the agent to be converted > to golang too. > > Yes agent has to be independent from the code level with other components of the framework but it should have API level dependencies to communicate with the rest of the system. These are implemented on top of protobuf for Resource and Secret services which can be easily built for golang and consul level messaging dependencies with the controller which can be fulfilled through a golang consul client or just following consul HTTP API. 4) I was also thinking whether the combination of transports available per > agent also should be independently installable. So, that each agent could > decide which transports are required. Plus they could introduce new > transport plugins. (A kind of an enhancement) > +1 and this is a good feature to have in agents. Agent have a feature [1] to inform controller about supported protocols but this not still validated from the controller when submitting a transfer job. We need to implement this feature [1] https://github.com/apache/airavata-mft/blob/master/agent/src/main/java/org/apache/airavata/mft/agent/MFTAgent.java#L245 > > 5) I'm just doubt about the "LOCAL" transport. how does it work with > multiple agents. According to the architecture where does the local storage > exist? Is it inside the agent? > Local resource is attached to the Agent. If and Agent receives a local resource transfer request, it looks at the local file system of the machine where it was installed. > Regards, > Dinuka >
