Sudharma, Any updates?
regards, sathwik On Fri, May 29, 2015 at 5:26 PM, Sathwik B P <sathwik...@gmail.com> wrote: > Sudharma, > > Can you elaborate on your option 1). > > Response to your option 2). > > Process Store is the component that handles process metadata, > compilation and deployment in ODE. Integration layers in ODE (Axis2, JBI) > use the process store. > Future implementations of IL for ODE will also use the process store. > We should not be thinking of moving the process store functionality to the > integration layers. > > > On Thu, May 28, 2015 at 9:33 PM, sudharma subasinghe < > suba...@cse.mrt.ac.lk> wrote: > >> Hi, >> >> I understood the problem within dynamic master/slave configuration. In my >> approach, when a deployment request is routed to a slave node there will >> not be a deployment. I suggest two options to avoid it. >> 1) Have static master/slave configuration only for deploy process >> > 2) Modify the deployment web service to complie and verify the process and >> then copy it to the deploy folder irrespective of whether its a master or >> slave, then deployment poller should take care of the deployment >> >> > >> >> On 28 May 2015 at 14:43, Sathwik B P <sathwik...@gmail.com> wrote: >> >> > Sudharma, >> > >> > We definitely need a master/slave in the hazelcast cluster. This is >> > probably needed for the job migration in the Scheduler to migrate the >> jobs >> > associated with a down node. Let hold on this topic for future >> discussion. >> > >> > Going by the explanation where the master/slave nodes have certain >> > predefined tasks to perform is perfectly fine. >> > >> > I have this scenario, >> > >> > I am using HAProxy as my load balancer and configured 3 nodes in the >> > cluster. >> > >> > Node1 - Active >> > Node2 - Active >> > Node3 - Backup >> > >> > Load balancing algorithm: RoundRobin >> > >> > A Backup node (Node3) is one which the load balancer will not route >> > requests to, until one of the Active node i.e either Node1 or Node2 has >> > gone down. >> > >> > All these 3 nodes are also part of the hazelcast cluster as well. >> > >> > In the hazelcast cluster, assume Node1 is elected as the leader/master >> and >> > Node2,Node3 as slaves. >> > >> > I initiate the deploy operation on the DeploymentWebService which the >> load >> > balancer routes it to one of the Active nodes in the cluster, lets say >> it's >> > the Node1. Since Node1 is also the master in the hazelcast cluster, >> > deployment is a success. >> > >> > I initiate another deploy operation on the DeploymentWebService which >> the >> > load balancer routes it to the next active node which is Node2. Since >> Node2 >> > is a slave in the Hazelcast cluster, What happens to the deployment? >> > >> > regards, >> > sathwik >> > >> > On Wed, May 27, 2015 at 10:55 PM, sudharma subasinghe < >> > suba...@cse.mrt.ac.lk >> > > wrote: >> > >> > > Hi, >> > > >> > > I will explain my approach as much as possible. The oldest node in the >> > > hazelcast cluster is elected as the master node. In the failure of the >> > > master node, next oldest node will be elected as the master node. This >> > > master-slave configuration is just for deployment. When the hazelcast >> > > cluster elected the master node, that node becomes a master node for >> > > deploying process. So it will do the deploying artifacts. If you want >> to >> > > get the idea of electing master node please refer the code which I >> have >> > > located in the github. ( >> > > https://github.com/Subasinghe/ode/tree/ode_clustering) >> > > >> > > I identified separated actions which should be followed by the master >> and >> > > salve nodes. >> > > Actions which are followed by master node only >> > > 1) create deployment unit >> > > 2) set the version nu to deployment unit >> > > 3) compile deployment unit >> > > 4) scan deployment unit >> > > 5) retire previous versions >> > > Master node and slave nodes should create _processes which stores >> > > ProcessConfImpl >> > > Only master node will write the version nu to database, create >> .deployed >> > > file >> > > >> > > So there are some actions which should be followed only by master node >> > > while other actions should be followed by all the nodes.The idea of >> > having >> > > a master node is deploying artifacts and avoid others from writing the >> > > version nu to database. >> > > Whether a node is active or passive, all nodes should do the >> > > deployment.Master >> > > and slaves will follow necessary actions as in above. >> > > >> > > >> > > >> > > >> > > >> > > On 27 May 2015 at 15:49, Sathwik B P <sathwik...@gmail.com> wrote: >> > > >> > > > Nandika, >> > > > >> > > > I very well understand what you have put across, but it's secondary >> to >> > me >> > > > now. >> > > > >> > > > Sudharma, >> > > > My primary concern is to understand at a high level the deployment >> > > > architecture and how would master-slave configuration fit in. Are >> there >> > > any >> > > > restrictions imposed by the in-progress design? >> > > > >> > > > Firstly, how would ODE process deployment work under these cluster >> > > > configurations? >> > > > >> > > > Sample Cluster configurations: A load balancer is frontending the >> > > servers. >> > > > 1) Cluster consisting of 2 nodes all Active-Active. >> > > > 2) Cluster consisting of 2 nodes Active-Passive. >> > > > 3) Cluster with 2+ nodes with additional nodes either in Active or >> > > Passive. >> > > > >> > > > regards, >> > > > sathwik >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > On Wed, May 27, 2015 at 3:04 PM, Nandika Jayawardana < >> > jayaw...@gmail.com >> > > > >> > > > wrote: >> > > > >> > > > > Hi Sathwik, >> > > > > >> > > > > According to my understanding, in the clustering scenario, the >> master >> > > > node >> > > > > should perform all the deployment actions and the slave nodes also >> > need >> > > > to >> > > > > perform some deployment actions. For example, the slave nodes also >> > > should >> > > > > handle the process ACTIVATED event so that the process >> configuration >> > is >> > > > > added to the engine and necessary web services are created so that >> > when >> > > > the >> > > > > load balancer send requests to any node in the cluster, it is >> ready >> > to >> > > > > accept those requests. >> > > > > >> > > > > Regards >> > > > > Nandika >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > On Wed, May 27, 2015 at 12:30 PM, Sathwik B P < >> sathwik...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Sudharma, >> > > > > > >> > > > > > Where are you going to configure the master-slaves, is it in the >> > web >> > > > > > application level or at the load balancer? >> > > > > > >> > > > > > regards, >> > > > > > sathwik >> > > > > > >> > > > > > On Tue, May 26, 2015 at 7:42 PM, sudharma subasinghe < >> > > > > > suba...@cse.mrt.ac.lk> >> > > > > > wrote: >> > > > > > >> > > > > > > Hi Tammo, >> > > > > > > >> > > > > > > Can you suggest the best method from these to implement? As >> > first I >> > > > > > > suggested the master-slaves scenario I think it is easy to >> > > implement >> > > > > than >> > > > > > > distributed lock scenario. However if you can suggest one from >> > > these >> > > > > two, >> > > > > > > then I can think about it. >> > > > > > > >> > > > > > > Thank you >> > > > > > > >> > > > > > > On 21 May 2015 at 12:40, Sathwik B P <sathwik...@gmail.com> >> > wrote: >> > > > > > > >> > > > > > > > With respect to the hotdeployment, >> > > > > > > > >> > > > > > > > We can drop the deployment archive onto the deployment >> folder. >> > > > Since >> > > > > > the >> > > > > > > > DeploymentPoller are acquiring the distributed lock for the >> > > > > > > DeploymentUnit, >> > > > > > > > only one of the nodes will get the lock and initiate the >> > > > deployment. >> > > > > > > > DeploymentPollers on other nodes will fail in acquiring the >> > lock >> > > > and >> > > > > > > hence >> > > > > > > > will silently ignore it. >> > > > > > > > >> > > > > > > > On Thu, May 21, 2015 at 12:30 PM, Sathwik B P < >> > > > sathwik...@gmail.com> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > Hi Tammo, >> > > > > > > > > >> > > > > > > > > The distributed lock acquisition on the DeploymentUnit >> should >> > > be >> > > > > > added >> > > > > > > to >> > > > > > > > > both DeploymentWebService and DeploymentPoller. >> > > > > > > > > >> > > > > > > > > When a deployment operation is initiated through the >> > > > > > > > DeploymentWebService, >> > > > > > > > > The load balancer routes it to any of the available nodes. >> > > > > > > > > >> > > > > > > > > On the routed node, the DeploymentWebService acquires the >> > > > > Distributed >> > > > > > > > > lock. On the remaining nodes the DeploymentPoller will >> try to >> > > > > acquire >> > > > > > > the >> > > > > > > > > distributed lock and will not get it and hence will >> silently >> > > > ignore >> > > > > > it. >> > > > > > > > > >> > > > > > > > > Once the routed node completes the deployment, it will >> > release >> > > > the >> > > > > > > lock. >> > > > > > > > > This way we don't have to stall the DeploymentPoller in >> other >> > > > > nodes. >> > > > > > > > > >> > > > > > > > > Does it answer the concerns? >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > Now, if we give the responsibility of identifying the >> master >> > > node >> > > > > to >> > > > > > > the >> > > > > > > > > hazelcast, how do we plan to intimate the load balancer to >> > > change >> > > > > > it's >> > > > > > > > > configuration about the master node? >> > > > > > > > > Assuming there are 3 nodes in the cluster, >> > > > > > > > > node1 -master >> > > > > > > > > node2 - slave >> > > > > > > > > node3 - slave >> > > > > > > > > >> > > > > > > > > Node1 goes down, the LB will promote Node2 as master node, >> > but >> > > > > > > hazelcast >> > > > > > > > > might promote Node3 as master node. They are out of sync. >> > > > > > > > > >> > > > > > > > > Is this argument valid? >> > > > > > > > > >> > > > > > > > > regards, >> > > > > > > > > sathwik >> > > > > > > > > >> > > > > > > > > On Wed, May 20, 2015 at 1:51 PM, Tammo van Lessen < >> > > > > > > tvanles...@gmail.com> >> > > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > >> Hi Sudharma, >> > > > > > > > >> >> > > > > > > > >> what do you expect from the "other nodes deployment"? >> > > > Compilation >> > > > > is >> > > > > > > not >> > > > > > > > >> needed since the CBP file is written to the (shared) FS. >> > > > > > Registration >> > > > > > > is >> > > > > > > > >> also not needed, since it is done via the shared >> database. >> > So >> > > > the >> > > > > > only >> > > > > > > > >> thing that might be needed is to tell the engine that >> there >> > > is a >> > > > > new >> > > > > > > > >> deployment. I'd need to check that. If this is needed, I >> > > revert >> > > > my >> > > > > > > last >> > > > > > > > >> statement, then it is perhaps better to just send an >> event >> > > over >> > > > > > > > Hazelcast >> > > > > > > > >> to all nodes that the deployment has changed. >> > > > > > > > >> >> > > > > > > > >> Best, >> > > > > > > > >> Tammo >> > > > > > > > >> >> > > > > > > > >> On Wed, May 20, 2015 at 10:13 AM, sudharma subasinghe < >> > > > > > > > >> suba...@cse.mrt.ac.lk >> > > > > > > > >> > wrote: >> > > > > > > > >> >> > > > > > > > >> > Hi Tammo, >> > > > > > > > >> > >> > > > > > > > >> > The master node writes meta data. But runtime >> information >> > > must >> > > > > be >> > > > > > > > >> available >> > > > > > > > >> > in all nodes.Since the folder is shared, all nodes will >> > see >> > > > the >> > > > > > > > >> > availability of a new process. My idea is for master >> node >> > to >> > > > > write >> > > > > > > the >> > > > > > > > >> meta >> > > > > > > > >> > data and other nodes to just read the meta data and >> load >> > > > > > process.So >> > > > > > > we >> > > > > > > > >> need >> > > > > > > > >> > a small delay between master node deployment and other >> > nodes >> > > > > > > > deployment. >> > > > > > > > >> > >> > > > > > > > >> > Is there anyway to set the delay between master node >> and >> > > > slaves >> > > > > > > until >> > > > > > > > >> > master node finish the deployment? >> > > > > > > > >> > >> > > > > > > > >> > Thank you >> > > > > > > > >> > Sudharma >> > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > On 20 May 2015 at 13:01, Tammo van Lessen < >> > > > tvanles...@gmail.com >> > > > > > >> > > > > > > > wrote: >> > > > > > > > >> > >> > > > > > > > >> > > Hi Sathwik, >> > > > > > > > >> > > >> > > > > > > > >> > > On Wed, May 20, 2015 at 6:40 AM, Sathwik B P < >> > > > > > > sathwik...@gmail.com> >> > > > > > > > >> > wrote: >> > > > > > > > >> > > >> > > > > > > > >> > > > Sudharma/Tammo, >> > > > > > > > >> > > > >> > > > > > > > >> > > > 1) How do we plan to decide which is the master >> node >> > in >> > > > the >> > > > > > > > cluster? >> > > > > > > > >> > > > >> > > > > > > > >> > > >> > > > > > > > >> > > I think the easiest approach is to always elect the >> > oldest >> > > > > node >> > > > > > in >> > > > > > > > the >> > > > > > > > >> > > cluster to be the master. AFAIK Hazelcast can easily >> > asked >> > > > for >> > > > > > > this >> > > > > > > > >> > > information. >> > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > > >> > > > 2) Don't we need to stall the Deployment Pollers in >> > the >> > > > > slave >> > > > > > > > nodes? >> > > > > > > > >> > > > >> > > > > > > > >> > > > >> > > > > > > > >> > > Absolutely. >> > > > > > > > >> > > >> > > > > > > > >> > > Suggestion: >> > > > > > > > >> > > > I am not sure whether do we need Master-SLaves. Why >> > not >> > > > give >> > > > > > > every >> > > > > > > > >> node >> > > > > > > > >> > > in >> > > > > > > > >> > > > the cluster the same status (Active-Active). >> > > > > > > > >> > > > >> > > > > > > > >> > > > When a new deployment is made, the load balancer >> can >> > > push >> > > > it >> > > > > > to >> > > > > > > > any >> > > > > > > > >> of >> > > > > > > > >> > > the >> > > > > > > > >> > > > available nodes. That node will probably acquire a >> > > > > distributed >> > > > > > > > lock >> > > > > > > > >> on >> > > > > > > > >> > > the >> > > > > > > > >> > > > deployment unit and acts as master for that >> > deployment. >> > > > This >> > > > > > > > ensures >> > > > > > > > >> > > > optimum usage of the cluster nodes. Probably no >> static >> > > > > > > > >> configuration of >> > > > > > > > >> > > > Master-Slave in the load balancer nor in the >> > hazelcast. >> > > > > > > > >> > > > >> > > > > > > > >> > > >> > > > > > > > >> > > But this would not allow to have the hotdeployment >> via >> > > > > > filesystem >> > > > > > > > >> still >> > > > > > > > >> > > enabled, right? >> > > > > > > > >> > > >> > > > > > > > >> > > Best, >> > > > > > > > >> > > Tammo >> > > > > > > > >> > > >> > > > > > > > >> > > >> > > > > > > > >> > > -- >> > > > > > > > >> > > Tammo van Lessen - http://www.taval.de >> > > > > > > > >> > > >> > > > > > > > >> > >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> -- >> > > > > > > > >> Tammo van Lessen - http://www.taval.de >> > > > > > > > >> >> > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> > >
