I was thinking exactly in that direction - having two states is the right thing to do. Before we get there, one more question -
- when you get a request for a job, how do you know if that job is old or new? Is there a database that provides the mapping between job and node On Mon, May 11, 2020 at 12:44 PM santosh gujar <[email protected]> wrote: > Thank You Kishore, > > During drain process N2 will start new jobs, the requests related to old > jobs need to go to N1 and requests for new jobs need to go to N2. Thus > during drain on N1, the partition could be present on both nodes. > > My current thinking is that in helix somehow i need to model is > as Partition P with two different states on these two nodes. . e.g. N1 > could have partition P in Drain State and N2 can have partition P in > START_UP state. > I don't know if my thinking about states is correct, but looking for any > pointers. > > Regards > Santosh > > On Tue, May 12, 2020 at 1:01 AM kishore g <[email protected]> wrote: > >> what happens to request during the drain process i.e when you put N1 out >> of service and while N2 is waiting for N1 to finish the jobs, where will >> the requests for P go to - N1 or N2 >> >> On Mon, May 11, 2020 at 12:19 PM santosh gujar <[email protected]> >> wrote: >> >>> Hello, >>> >>> I am looking for some clues or inputs on how to achieve following >>> >>> I am working on a service that involves running a statetful long running >>> jobs on a node. These long running jobs cannot be preempted and continue on >>> other nodes. >>> >>> Problem Requirements : >>> 1. In helix nomenclature, I let's say an helix partition P that involves >>> J number of such jobs running on a node. (N1) >>> 2. When I put the node in a drain, I want helix to assign a new node to >>> this partition (P) is also started on the new node (N2). >>> >>> 3. N1 can be put out of service only when all running jobs (J) on it are >>> over, at this point only N2 will serve P request. >>> >>> Questions : >>> 1. Can drain process be modeled using helix? >>> 2. If yes, Is there any recipe / pointers for a helix state model? >>> 3. Is there any custom way to trigger state transitions? From >>> documentation, I gather that Helix controller in full auto mode, triggers >>> state transitions only when number of partitions change or cluster changes >>> (node addition or deletion) >>> 3.I guess spectator will be needed, to custom routing logic in such >>> cases, any pointers for the the same? >>> >>> Thank You >>> Santosh >>> >>
