I agree with Inder. We need a graceful shutdown which will shutdown agent only after the channel has been fully drained.
Tejinder On Mon, Jul 9, 2012 at 12:02 AM, Inder Pall <[email protected]> wrote: > Arvind, > > to me this is an important use-case for frequent prod rollouts. How about > thinking in the direction of supporting graceful shutdown for agents. > > I can't think of an elegant solution at the moment which will address all > cases however what are thoughts regarding something on the lines -> > > 1. agent receives a shutdown signal. > 2. puts all it's channel in isolation mode(wherein no sources can put > stuff into it.) > 3. when the sinks attached to this channel drain( we do the real shutdown). > > i know we can find issues with this algo however i want to highlight the > importance of graceful shutdown being supported as a first class use-case > here. > > - Inder > > > On Mon, Jul 9, 2012 at 12:16 PM, alo alt <[email protected]> wrote: > >> Simple solution: >> >> Two configs on different ports, iptables with transparent forwarding to >> both ports. Block the first one, all events will be redirected to the other >> port. Wait 5 minutes, the mem channel should be clear now. Do you changes, >> start the new config, redirect the traffic to these port and change the >> other config. >> >> cheers, >> alex >> >> >> On Jul 9, 2012, at 8:29 AM, Arvind Prabhakar wrote: >> >> > Hi, >> > >> > On Sun, Jul 8, 2012 at 11:18 PM, Senthilvel Rangaswamy < >> [email protected] >> >> wrote: >> > >> >> We are using Flume 1.2.0 with memory channel. When we rollout new >> >> configs/decorators >> >> we may need to restart flume at which point any events in memory >> channel >> >> is gone. Any >> >> ways to avoid this ? >> >> >> > >> > One way to address this would be to make sure that the upstream sink or >> > client can be routed to a different agent when necessary. That way when >> you >> > do want to restart the file channel, you would first route all the >> traffic >> > elsewhere, drain the channel and then do the shutdown as necessary. Once >> > the system is back up, you could route the traffic back to this agent. >> > >> > I am sure that there are multiple other ways of doing this. >> > >> > Regards, >> > Arvind Prabhakar >> > >> > >> > >> >> >> >> Thanks, >> >> -- >> >> ..Senthil >> >> >> >> "If there's anything more important than my ego around, I want it >> >> caught and shot now." >> >> - Douglas Adams. >> >> >> >> >> >> >> -- >> Alexander Alten-Lorenz >> http://mapredit.blogspot.com >> German Hadoop LinkedIn Group: http://goo.gl/N8pCF >> >> > > > -- > Thanks, > - Inder > Tech Platforms @Inmobi > Linkedin - http://goo.gl/eR4Ub > -- *Tejinder Aulakh *Senior Software Engineer, ShareThis *e:* [email protected] *m:* 510.708.2499* * ** <http://pinterest.com/sharethis/>* <http://sharethis.com/> **Learn More:* SQI (Social Quality Index) - A Universal Measure of Social Quality <http://sharethis.com/sqi> [image: Facebook] <http://www.facebook.com/sharethis> [image: Twitter]<https://twitter.com/#!/SHARETHIS> [image: LinkedIn]<http://www.linkedin.com/company/207839?trk=pro_other_cmpy> [image: pinterest] <http://pinterest.com/sharethis/>
