On Nov 24, 2012 9:23 PM, "Roman Shaposhnik" <[email protected]> wrote: > > On Sat, Nov 24, 2012 at 1:42 PM, Jos Backus <[email protected]> wrote: > > On Nov 23, 2012 3:21 AM, "Steve Loughran" <[email protected]> wrote: > >> > >> I've opened a JIRA on the idea of having the hadoop processes optionally > >> create exclusive write-locked files on startup, failing if the lock > >> couldn't be acquired > >> > >> https://issues.apache.org/jira/browse/HADOOP-9086 > >> > >> I have to plans to implement this right now -but we all know that the > >> current pid file approach is flawed. > >> > >> -steve > > > > Ugh. Why not use a real process supervisor like daemontools or Upstart? Why > > duplicate this type of code in every application, poorly? The process table > > already tracks this information, without any races or extra files needed. > > Here's what I'd like to see happen for Bigtop 0.6.0 (scheduled for the end > of Q1 2013): > #1 we should totally migrate to upstart/systemd for all the platforms that > allow that > #2 we should figure out whether we still care about things like CentOS5 > and if we do we should do enough systemd/daemontools backporting > to make it work. > > I expect to be very much involved in #1. In fact, looking at the test results > for Quetazal/OpenSUSE12.2 we actually might not have much choice but > to implement this for Bigtop 0.5.0 (more on this next week). > > Thoughts?
Fwiw, at $work we already run Hadoop under daemontools in a few places, and going forward it will be the standard way of running Hadoop. It works great! Cheers, Jos > Thanks, > Roman.
