David, how about waking up my second map reduce job as soon as I see some rows updated in that table. any thoughts on observing a column update?
thanks, Vishal On Fri, Mar 25, 2011 at 2:56 PM, Buttler, David <[email protected]> wrote: > What about just storing some metadata in a special table? > Then on you second job startup you can read that meta data and set your scan > /input splits appropriately? > Dave > > -----Original Message----- > From: Vishal Kapoor [mailto:[email protected]] > Sent: Friday, March 25, 2011 11:21 AM > To: [email protected] > Subject: Observer/Observable MapReduce > > Can someone give me a direction on how to start a map reduce based on > an outcome of another map reduce? ( nothing common between them apart > from the first decides about the scope of the second. > > I might also want to set the scope of my second map reduce > (from/after) my first map reduce(scope as in scan(start,stop) > > typically data comes in a few tables for us and we start crunching it > and then adding some more data to man tables like info etc to get rid > of table joins. > > a light weight framework will do better than a typical workflow management > tool. > > thanks, > Vishal Kapoor >
