Hi Lohit, Thank you for the response. If jobs cannot be recursively launched, does it mean that I have to pre-process all the catalogs and putting only filenames into a sequence file and then use hadoop from that point on?
-- Jim On 10/29/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > >Can I recursively launch a job within the map() method of a mapper? > > Interesting. If your launch is blocking, then I am thinking possibility of > deadlock. Lets say you have N nodes and your initial map tasks are scheduled > on all N nodes. Now if each of the map tasks launch few more jobs, then the > new ones are sitting there waiting for free nodes. > > Thanks, > Lohit > ----- Original Message ---- > From: Jim the Standing Bear <[EMAIL PROTECTED]> > To: [email protected] > Sent: Monday, October 29, 2007 1:28:17 PM > Subject: can jobs be launched recursively within a mapper ? > > > Hi, > > Can I recursively launch a job within the map() method of a mapper? > The task i am facing involves extracting metadata for filenames and > catalog names from hierarchical catalogs, and for each catalog name > extract, I must recursively do the same thing until no more catalog > names present. Therefore, it will be convenient if I can launch a job > within the map(), but I want to make certain that it will work before > investing too much efforts into the coding. > > If it won't work, could you suggest the proper way to go about doing > so? Much appreciated in advance. > > -- Jim > > > > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com > -- -------------------------------------- Standing Bear Has Spoken --------------------------------------
