Mridul, Even then , each mapper would finish very fast as it wont be loading data after it finds an EOF.
Thanks, Souri On Tue, May 3, 2011 at 1:12 AM, Mridul Muralidharan <[email protected]>wrote: > > This would still be on a single mapper - not all mappers : which, I > suppose, is the intention ? > I was pretty confused by this requirement (still am actually :) ). > > > Regards, > Mridul > > > On Tuesday 03 May 2011 01:04 AM, Dmitriy Ryaboy wrote: > >> Ok, so you don't want to exit from a UDF, you want to stop processing the >> mapper. >> >> What you want is an early EOF from your InputFormat. So I guess the answer >> is to have a custom inputFormat that monitors some object that your UDF >> can >> modify, and the input format can report EOF if the condition is satisfied >> (or it's actually out of data). >> >> D >> >> >> On Sun, May 1, 2011 at 11:11 PM, souri datta<[email protected] >> >wrote: >> >> I think I was not able to explain my problem here. >>> >>> So, the udf runs in a mapper job which is doing some very expensive i/o >>> operation. If it processes all the input files, the time consumed will be >>> huge. So, I want to exit from the mapper on meeting the particular >>> condition >>> so that I don't process all the input files. >>> >>> Thanks, >>> Souri >>> >>> On Mon, May 2, 2011 at 1:46 AM, Dmitriy Ryaboy<[email protected]> >>> wrote: >>> >>> Right. I assume there is a reason you don't want to or are unable to >>>> have >>>> your udf check your condition and call return? >>>> >>>> -----Original Message----- >>>> From: "souri datta"<[email protected]> >>>> To: [email protected] >>>> Sent: 5/1/2011 8:50 AM >>>> Subject: Re: Early exit from Pig udf >>>> >>>> Meaning it should be able to finish quickly(return from the method). >>>> >>>> On Fri, Apr 29, 2011 at 9:52 PM, Dmitriy Ryaboy<[email protected]> >>>> wrote: >>>> >>>> What do you mean by return? >>>>> >>>>> >>>>> On Fri, Apr 29, 2011 at 5:01 AM, souri datta<[email protected] >>>>> >>>>>> wrote: >>>>>> >>>>> >>>>> Hi, >>>>>> I have a pig udf.My requirement is , on meeting certain criteria, I >>>>>> >>>>> want >>>> >>>>> to >>>>>> return from Pig udf.Is there any way I can early exit from Pig udf? >>>>>> Also, how can it be done in a Map/Reduce job? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Souri >>>>>> >>>>>> >>>>> >>>> >>>> >>> >
