Thank you Owen and Aaron, I will spawn a sub process and handle third party errors!
-- Ashwin! > Date: Tue, 9 Aug 2011 18:11:40 +0000 > Subject: Re: Skipping Bad Records in M/R Job > From: [email protected] > To: [email protected] > > On Tue, Aug 9, 2011 at 5:28 PM, Maheshwaran Janarthanan < > [email protected]> wrote: > > > > > Hi, > > > > I have written a Map reduce job which uses third party libraries to process > > unseen data which makes job fail because of errors in records. > > > > I realized 'Skipping Bad Records' feature in Hadoop Map/Reduce. Can Anyone > > send me the code snippet which enables this feature by setting properties on > > JobConf > > > > I wouldn't recommend using the bad record skipping, since it was always > experimental and I don't think it has been well maintained. > > If your 3rd part library crashes the jvm, I'd suggest using a subprocess to > call it and handle the errors yourself. > > -- Owen
