Thanks I will look into logs to see if I see anything else… sanjay
From: Colin McCabe <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, July 10, 2013 11:52 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Re: EBADF: Bad file descriptor To clarify a little bit, the readahead pool can sometimes spit out this message if you close a file while a readahead request is in flight. It's not an error and just reflects the fact that the file was closed hastily, probably because of some other bug which is the real problem. Colin On Wed, Jul 10, 2013 at 11:50 AM, Colin McCabe <[email protected]<mailto:[email protected]>> wrote: That's just a warning message. It's not causing your problem-- it's just a symptom. You will have to find out why the MR job failed. best, Colin On Wed, Jul 10, 2013 at 8:19 AM, Sanjay Subramanian <[email protected]<mailto:[email protected]>> wrote: 2013-07-10 07:11:50,131 WARN [Readahead Thread #1] org.apache.hadoop.io.ReadaheadPool: Failed readahead on ifile EBADF: Bad file descriptor at org.apache.hadoop.io.nativeio.NativeIO.posix_fadvise(Native Method) at org.apache.hadoop.io.nativeio.NativeIO.posixFadviseIfPossible(NativeIO.java:145) at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:205) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Hi I have a Oozie workflow that runs a MR job and I have started getting this error past two days in one of the MR jobs that is being processed. However if I run it again , it succeeds :-( but about 1 hr is wasted in the process. Any clues ? Or should I post this issue in the Oozie postings ? Thanks sanjay Configuration Name Value impression.log.record.cached.tag cached= impression.log.record.end.tag [end impressions.mapreduce.conf.file.full.path /workflows/impressions/config/aggregations.conf<http://thv01:8888/filebrowser/view/workflows/impressions/config/aggregations.conf> mapred.job.queue.name<http://mapred.job.queue.name> default mapred.mapper.new-api true mapred.reducer.new-api true mapreduce.input.fileinputformat.inputdir /data/input/impressionlogs/outpdirlogs/9999-99-99<http://thv01:8888/filebrowser/view/data/input/impressionlogs/outpdirlogs/9999-99-99> mapreduce.job.inputformat.class com.wizecommerce.utils.mapred.ZipMultipleLineRecordInputFormat mapreduce.job.map.class com.wizecommerce.parser.mapred.OutpdirImpressionLogMapper mapreduce.job.maps 500 mapreduce.job.name<http://mapreduce.job.name> OutpdirImpressions_0000475-130611151004460-oozie-oozi-W mapreduce.job.output.value.class org.apache.hadoop.io.Text mapreduce.job.outputformat.class com.wizecommerce.utils.mapred.NextagTextOutputFormat mapreduce.job.reduce.class com.wizecommerce.parser.mapred.OutpdirImpressionLogReducer mapreduce.job.reduces 8 mapreduce.map.output.compress true mapreduce.map.output.compress.codec org.apache.hadoop.io.compress.SnappyCodec mapreduce.map.output.key.class org.apache.hadoop.io.Textorg.apache.hadoop.io.Text mapreduce.map.output.value.class com.wizecommerce.parser.dao.OutpdirLogRecord mapreduce.output.fileoutputformat.compress true mapreduce.output.fileoutputformat.compress.codec com.hadoop.compression.lzo.LzopCodec mapreduce.output.fileoutputformat.outputdir /data/output/impressions/outpdir/9999-99-99/0000475-130611151004460-oozie-oozi-W/outpdir_impressions_ptitle<http://thv01:8888/filebrowser/view/data/output/impressions/outpdir/9999-99-99/0000475-130611151004460-oozie-oozi-W/outpdir_impressions_ptitle> mapreduce.tasktracker.map.tasks.maximum 12 mapreduce.tasktracker.reduce.tasks.maximum 8 outpdir.log.exclude.processing.datatypes header,sellerhidden CONFIDENTIALITY NOTICE ====================== This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator. CONFIDENTIALITY NOTICE ====================== This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
