I have check it. As datanode logs shown that,
2013-07-16 00:05:31,294 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201307041810_0138_m_000259_0,53) failed : org.mortbay.jetty.EofException: timeout This may be caused by a so-called “data skew” problem. Thanks, Devaraj k. 发件人: Devaraj k [mailto:[email protected]] 发送时间: 2013年7月16日 15:37 收件人: [email protected] 主题: RE: hive task fails when left semi join Hi, In the given image, I see there are some failed/killed map& reduce task attempts. Could you check why these are failing, you can check further based on the fail/kill reason. Thanks Devaraj k From: [email protected] [mailto:[email protected]] Sent: 16 July 2013 12:57 To: [email protected] Subject: hive task fails when left semi join Hello, I am trying to filter out some records in a table in hive. The number of lines in this table is 4billions+, I make a left semi join between above table and a small table with 1k lines. However, after 3 hours job running, it turns out a fail status. My question are as follows, 1. How could I address this problem and final solve it? 2. Is there any other good methods could filter out records with give conditions? The following picture is a snapshot of the failed job.
<<image001.jpg>>
