Hi Vinod, Do you use remote server configuration ? What version of hive do you use ? We are using 0.7.1 where we see this issue.
Sent from my iPhone On Aug 11, 2012, at 12:39 AM, Vinod Singh <[email protected]> wrote: > We run Hive jobs on 20+ TB data without any issues. > > Thanks, > Vinod > > On Sat, Aug 11, 2012 at 9:07 AM, Anurag Tangri <[email protected]> > wrote: > Hi, > We are facing this issue where we run a hive job over huge data about ~6 TB > input. > > We run this from hive client and hive metastore server is on another machine. > > > If we have smaller input, this job succeeds but for above input size, it > fails with error : > > 2012-08-11 01:34:01,722 Stage-1 map = 100%, reduce = 100% > > 2012-08-11 01:35:02,195 Stage-1 map = 100%, reduce = 100% > > 2012-08-11 01:36:02,682 Stage-1 map = 100%, reduce = 100% > > 2012-08-11 01:37:03,215 Stage-1 map = 100%, reduce = 100% > > 2012-08-11 01:38:03,719 Stage-1 map = 100%, reduce = 100% > > 2012-08-11 01:39:04,311 Stage-1 map = 100%, reduce = 100% > > Ended Job = job_201207072204_34432 > > Loading data to table default.atangri_test_1 > > Failed with exception Unable to fetch table atangri_test_1 > > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.MoveTask > > > > If we have smaller input (~2 TB), this job succeeds but for above input size, > it fails with error : We have set hive.metastore.client.socket.timeout to big > value like 86400 but still it fails after about 8-9 hours. > > Does anyone face the same issue or any pointers ? > > The job succeeds if it is directly run on hive server. > > Thanks, > Anurag Tangri >
