Seems to be fixed in 0.6. Here's what I got: test.q: alter table tmp_pyang_t ADD PARTITION (ds='2') LOCATION '/tmp/blah2'; alter table tmp_pyang_t ADD PARTITION (ds='3') LOCATION '/tmp/blah2'; alter table tmp_pyang_t ADD PARTITION (ds='4') LOCATION '/tmp/blah2'; alter table tmp_pyang_t ADD PARTITION (ds='5') LOCATION '/tmp/blah2'; alter table tmp_pyang_t ADD PARTITION (ds='6') LOCATION '/tmp/blah2'; alter table tmp_pyang_t ADD PARTITION (ds='7') LOCATION '/tmp/blah2'; alter table tmp_pyang_t ADD PARTITION (ds='8') LOCATION '/tmp/blah2';
Hive history file=/tmp/pyang/hive_job_log_pyang_201004081410_378771152.txt OK Time taken: 4.101 seconds OK Time taken: 0.558 seconds OK Time taken: 0.453 seconds OK Time taken: 0.416 seconds OK Time taken: 0.378 seconds OK Time taken: 0.457 seconds OK Time taken: 0.454 seconds Can you the stack trace in /tmp/<username>/hive.log? -----Original Message----- From: Prasad Chakka [mailto:[email protected]] Sent: Thursday, April 08, 2010 1:03 PM To: [email protected] Subject: Re: enough alter tables in the same .q file eventually fail There was a bug that got fixed where each request was creating a separate metastore client. That could be it or something similar that hasn't gotten fixed. On Apr 8, 2010, at 11:47 AM, Edward Capriolo wrote: > Hive 5.0 mysql as metastore backend. Using external tables with location for > partitions... > > alter table XXXX_action ADD PARTITION (hit_date = '20100329' , mid = > '000843') LOCATION 'hit_date=20100329/mid=000843'; > alter table XXXX_action ADD PARTITION (hit_date = '20100329' , mid = > '000844') LOCATION 'hit_date=20100329/mid=000844'; > alter table XXXX_action ADD PARTITION (hit_date = '20100329' , mid = > '000849') LOCATION 'hit_date=20100329/mid=000849'; > alter table XXXX_action ADD PARTITION (hit_date = '20100329' , mid = > '000850') LOCATION 'hit_date=20100329/mid=000850'; > alter table XXXX_action ADD PARTITION (hit_date = '20100329' , mid = > '000851') LOCATION 'hit_date=20100329/mid=000851'; > alter table XXXX_action ADD PARTITION (hit_date = '20100329' , mid = > '000852') LOCATION 'hit_date=20100329/mid=000852'; > > Eventually this fails after a number of entries. > > Time taken: 0.159 seconds > OK > Time taken: 0.17 seconds > OK > Time taken: 0.241 seconds > FAILED: Error in metadata: Unable to fetch table XXXXX_action > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.DDLTask > > Restarting the process after removing the already added tables works until it > breaks again. Anyone ever dealt with this? > > Doing one hive -e per table always works but takes a lot longer ...3 seconds > a partition rather then ~.5 seconds. > >
