Actually, I wasn't clear earlier... we are currently using this syntax for loading data into the table/partition:
INSERT OVERWRITE TABLE ourtable PARTITION(dt='2010-03-16') ... If I execute this multiple times, I believe the data will simply be overwritten instead of appended, right? On Wed, Mar 17, 2010 at 4:01 PM, Ryan LeCompte <[email protected]> wrote: > Awesome! I didn't know this. :) I'll get it a shot, thanks! > > > > On Wed, Mar 17, 2010 at 3:57 PM, Edward Capriolo <[email protected]>wrote: > >> >> >> On Wed, Mar 17, 2010 at 3:30 PM, Ryan LeCompte <[email protected]>wrote: >> >>> Hello all, >>> >>> Is it possible in Hive 0.5 to run multiple inserts into the same Hive >>> table/partition? Or is this not supported due to the fact that Hadoop >>> doesn't support appends properly? >>> >>> For example, it would be nice to periodically add new data every 5 >>> minutes to a table that has a partition column for "date" via multiple >>> periodic INSERT statements. >>> >>> Thanks! >>> >>> Ryan >>> >>> Ryan, >> >> Every file inside the partition makes up the partiion. So with 'LOAD DATA >> INFILE (X)', if X is a unique name it will be "appended". >> >> This works for us since our 5 minute log files all have unique names . >> >> Edward >> > >
