Jagat Hive has a beautiful functionality for doing so, Dynamic Partition Insert. More details: https://cwiki.apache.org/Hive/dynamicpartitions.html
Regards Bejoy KS ________________________________ From: Jagat <[email protected]> To: [email protected] Sent: Friday, April 20, 2012 8:38 PM Subject: How you manage inserts into dynamic partitions Hello Everyone , We have a pig based ETL process and output of those are processed csv files. Now just want to learn about best practices on how you guys insert the data in dynamically generated partitions , do you use some shell script ? or some other utility to do the insertion ? How you handle situations like a csv having data which should go to two separate partitions in hive ? Do you use Pig spilt to do this kind of division or outside pig. Just curious to know best practices to handle insertion into dynamic partitions , when data has been partitioned on time factor ( e,g Month , Week , Day etc) Have a great weekend ahead, Thanks and Regards, Jagat Singh
