Better yet, it would've seem to be logical if partitioning and advise on partition #s is somehow tailored to a storefunc . It would stand to reason that for as long as we are not storing to hdfs, store func is in the best position to determine optimal save parameters such as order, partitioning and parallelism.
On Mon, Jan 24, 2011 at 11:47 AM, Dmitriy Lyubimov <[email protected]>wrote: > Hi, > > so it seems to be more efficient if storing to hbase partitions by regions > and orders by hbase keys. > > I see that pig 0.8 (pig-282) added custom partitioner in a group but i am > not sure if order is enforced there. > > Is there a way to run single MR that orders and partitions data as per > above and uses an explicitly specifed store func in reducers? > > Thank you. >
