On Thu, Jun 28, 2012 at 1:40 AM, samar kumar <[email protected]> wrote: > Thanks for the replies . I am aware of the apis but can anyone give me > little bit more insight on the details. After creating the HFiles and > calling IncrementalLoadHFile how does it internally change the RS, Catalog > tables etc? > Can anyone explain the flow.
It does not change catalog tables; catalog table only has region layout, not what files make up a region. The loading of an hfile into a region just atomically updates the set of hfiles the currently open region is using swapping in the new one. Study the bulk load tool though. Notice how it will make the proffered set of hfiles fit the current region layout splitting hfiles if necessary so they fit w/i the current region confines. See how it also threads the updating because update is slow done serially. Back up more to see how the files are created using total order partitioner. Leverage what bulk loader does if you want to avoid long solved issues. St.Ack
