Thanks for pointing me out the Jira, that's useful for my understanding. I'm using HBase 0.94.3, and regions of main and index table are co-located on the same RS as in Anoop's design. I'll browse the API tomorrow to find out how to not use HTable but inter-CPs communication.
On Sun, Jan 13, 2013 at 11:04 AM, ramkrishna vasudevan < [email protected]> wrote: > In Anoop's soln its basicallly the put happens directly on the index region > rather than doing a put thro HTable. > > Regards > Ram > > On Sun, Jan 13, 2013 at 9:28 AM, Andrew Purtell <[email protected] > >wrote: > > > Yes, especially if the cross region communication is in process. > > > > On Jan 12, 2013, at 6:48 PM, Ted Yu <[email protected]> wrote: > > > > > bq. Coprocessors are meant to operate on the region to which they are > > > associated. > > > > > > For Anoop's case, the secondary table(s) have their regions aligned > with > > > the corresponding region from primary table. Meaning, related regions > are > > > served by the same region server. > > > Would writes to such regions of secondary table(s) be acceptable ? > > > > > > Thanks > > > > > > On Sat, Jan 12, 2013 at 6:39 PM, Andrew Purtell <[email protected]> > > wrote: > > > > > >>> In pre-put, I trigger another Put() in an external table (to build > the > > >> secondary index). > > >> > > >> We should probably call this a Coprocessor anti-pattern. > > >> > > >> Coprocessors are meant to operate on the region to which they are > > >> associated. They are a way you can extend HBase function while it > > operates > > >> in region on data for the region. Think of them as loadable kernel > > modules. > > >> They are not a general purpose server side platform for programming as > > if > > >> you are building a HBase client (with HTable, etc.). Just because you > > can > > >> do this doesn't mean you should. > > >> > > >> > > >> On Sat, Jan 12, 2013 at 4:06 PM, Adrien Mogenet < > > [email protected] > > >>> wrote: > > >> > > >>> Hi there, > > >>> > > >>> I'm experiencing some issues with CP. I'm trying to implement an > > indexing > > >>> solution (inspired by Annop's slides). In pre-put, I trigger another > > >> Put() > > >>> in an external table (to build the secondary index). It works perfect > > for > > >>> one client, but when I'm inserting data from 2 separate clients, I > met > > >>> issues with HTable object (the one used in pre-Put()), because it's > not > > >>> thread-safe. I decided to move on TablePool and that fixed my issue. > > >>> > > >>> But if I increase the write-load (and concurrency) HBase is throwing > a > > >> OOM > > >>> exception because it can't create new native threads. Looking at > HBase > > >>> metrics "threads count", I see that roughly 3500 threads are created. > > >>> > > >>> I'm looking for documentation about how CPs are working with threads > : > > >>> what/when should I protect against concurrency issues ? How may I > solve > > >> my > > >>> issue ? > > >>> > > >>> Help is welcome :-) > > >>> > > >>> -- > > >>> Adrien Mogenet > > >>> 06.59.16.64.22 > > >>> http://www.mogenet.me > > >> > > >> > > >> > > >> -- > > >> Best regards, > > >> > > >> - Andy > > >> > > >> Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > >> (via Tom White) > > >> > > > -- Adrien Mogenet 06.59.16.64.22 http://www.mogenet.me
