Thanks Stack. o HCM.getConnection(Configuration) is what we would use. The internal list is fine, I don't think that needs to change (at least not for our purposes). Looking at the code it seems the client code could also just create a new HConnectionImplementation.
o I think an ES per HCI would be good enough. Could be passed in, too. o Didn't think creating HTables from HCI, would fly :) Just a crazy idea I had. The flow would be (1) get a Connection to your cluster (2) when you need to access table, ask the connection for a proxy (i.e. an HTable instance). (What's wrong with HCI? :)) OK, I'll work out a simple patch that allows HTable to be optionally created with an externally managed ExecutorService and HConnection instance. That would definitely be good enough for us. -- Lars ________________________________ From: Stack <[email protected]> To: [email protected]; lars hofhansl <[email protected]> Sent: Wednesday, November 16, 2011 2:13 PM Subject: Re: Rethinking the HBase client On Wed, Nov 16, 2011 at 1:53 PM, lars hofhansl <[email protected]> wrote: > The HConnection code itself seems clean, so ideally what we want to do is > managing HConnections. These are what identify and connect us to a cluster, > and should drive everything. > Currently you can't get your hands directly on a particular HConnection instance. You have to do HCM.getConnection(Configuration). Wouldn't you want to change this too -- i.e. not have HCM keep up an internal list? Or at least have the option to by-pass HCM accounting? > HConnectionImplementation could have the thread pool on it that is shared by > the various HTables, and HTables could be created by a method on HConnection. > (Can we give this class a better name than HCI? Smile) You want to ask HCI to create an ExecutorService for batch operations? Would you want to pass it one so one ES for all HCIs in a hosting service? Having HConnection create HTables seems wrong. HTable adds the retrying Callable stuff in it. > So... An alternate (stop-gap) approach is to add a new constructor to HTable > that allows for an optional HConnection and a ThreadPool to be provided. When you say ThreadPool, you mean ExecutorService? The pool instance in HTable? This seems fine to me. > Long term it seems a new client is needed (maybe based on asynchbase, with an > additional synchronous layer), but that is a different story. > Sounds good to me. St.Ack
