Hey Sean, Take a look at https://github.com/apache/phoenix/tree/master/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal
Basically, we use the WAL for storing the mutable index updates. They are carried along with the primary WAL update (as added key values) and then restored independently on WAL replay (filtered at replay and written to the correct table by a phoenix writer). As long as we can do that, should be ok. I was hoping the phoenix codec should be enough, but it gets a little complicated as we do need to manipulate the WAL at Region#batch time too. ------------------- Jesse Yates @jesse_yates jyates.github.com On Fri, Sep 12, 2014 at 9:52 AM, Sean Busbey <[email protected]> wrote: > Hey Folks! > > Can someone give me a few pointers on what Phoenix needs from the HBase > WAL? > > Looks like the docs are all user-facing, so they skip things like how > Phoenix's parts fit into HBase. I'll start looking at code, but I'd rather > hear directly from y'all than just make my own guesses. > > A pointer to a design doc or some JIRAs would be great. > > Background: I'm in the process of refactoring the HBase wal internally > (HBASE-10378). I've run into two major issues. > > 1) I'd like to make some changes to our Coprocessor limited APIs around > WALs. I want to figure out what parts of this I can alter without overly > burdening Phoenix. Even if the changes can't land in HBase 0.98.x I'd like > to get an idea in advance of how they'd impact Phoenix moving to an > eventual HBase 1.0+ > > 2) I've already found some changes I'm planning for > InterfaceAudience.Private classes within HBase that will break at least > current Phoenix tests. This second point makes me very nervous. I'd prefer > not to re-scope all of HBase's WAL stuff to be non-Private. > > -- > Sean >
