Few thoughts... Blur.io may also add some new ideas.
I am exploring secondary index designs which is related. Would love to hear more about your goals. On Sep 20, 2012 2:32 PM, "Stack" <[email protected]> wrote: > On Thu, Sep 20, 2012 at 12:43 PM, Andrew Purtell <[email protected]> > wrote: > > The issue with the patch on HBASE-3529 is it relies on modifications > > to HDFS that the author of HBASE-3529 proposed to the HDFS project as > > https://issues.apache.org/jira/browse/HDFS-2004. The proposal was > > vetoed. Therefore, further progress on HBASE-3529 as currently > > implemented is not possible. > > > > Jason's approach had much merit (IMO). It warrants study at least. > > Though the indices were written to HDFS, Jason had it so lucene was > getting local filesystem access by going via the local read > short-circuit facility [1]. Being able to do this made it so he got > close to native speeds querying the "HDFS-based" indices. When Jason > left it -- he had to get a real job unfortunately -- he was blocked on > what to do when a region moved. He wanted to be able to be able to > immediately pull the indices local on region reopen. The HDFS fellas > who commented in the issue cited by Andrew above thought it a little > dodgy adding API for this special case. > > If you wanted to follow in Jasons footsteps, lets chat. > St.Ack > > 1. http://hbase.apache.org/book.html#perf.hdfs.configs >
