I'm not sure how an alternative architecture would look, where there'd be multiple replicated indexes pointing at only one region? What if that region went down, that's a single point of failure? It's far easier to simply keep the region n'sync with an attached index. Than to replicate the data from multiple regions to an index on another server? That sounds extremely redundant and problematic, eg, all of the database transactional functionality probably goes out the window. One can easily build a multi-tier batch indexing application today with HBase and Lucene, though it'd still need to scale [somehow] and with realtime search all of these issues encroach again.
On Mon, Feb 14, 2011 at 12:20 PM, Ted Dunning <tdunn...@maprtech.com> wrote: > There is already going to be a serious imbalance because the number of > indexes is highly unlikely to be the same as the number of regions in an > optimal setup. > > On Mon, Feb 14, 2011 at 11:21 AM, Bruno Dumon <br...@outerthought.org>wrote: > >> One option might be to introduce replication only for the indexes, and >> leave >> the regions as they are today, at the cost of some imbalance in the design >> (meaning, hbase master would need to be aware of the two different >> concepts). >> >> On Mon, Feb 14, 2011 at 7:55 PM, Ted Dunning <tdunn...@maprtech.com> >> wrote: >> >> > Fixing this is likely quite difficult since it requires distributed >> > transactions. It would also typically kill update performance because of >> > the distributed transaction problem. >> > >> > On Mon, Feb 14, 2011 at 9:48 AM, Jason Rutherglen < >> > jason.rutherg...@gmail.com> wrote: >> > >> > > > Yep, really. The replication is [only] on the HDFS-level. For HBase, >> > this >> > > is >> > > > not much of a problem as long as the requests are not strongly skewed >> > > > towards one region (this requires good consideration from users when >> > > > choosing row keys), but for search this could be a real issue. >> > > >> > > I think this can be solved rather easily? Or is there an underlying >> > > design rationale? >> > >> >> >> >> -- >> Bruno Dumon >> Outerthought >> http://outerthought.org/ >> >
