The devil is often in the details. On the surface it looks simple. How specifically are the stale indexes ignored? Are the guaranteed to be no races? Is deletion handled correctly?Does it work with multiple versions? What happens when the client dies 1/2 way through an update? It's easy to do eventually consistent indexes. Truly consistent indexes without transactions are tricky.
Also, scanning an index and then doing point-gets against a main table is slow (unless the index is very selective. The Phoenix team measured that there is only an advantage if the index filters out 98-99% of the data). So then one would revert to covered indexes and suddenly is not so easy to detect stale index entries. I blogged about these issues here: http://hadoop-hbase.blogspot.com/2012/10/musings-on-secondary-indexes.html http://hadoop-hbase.blogspot.com/2012/10/secondary-indexes-part-ii.html Phoenix has a (pretty involved) solution now that works around the fact that HBase has no transactions. -- Lars ________________________________ From: Henning Blohm <henning.bl...@zfabrik.de> To: user <user@hbase.apache.org> Sent: Sunday, December 22, 2013 2:11 AM Subject: secondary index feature Lately we have added a secondary index feature to a persistence tier over HBASE. Essentially we implemented what is described as "Dual-Write Secondary Index" in http://hbase.apache.org/book/secondary.indexes.html. I.e. while updating an entity, actually before writing the actual update, indexes are updated. Lookup via the index ignores stale entries. A recurring rebuild and clean out of stale entries takes care the indexes are trimmed and accurate. None of this was terribly complex to implement. In fact, it seemed like something you could do generically, maybe not on the HBASE level itself, but as a toolbox / utility style library. Is anybody on the list aware of anything useful already existing in that space? Thanks, Henning Blohm *ZFabrik Software KG* T: +49 6227 3984255 F: +49 6227 3984254 M: +49 1781891820 Lammstrasse 2 69190 Walldorf henning.bl...@zfabrik.de <mailto:henning.bl...@zfabrik.de> Linkedin <http://www.linkedin.com/pub/henning-blohm/0/7b5/628> ZFabrik <http://www.zfabrik.de> Blog <http://www.z2-environment.net/blog> Z2-Environment <http://www.z2-environment.eu> Z2 Wiki <http://redmine.z2-environment.net>
