My idea is to have one bit index per logical column+value stored in a value where the key also have some means of partitioning the data, maybe over time. So high cardinality columns will generate lots of keys+values.
I was thinking of storing multiple indexes in each key+value, making values bigger but fewer. Say around a few hundreds kilobytes each. Are there any trade offs between many smaller key+value vs fewer larger ones? I'm more concerned about read performance. On Sun, Jul 12, 2015 at 12:17 PM, Howard Chu <[email protected]> wrote: > Kristoffer Sjögren wrote: >> >> Hi >> >> I'm doing some experiments with LMDB trying to emulate a columnar >> storage database using roaring bitmaps and other tricks. >> >> The initial results are promising, but I ask myself, is a row based >> storage like LMDB appropriate for implementing a columnar database or >> are there better, more efficient ways/formats? > > > What makes you think LMDB is either row- or column-based? It has no concept > of either, that's purely an abstraction created by higher level code. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/
