hello Chris! (if you are talking about serving language models and/or phrase tables)
i had a student look at using HBase for LMs this summer. i don't think it is sufficiently quick to deal with millions of queries per second, but that may be due to blunders on our part. it may be possible that Hypertable might work for you ... never tried it yet. (i'm also interested in this problem, so we could talk offline about it) Miles 2008/9/18 Chris Dyer <[EMAIL PROTECTED]>: > Hi all- > One more question. > > I'm looking for a lightweight way to serve data stored as key-value > pairs in a series of MapFiles or SequenceFiles. HBase/Hypertable > offer a very robust, powerful solution to this problem with a bunch of > extra features like updates and column types, etc., that I don't need > at all. But, I'm wondering if there might be something > ultra-lightweight that someone has come up with for a very restricted > (but important!) set of use cases. Basically, I'd like to be able to > load the entire contents of a file key-value map file in DFS into > memory across many machines in my cluster so that I can access any of > it with ultra-low latencies. I don't need updates--I just need > ultra-fast queries into a very large hash map (actually, just an array > would be sufficient). This would correspond, approximately to the > "sstable" functionality that BigTable is implemented on top of, but > which is also useful for many, many things directly (refer to the > BigTable paper or > http://www.techworld.com/storage/features/index.cfm?featureid=3183). > > This question may be better targeted to the HBase community, if so, > please let me know. Has anyone else tried to deal with this? > > Thanks-- > Chris > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
