On Tuesday, 17 December 2013 at 19:09:49 UTC, H. S. Teoh wrote:
Another OT thread to pick your brains. :)
What's a good, efficient file structure for storing extremely
large
lookup tables? (Extremely large as in > 10 million entries,
with keys
and values roughly about 100 bytes each.) The structure must
support
efficient adding and lookup of entries, as these two operations
will be
very frequent.
What kind of key do you have? vector of floats? integers?
How about some uberfast compression algorithm (eg: LZ4) to
compress pages when you have to offload data to the disk?
