In article mailman.1173.1269496428.23598.python-l...@python.org,
=?GB2312?B?zPC5zw==?= littlesweetme...@gmail.com wrote:
Well, Database is not proper because 1. the table is very big (~10^9
rows) 2. we should support very fast *simple* query that is to get
value corresponding to single key (~10^7
On 3/25/10 4:28 AM, 甜瓜 wrote:
Howdy,
Recently, I am finding a good library for build index on binary data.
Xapian Lucene for python binding focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary data no matter whether
甜瓜 littlesweetme...@gmail.com writes:
Well, Database is not proper because 1. the table is very big (~10^9
rows) 2. we should support very fast *simple* query that is to get
value corresponding to single key (~10^7 queries / second).
Just one numeric key/value pair in each row? What's wrong
Thank you Rubin! Let me have a look at Judy. It seems good at first glance.
--
ShenLei
2010/3/25 Paul Rubin no.em...@nospam.invalid:
甜瓜 littlesweetme...@gmail.com writes:
Well, Database is not proper because 1. the table is very big (~10^9
rows) 2. we should support very fast *simple* query
...@-nospam-xs4all.nl:
On 3/25/10 4:28 AM, 甜瓜 wrote:
Howdy,
Recently, I am finding a good library for build index on binary data.
Xapian Lucene for python binding focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary data
On 25-3-2010 10:55, 甜瓜 wrote:
Thank you irmen. I will take a look at pytable.
FYI, let me explain the case clearly.
Originally, my big data table is simply array of Item:
struct Item
{
long id;// used as key
BYTE payload[LEN]; // corresponding value with fixed length
};
Many thanks for your kind reply. As you mentioned, a sparse array may
be the best choice.
Storing offset rather than payload itself can greatly save memory space.
1e7 queries per second is my ideal aim. But 1e6 must be achieved.
Currently I have implemented 5e6 on one PC (without incremental
甜瓜 wrote:
Well, Database is not proper because 1. the table is very big (~10^9
rows) 2. we should support very fast *simple* query that is to get
value corresponding to single key (~10^7 queries / second).
Ah, crypto rainbow tables.
John Nagle
--
Howdy,
Recently, I am finding a good library for build index on binary data.
Xapian Lucene for python binding focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary data no matter whether it is written in
python?
In my
for indexing binary data no matter whether it is written in
python?
In my case, there is a very big datatable which stores structured
binary data, eg:
struct Item
{
long id; // used as key
double value;
};
I want to build the index on id field to speed on searching. Since
this datatable
focus on text digestion rather than
binary data. Could anyone give me some recommendation? Is there any
library for indexing binary data no matter whether it is written in
python?
In my case, there is a very big datatable which stores structured
binary data, eg:
struct Item
{
long id
11 matches
Mail list logo
