2011/8/9 David Garfield :
> Having said that, let me present a database for consideration: Any
> filesystem. Split the hex of the MD5 into directory levels and make
> what you need. Might be slower, particularly with some OSes, but the
> tools are easy.
> --David
Richard Hipp writes:
> This is a locality of reference problem. The caching mechanisms (both in
> SQLite and in the filesystem of your computer) begins to break down when the
> size of the database exceeds available RAM. And when the cache stops
> working well, you have to wait on physical I/O
2011/8/9 Jaco Breitenbach :
> I am building an application that filters out duplicate input data by
> generating an MD5 hash of each input, and implicitly comparing that against
> a set of keys already stored in the SQLite database by doing an insert into
> a
If non-GPL is a firm requirement, you might also look at CouchDB (
http://couchdb.org).
On Tue, Aug 9, 2011 at 07:02, gabriel.b...@gmail.com wrote:
> i would sugest Mongo db
>
> just use it from its binaries packages and don't worrie
>
> like it says
> If you are using
On 9 Aug 2011, at 2:27pm, Jaco Breitenbach wrote:
> The problem that I'm facing, is that I would ultimately need to process
> 1,000,000,000 records a day, with history to be kept for up to 128 days. I
> am currently creating a new data file per day, with hourly tables. However,
> that will
On Tue, Aug 9, 2011 at 9:27 AM, Jaco Breitenbach wrote:
> Unfortunately the performance rate of the inserts
> into the indexed tables decreases significantly as the number of records in
> the tables increases. This seems to be because of a CPU bottleneck rather
> than
i would sugest Mongo db
just use it from its binaries packages and don't worrie
like it says
If you are using a vanilla MongoDB server from either source or binary
packages you have NO obligations. You can ignore the rest of this page.
http://www.mongodb.org/display/DOCS/Licensing
2011/8/9
enbach [jjbreitenb...@gmail.com]
Sent: Tuesday, August 09, 2011 8:51 AM
To: General Discussion of SQLite Database
Subject: EXT :Re: [sqlite] Maximum number of tables in a data file
Hi Gabriel,
Is there such a database that is both free and non-GPL that you can
recommend?
Jaco
On 9 August 2011
Hi Gabriel,
Is there such a database that is both free and non-GPL that you can
recommend?
Jaco
On 9 August 2011 14:38, gabriel.b...@gmail.com wrote:
> Heve you ever considere using a NOSQL database I think it would serve you
> better
>
> 2011/8/9 Jaco Breitenbach
Yes, but each input record also contains a timestamp that can be used to
identify the relevant table.
On 9 August 2011 14:43, Igor Tandetnik wrote:
> Jaco Breitenbach wrote:
> > I am building an application that filters out duplicate input data by
Jaco Breitenbach wrote:
> I am building an application that filters out duplicate input data by
> generating an MD5 hash of each input, and implicitly comparing that against
> a set of keys already stored in the SQLite database by doing an insert into
> a unique-indexed
Heve you ever considere using a NOSQL database I think it would serve you
better
2011/8/9 Jaco Breitenbach
> Hi Igor and Michael,
>
> Yes, of course, 1440 minutes in a day. :-)
>
> I am building an application that filters out duplicate input data by
> generating an MD5
Hi Igor and Michael,
Yes, of course, 1440 minutes in a day. :-)
I am building an application that filters out duplicate input data by
generating an MD5 hash of each input, and implicitly comparing that against
a set of keys already stored in the SQLite database by doing an insert into
a
Jaco Breitenbach wrote:
> Can anyone please tell me if there is a limit to the number of tables that
> can be held in a single data file? I am considering an application that
> will require a table for every minute in a day, i.e. 3600+ tables in a
> single database or
09, 2011 8:05 AM
To: General Discussion of SQLite Database
Subject: EXT :[sqlite] Maximum number of tables in a data file
Dear experts,
Can anyone please tell me if there is a limit to the number of tables that
can be held in a single data file? I am considering an application that
will require
Dear experts,
Can anyone please tell me if there is a limit to the number of tables that
can be held in a single data file? I am considering an application that
will require a table for every minute in a day, i.e. 3600+ tables in a
single database or data file.
Regards,
Jaco
16 matches
Mail list logo